telefonicaid / perseo-core Goto Github PK
View Code? Open in Web Editor NEWCore/backend for the Perseo CEP
License: GNU General Public License v2.0
Core/backend for the Perseo CEP
License: GNU General Public License v2.0
Include the following fields in log traces, as per request from Operations.
Then the same rulename could not be used by different service and subservices
PUT in
GET in
Generate usage examples. Some may be found at the popper PR, but some more exploiting "from pattern interval" could be useful.
From [email protected] on 02/08/2023
You are receving this email because your Read the Docs project is impacted by an upcoming deprecation.
Historically, Read the Docs has installed MkDocs 0.17.3 by default if the project didn't define a specific version of it. Now, this version is a pretty old and we recommend our users to use the latest version of MkDocs: 1.5.1.
To avoid failing builds we kept the old default version of MkDocs. We are changing that behavior to always install the latest available version and we wanted to let you know about that so you can prepare your project.
Starting on October 3rd Read the Docs will install the latest available version of MkDocs. If you want to use a different version, make sure you are pinning your MkDocs dependency.
Here you have a pretty simple example of the section required on the .readthedocs.yaml configuration file:
python:
install:
- requirements: docs/requirements.txt
The content of docs/requirements.txt would be similar to:
mkdocs==1.5.1
We are sending you this email because you are a maintainer of the following projects that are affected by this removal:
• fiware-orion
• fiware-perseo-core
Read more about this in our Reproducible builds guide for more details.
Perseo-core is using:
https://logging.apache.org/log4j/1.2/
Security Vulnerabilities
A security vulnerability, CVE-2019-17571 has been identified against Log4j 1. Log4j includes a SocketServer that accepts serialized log events and deserializes them without verifying whether the objects are allowed or not. This can provide an attack vector that can be expoited. Since Log4j 1 is no longer maintained this issue will not be fixed. Users are urged to upgrade to Log4j 2.
Currently perseo core logs into a file which outputs through a tail. It should log directly to stdout.
cast(cast(status_datetime?,string),date, dateformat: 'yyyy-MM-dd HH:mm:ss').format("yyyy-MM-dd'T'HH:mm:ss.SSSX") as statusdatetime
time=2022-06-29T09:34:52.668Z | lvl=ERROR | from=::ffff:172.17.0.16 |
corr=85f76744-5ba6-4080-ad44-7c7f5d4434bb; cbnotif=3; node=mSox4tLGxT;
perseocep=132 | trans=b7a07260-91cb-40cf-82ed-e9b0a7abc962 |
srv=smartcity | subsrv=/CEP | op=handleException | comp=perseo-core |
msg=Exception encountered processing deployment-id '91c27686-4208-4b0d-
bb5c-8af42c14f89d' statement ......
java.lang.NullPointerException: null
at
com.espertech.esper.common.internal.view.core.ZeroDepthStreamNoIterate.
insert(ZeroDepthStreamNoIterate.java:41) ~[esper-common-8.4.0.jar:?]
at
com.espertech.esper.common.internal.context.activator.ViewableActivator
Filter$1.matchFound(ViewableActivatorFilter.java:76) ~[esper-common-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processStatementFilterSingle(EPEventServiceImpl.java:796) ~[esper-
runtime-8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processMatches(EPEventServiceImpl.java:675) ~[esper-runtime-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processWrappedEvent(EPEventServiceImpl.java:453) ~[esper-runtime-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
sendEventMap(EPEventServiceImpl.java:395) ~[esper-runtime-8.4.0.jar:?]
at
com.telefonica.iot.perseo.EventsServlet.doPost(EventsServlet.java:91)
~[classes/:?]
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:681) ~[servlet-
api.jar:4.0.FR]
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:764) ~[servlet-
api.jar:4.0.FR]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
ationFilterChain.java:227) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
terChain.java:162) ~[catalina.jar:9.0.64]
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
~[tomcat-websocket.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
ationFilterChain.java:189) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
terChain.java:162) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVal
ve.java:197) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextVal
ve.java:97) ~[catalina.jar:9.0.64]
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticato
rBase.java:541) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav
a:135) ~[catalina.jar:9.0.64]
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.jav
a:92) ~[catalina.jar:9.0.64]
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccess
LogValve.java:687) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve
.java:78) ~[catalina.jar:9.0.64]
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
360) ~[catalina.jar:9.0.64]
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:3
99) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight
.java:65) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPr
otocol.java:890) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoin
t.java:1787) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.
java:49) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolE
xecutor.java:1191) ~[tomcat-util.jar:9.0.64]
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPool
Executor.java:659) ~[tomcat-util.jar:9.0.64]
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThre
ad.java:61) ~[tomcat-util.jar:9.0.64]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
and
java.lang.NumberFormatException: For input string: ".E367E22"
at
jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.
java:2054) ~[?:?]
at
jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
~[?:?]
at java.lang.Double.parseDouble(Double.java:543) ~[?:?]
at java.text.DigitList.getDouble(DigitList.java:169) ~[?:?]
at java.text.DecimalFormat.parse(DecimalFormat.java:2126)
~[?:?]
at
java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2240) ~[?:?]
at
java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1541) ~[?:?]
at java.text.DateFormat.parse(DateFormat.java:393) ~[?:?]
at
com.espertech.esper.common.internal.epl.expression.funcs.ExprCastNode$S
tringToDateWStaticFormatComputer.stringToDateWStaticFormatParseSafe(Exp
rCastNode.java:501) ~[esper-common-8.4.0.jar:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.m20(Sele
ctExprProcessorImpl.java:496) ~[?:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.m19(Sele
ctExprProcessorImpl.java:486) ~[?:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.m1(Selec
tExprProcessorImpl.java:345) ~[?:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.m0(Selec
tExprProcessorImpl.java:327) ~[?:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.process(
SelectExprProcessorImpl.java:317) ~[?:?]
at
com.espertech.esper.common.internal.epl.resultset.handthru.ResultSetPro
cessorHandThroughUtil.getSelectEventsNoHavingHandThruView(ResultSetProc
essorHandThroughUtil.java:45) ~[esper-common-8.4.0.jar:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$RSP.processViewResult(RSP.java:1
31) ~[?:?]
at
generated.OutputProcessViewFactoryProvider_3bdadcdf45383c454b0545a45945
ea037a4d1438_ruleTest64smartcity47CEP$OPV.update(OPV.java:65) ~[?:?]
at
com.espertech.esper.common.internal.view.core.ZeroDepthStreamNoIterate.
insert(ZeroDepthStreamNoIterate.java:41) ~[esper-common-8.4.0.jar:?]
at
com.espertech.esper.common.internal.context.activator.ViewableActivator
Filter$1.matchFound(ViewableActivatorFilter.java:76) ~[esper-common-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processStatementFilterSingle(EPEventServiceImpl.java:796) ~[esper-
runtime-8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processMatches(EPEventServiceImpl.java:675) ~[esper-runtime-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processWrappedEvent(EPEventServiceImpl.java:453) ~[esper-runtime-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
sendEventMap(EPEventServiceImpl.java:395) ~[esper-runtime-8.4.0.jar:?]
at
com.telefonica.iot.perseo.EventsServlet.doPost(EventsServlet.java:91)
~[classes/:?]
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:681) ~[servlet-
api.jar:4.0.FR]
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:764) ~[servlet-
api.jar:4.0.FR]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
ationFilterChain.java:227) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
terChain.java:162) ~[catalina.jar:9.0.64]
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
~[tomcat-websocket.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
ationFilterChain.java:189) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
terChain.java:162) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVal
ve.java:197) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextVal
ve.java:97) ~[catalina.jar:9.0.64]
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticato
rBase.java:541) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav
a:135) ~[catalina.jar:9.0.64]
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.jav
a:92) ~[catalina.jar:9.0.64]
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccess
LogValve.java:687) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve
.java:78) ~[catalina.jar:9.0.64]
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
360) ~[catalina.jar:9.0.64]
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:3
99) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight
.java:65) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPr
otocol.java:890) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoin
t.java:1787) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.
java:49) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolE
xecutor.java:1191) ~[tomcat-util.jar:9.0.64]
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPool
Executor.java:659) ~[tomcat-util.jar:9.0.64]
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThre
ad.java:61) ~[tomcat-util.jar:9.0.64]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Currently there is no way to change log level without use API.
Currently default log level is INFO
A docker en var to set a a different log level would be desirable.
(Rasied in #151 (comment))
It seems that the checkstyle pass (based in telefonica_checkstyle.xml ruleset) is not working. Although the pom.xml is supposedly configured to use it, it seems that mvn checkstyle:check
uses the default sun_checks.xml ruleset.
I did an attemp of fixing it, but not sure if it's ok. I'm including it below, it could be a good starting point.
diff --git a/pom.xml b/pom.xml
index a210429..d8bcaea 100644
--- a/pom.xml
+++ b/pom.xml
@@ -41,7 +41,8 @@
**/src/main/java/org/json/**, org/json/**
</sonar.exclusions>
<sonar.language>java</sonar.language>
- <sonar.dynamic>reuseReports</sonar.dynamic>
+ <sonar.dynamic>reuseReports</sonar.dynamic>
+ <checkstyle.config.location>telefonica_checkstyle.xml</checkstyle.config.location>
</properties>
<build>
@@ -121,6 +122,11 @@
<configuration>
</configuration>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-checkstyle-plugin</artifactId>
+ <version>3.0.0</version>
+ </plugin>
</plugins>
</build>
<reporting>
@@ -146,9 +152,7 @@
<artifactId>maven-checkstyle-plugin</artifactId>
<version>3.0.0</version>
<configuration>
- <configLocation>telefonica_checkstyle.xml</configLocation>
- <includeTestSourceDirectory>false</includeTestSourceDirectory>
- <failOnViolation>false</failOnViolation>
+ <configLocation>telefonica_checkstyle.xml</configLocation>
</configuration>
</plugin>
<plugin>
diff --git a/telefonica_checkstyle.xml b/telefonica_checkstyle.xml
index 5a6539e..1dfab02 100644
--- a/telefonica_checkstyle.xml
+++ b/telefonica_checkstyle.xml
@@ -196,7 +196,7 @@
<module name="MagicNumber"/>
-->
<module name="MissingSwitchDefault"/>
- <module name="RedundantThrows"/>
+ <!--module name="RedundantThrows"/-->
<module name="SimplifyBooleanExpression"/>
<module name="SimplifyBooleanReturn"/>
@@ -229,4 +229,4 @@
</module>
</module>
-</module>
\ No newline at end of file
+</module>
Ideally, checkstyle should be included in .travis.yml.
a rule like
{
"name": "entityupdate",
"text": "select *, 27 as numero, false as bool, null as nulo from iotEvent where type?='WaterTank'",
"action": {
"type": "update",
"parameters": {
"id": "TestEntity2",
"type": "TestEntity",
"attributes": [
{
"name": "numero",
"value": "${numero}",
"type": "Number"
},
{
"name": "json",
"value": "[${nulo},1,2]",
"type": "Object"
},
{
"name": "nulo",
"value": "${nulo}",
"type": "Object"
},
{
"name": "bool",
"value": "${bool}",
"type": "Boolean"
}
]
}
}
}
is updating an entity in a bad way:
{
"id": "TestEntity2",
"type": "TestEntity",
"bool": {
"type": "Boolean",
"value": false,
"metadata": {}
},
"json": {
"type": "Object",
"value": "[null,1,2]",
"metadata": {}
},
"nulo": {
"type": "Object",
"value": "null",
"metadata": {}
},
"numero": {
"type": "Number",
"value": 27,
"metadata": {}
}
}
This another rule:
{
"name": "entityupdate",
"text": "select *, 27 as numero, false as bool, null as nulo from iotEvent where type?='WaterTank'",
"action": {
"type": "update",
"parameters": {
"id": "TestEntity2",
"type": "TestEntity",
"attributes": [
{
"name": "numero",
"value": "${numero}",
"type": "Number"
},
{
"name": "json",
"value": "[${numero},1,2]",
"type": "Object"
},
{
"name": "nulo",
"value": "${nulo}",
"type": "Object"
},
{
"name": "bool",
"value": "${bool}",
"type": "Boolean"
}
]
}
}
}
is updating entity:
{
"id": "TestEntity2",
"type": "TestEntity",
"bool": {
"type": "Boolean",
"value": false,
"metadata": {}
},
"json": {
"type": "Object",
"value": [
27,
1,
2
],
"metadata": {}
},
"nulo": {
"type": "Object",
"value": "null",
"metadata": {}
},
"numero": {
"type": "Number",
"value": 27,
"metadata": {}
}
}
at java.lang.Thread.run(Thread.java:829) ~[?:?]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-util.jar:9.0.62]
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-util.jar:9.0.62]
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-util.jar:9.0.62]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-coyote.jar:9.0.62]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743) ~[tomcat-coyote.jar:9.0.62]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890) ~[tomcat-coyote.jar:9.0.62]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat-coyote.jar:9.0.62]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) ~[tomcat-coyote.jar:9.0.62]
Hi,
we would like to use perseo in conjunction with orion-ld. However, perseo only supports ngsi-v2. When is NGSI-LD support planned?
regards,
rob
It's mandatory to allow the usage of astronomic clock concepts in Perseo time rules. Mainly Sunset and Sunrise based on LAT/LOG values.
I suppose this could be achieved by adding the proper Java Library to Esper deployment.
Maybe related #164
Use some conf like:
https://github.com/telefonicaid/fiware-cygnus/blob/master/cygnus-common/conf/log4j.properties.template
Add field from
in log traces
Implement the admin API to get current log level. This is:
GET /admin/log
{
"level": "INFO"
}
In addition, it has to be included in the documented, along with the PUT /admin/log?level=XXX
op.
Note that level is returned in UPPPERCASE.
The responses should be encoded in UTF-8
Effort: 2 man day
Remove this, it is wrong:
Software that merely uses or aggregates (i.e. links to) an otherwise unmodified version of existing software is not considered a derivative work, and therefore it does not need to be released as under the same license, or even released as open source.
More information, especially with incoming and outgoing requests
Effort: 2 man day
When the json sent to CEP is too long, the perseo-core fails parsing the json
{"error":"error post to http://10.95.82.36:8080/perseo-core/rules (Unterminated string at 2897 [character 2898 line 1])","data":null}
In addition to a fix in the code to check JSON length and report an error, the "too long" should defined (and documented) as part of the resolution of this bug.
Running mvn package
causes the following if the file /var/log/perseo/perseo-core.log doesn't exist:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running es.tid.fiware.perseo.GenericListenerTest
update
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /var/log/perseo/perseo-core.log (No existe el fichero o el directorio)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at java.io.FileOutputStream.<init>(FileOutputStream.java:136)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:295)
at org.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:176)
at org.apache.log4j.xml.DOMConfigurator.findAppenderByReference(DOMConfigurator.java:191)
at org.apache.log4j.xml.DOMConfigurator.parseChildrenOfLoggerElement(DOMConfigurator.java:523)
at org.apache.log4j.xml.DOMConfigurator.parseCategory(DOMConfigurator.java:436)
at org.apache.log4j.xml.DOMConfigurator.parse(DOMConfigurator.java:999)
at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:867)
at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:773)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254)
at es.tid.fiware.perseo.GenericListener.<clinit>(GenericListener.java:36)
at es.tid.fiware.perseo.GenericListenerTest.testUpdate(GenericListenerTest.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
Curiosly, the test runs without fail.
Thi excepction should be avoided. Maybe it is just a matter of adjusting the logger for the unit test target in order not using logging file or a logging file mock.
API to implement:
POST /admin/log?level=info
The value of the "level" parameter is interpreted in case insensitive way and the valid values are:
FATAL, ERROR, WARNING, INFO, DEBUG
Result in the happy case:
200 OK (without payload)
Result in the case of error:
400 Bad Request
{
"error": "not supported log level"
}
I've been using perseo project recently and i've tested some of its functionalities through the provided examples in Github . i have deployed perse-core as a docker container and perseo-fe source using npm.
i configured the perseo-fe along with the perseo-core container and i made sure they communicate with each other by using the examples in https://github.com/telefonicaid/perseo-fe/tree/master/examples . But then i began to use a new rule including the timer:at() function so that the perse-core fires an action at a specific time.
then i checked the perseo-core logs and made sure it fired an action but there was failure in communication between it and the perseo-fe as follows .
and i don't know what is the problem with perseo-fe and why it gives this error
i checked the perseo-core separately with a socket server listening on port 6000 and made the core fire towards it instead of 9090 and i made sure the perseo-core fires on time but i also found another problem where it fired only the values involved in the select clause and it didn't fire the json body included in the action field .
So there are two problems
1- the perseo-fe has a problem on receiving actions from the perseo-core with the timer:at() function
2-the perseo-core fires the values of the select clause and doesnot include the json included action field.
I hope there would be a solution for these issues @fgalan @dmoranj .
# This file should be copied by deployment process
# into /etc/perseo-core.properties, with the appropiate permissions
# URL for invoking actions when a rule is fired
action.url = http://<PERSEO_FE_URL>/actions/do
# Time in milliseconds (long) to "expire" a "dangling" rule
rule.max_age= 60000
Allow use Map instead of java.util.Map in EPL text
For java.util.Map, Collection, Object
The following MUST requirements for full GE status are not currently satisfied:
Missing Badges:
Missing items within the README
Missing on ReadtheDocs Documentation
No CREDITS file
An incubated enabler should be working to fulfilling these requirements
As the issue title says
I've tried to use a script expression in a rule, using two proposed esper syntax, but both are seen with incorrect syntax by perseo-core esper engine:
time=2018-03-14T13:13:15.842Z | lvl=ERROR | corr=n/a | trans=n/a | op=refreshCore | comp=perseo-fe | srv=n/a | subsrv=n/a | msg=[object Object] error put to http://perseo-core:8080/perseo-core/rules (Incorrect syntax near 'expression' at line 1 column 35 [context ctxt$testservice$testalain expression String js:trunc(name)[trunc(name); function trunc(n) {return n.substr(n.indexOf(":" + 1));}] select *,"my_rule_update_tto" as ruleName, *,cast(cast(ev.temperature?, String),float) + 1.0 as temp, trunc(cast(ev.id?, String)) as iddev from pattern [every ev=iotEvent(cast(cast(temperature?,String),float)<5.0 and type="thing")]])
time=2018-03-14T13:23:15.875Z | lvl=ERROR | corr=n/a | trans=n/a | op=refreshCore | comp=perseo-fe | srv=n/a | subsrv=n/a | msg=[object Object] error put to http://perseo-core:8080/perseo-core/rules (Incorrect syntax near 'expression' at line 1 column 35 [context ctxt$testservice$testalain expression String js:trunc(name)[name.substr(name.indexOf(":" + 1));] select *,"my_rule_update_tto" as ruleName, *,cast(cast(ev.temperature?, String),float) + 1.0 as temp, trunc(cast(ev.id?, String)) as iddev from pattern [every ev=iotEvent(cast(cast(temperature?,String),float)<5.0 and type="thing")]])
the used esper version (5.5.0) support script expression, any idea of the issue with the syntax ?
or is it a script expression support issue ?
Thanks in advance for help or feedback on this problem
Fix perseo-core as a rpm owner logs
In spite the modifications introduced in #102 aimted to show the Perseo log instead of Catalina log (from Tomcat) in docker container it seems it is not working:
$ sudo docker logs $(sudo docker ps | grep perseo-core | awk -F ' ' '{print $1}')
...
29-Oct-2018 01:38:48.696 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/usr/local/tomcat/webapps/perseo-core.war] has finished in [3,283] ms
29-Oct-2018 01:38:48.704 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
29-Oct-2018 01:38:48.725 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
29-Oct-2018 01:38:48.746 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3416 ms
CC: @cblanco just in case you know and easy solution for this :)
Create a version request (http) or message in log to Perseo-core.
examples:
http://host:port/version
In the case of receiving a transaction ID (in the incoming HTTP request in a given header), it has to be used (an propagated to any outgoing HTTP request derived from such incoming HTTP request) instead of auto-generating one.
If no transaction ID if provided in the HTTP request then CEP Core has to auto-generate one and propagate it to any outgoing HTTP request derived from that transaction.
Details about which particular header to use for transaction ID are under discussion in the moment of writing this.
Since Orion-LD supports NGSI-V2 out-of-the-box we should be able to use the Perseo services without any trouble.
After having the broker-perseo subscription setup and receiving the event in the perseo-core component, I've noticed that the DateTime
pseudo-attributes do not get generated.
Incoming message in perseo-core
{
"noticeId": "5ae4e520-5824-11ec-9ba9-a3d9baac0822",
"noticeTS": 1638967370866,
"id": "urn:ngsi-ld:Entity:001",
"type": "Delivery",
"isPattern": false,
"subservice": "/",
"service": "unknownt",
"requestedTimeslot__type": "Property",
"requestedTimeslot": "2014-04-29T13:18:05Z",
...
Expected incoming message in perseo-core
to have parsed pseudo-attributes for the DateTime
property :
...
"requestedTimeslot": "2014-04-29T13:18:05Z",
"requestedTimeslot__ts": 1398777485000,
"requestedTimeslot__day": 29,
...
The notification was triggered by the following request to the context broker (ngsi-ld interface). Mind the DateTime notation according to "EXAMPLE OF MIGRATION TO NGSI-LD" in NGSI-LD docs
curl --location --request PATCH 'http://orion:1026/ngsi-ld/v1/entities/urn:ngsi-Entity:001/attrs' \
--header 'Content-Type: application/ld+json' \
--data-raw '{
"requestedTimeslot": {
"value": {
"@type": "DateTime",
"@value": "2014-04-29T13:18:05Z"
},
"type": "Property"
},
}'
Fix null values in rotate logs using RPMs
Currently we are using as Java package name:
es.tid.fiware.perseo;
It will be desirable to align this with Sonar naming, thus it could be:
com.telefonica.iot.perseo;
Effort: 1 man day
https://esper.espertech.com/release-8.4.0/reference-esper/html_single/#apiruntime-time-internal
[17.6.1.4. Internal Timer Settings]()
This option can be used to disable the internal timer thread and such have the application supply external time events, as well as to set a timer resolution.
The next code snippet shows how to disable the internal timer thread via the configuration API:
[Configuration config = new Configuration();
config.getRuntime().getThreading().setInternalTimerEnabled(false);]()
This snippet of XML configuration leaves the internal timer enabled (the default) and sets a resolution of 200 milliseconds (the default is 100 milliseconds):
[<esper-configuration xmlns="http://www.espertech.com/schema/esper">
<runtime>
<threading>
<internal-timer enabled="true" msec-resolution="200"/>
</threading>
</runtime>
</esper-configuration>]()
We recommend that when disabling the internal timer, applications send an external timer event setting the start time before creating statements, such that statement start time is well-defined.
Esper 8.8.0 is now available:
https://github.com/espertechinc/esper/blob/master/changelog.txt
Currently we are using
<dependency>
<groupId>com.espertech</groupId>
<artifactId>esper</artifactId>
<version>4.7.0</version>
</dependency>
This seems to be pretty old. From http://www.espertech.com/esper/esper-documentation, current version (in the moment of writting this) is 7.0.0 and the older supported one seems to be 4.10.0 (dated 2013).
We have to upgrade to a newer version
When the perseo-core is packaged with mvn package
the file perseo-core.properties is not generated/copied when the war is deployed by tomcat
/usr/local/tomcat/logs
catalina.2019-11-29.log localhost_access_log.2020-01-10.txt localhost_access_log.2020-04-17.txt localhost_access_log.2020-07-20.txt localhost_access_log.2020-10-22.txt
catalina.2020-03-16.log localhost_access_log.2020-01-11.txt localhost_access_log.2020-04-18.txt
....
localhost_access_log.2020-01-07.txt localhost_access_log.2020-04-14.txt localhost_access_log.2020-07-17.txt localhost_access_log.2020-10-19.txt manager.2021-01-26.log
localhost_access_log.2020-01-08.txt localhost_access_log.2020-04-15.txt localhost_access_log.2020-07-18.txt localhost_access_log.2020-10-20.txt
localhost_access_log.2020-01-09.txt localhost_access_log.2020-04-16.txt localhost_access_log.2020-07-19.txt localhost_access_log.2020-10-21.txt
ls | wc -l
468
time=2023-05-23T09:52:32.626Z | lvl=WARN | from=::ffff:10.129.2.52 | corr=bec9201c-7b89-45ba-85d7-caf1c703f5f2; node=-7GlhSXN6v; perseocep=816679 | trans=645090e0-08c8-4088-8094-96592f65480d | srv=jcyl | subsrv=/silossal | op=remove | comp=perseo-core | msg=.removeFromNode (24) Could not find the filterCallback to be removed within the supplied node, params=[FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='service'}', filterOperator=EQUAL, filterValue=jcyl}, FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='subservice'}', filterOperator=EQUAL, filterValue=/silossal}, FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='service'}', filterOperator=IS, filterValue=jcyl}, FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='type?'}', filterOperator=EQUAL, filterValue=RoadFrostSensor}, FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='.boolean_expression'}', filterOperator=BOOLEAN_EXPRESSION, filterValue=com.espertech.esper.common.internal.filterspec.ExprNodeAdapterSSPlain@a3879}] filterCallback=com.espertech.esper.common.internal.context.util.EPStatementHandleCallbackFilter@61727b45
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.