Code Monkey home page Code Monkey logo

perseo-core's Introduction

Perseo-core (EPL server)

FIWARE Processing License: APGL Support badge
Quay badge Docker badge
CI Status

This is the code repository for the EPL-server, named perseo-core.

Index

quay.io 🐳 Docker Hub 🎯 Roadmap

License

Perseo Core is licensed under General Public License (GPL) version 2.

Further information on the use of the GPL open source license

Are there any legal issues with GPL 2.0? Is it safe for me to use?

There is absolutely no problem in using a product licensed under GPL 2.0. Issues with GPL (or AGPL) licenses are mostly related with the fact that different people assign different interpretations on the meaning of the term “derivate work” used in these licenses. Due to this, some people believe that there is a risk in just using software under GPL or AGPL licenses (even without modifying it).

For the avoidance of doubt, the owners of this software licensed under an GPL 2.0 license wish to make a clarifying public statement as follows:

Please note that software derived as a result of modifying the source code of this software in order to fix a bug or incorporate enhancements is considered a derivative work of the product. Software that merely uses or aggregates (i.e. links to) an otherwise unmodified version of existing software is not considered a derivative work, and therefore it does not need to be released as under the same license, or even released as open source.

perseo-core's People

Contributors

alvarovega avatar cblanco avatar cesarjorgemartinez avatar crbrox avatar dependabot[bot] avatar dmoranj avatar fgalan avatar fiqare-emergya-dev avatar frarape avatar ivanhdzc avatar jason-fox avatar jcalderin avatar jmcanterafonseca avatar leandroguillen avatar mapedraza avatar mrutid avatar rfgtid avatar samgh96 avatar vgarciag avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

perseo-core's Issues

GPL License

Remove this, it is wrong:

Software that merely uses or aggregates (i.e. links to) an otherwise unmodified version of existing software is not considered a derivative work, and therefore it does not need to be released as under the same license, or even released as open source.

Expose the possibility to change the log level through the API

API to implement:

POST /admin/log?level=info

The value of the "level" parameter is interpreted in case insensitive way and the valid values are:

FATAL, ERROR, WARNING, INFO, DEBUG

Result in the happy case:

200 OK (without payload)

Result in the case of error:

400 Bad Request

{
   "error": "not supported log level"
}

migrate log4j v1 to log4j v2

Perseo-core is using:

  • slf4j which uses log4j version 1.2,
  • log4j version 1.2

https://logging.apache.org/log4j/1.2/

Security Vulnerabilities

A security vulnerability, CVE-2019-17571 has been identified against Log4j 1. Log4j includes a SocketServer that accepts serialized log events and deserializes them without verifying whether the objects are allowed or not. This can provide an attack vector that can be expoited. Since Log4j 1 is no longer maintained this issue will not be fixed. Users are urged to upgrade to Log4j 2.

Propagate or generate transaction ID

In the case of receiving a transaction ID (in the incoming HTTP request in a given header), it has to be used (an propagated to any outgoing HTTP request derived from such incoming HTTP request) instead of auto-generating one.

If no transaction ID if provided in the HTTP request then CEP Core has to auto-generate one and propagate it to any outgoing HTTP request derived from that transaction.

Details about which particular header to use for transaction ID are under discussion in the moment of writing this.

log into stdout

Currently perseo core logs into a file which outputs through a tail. It should log directly to stdout.

time rules are not stored by service and subservice

Then the same rulename could not be used by different service and subservices

PUT in

rulesInfo.put(ruleName.get(0), jo.getJSONObject(i));

GET in

JSONObject rule = TimeRulesStore.getInstance().getRuleInfo(ruleName);

return rulesInfo.get(ruleName.split("@")[0]);

docker logs don't show the right log file

In spite the modifications introduced in #102 aimted to show the Perseo log instead of Catalina log (from Tomcat) in docker container it seems it is not working:

$ sudo docker logs $(sudo docker ps | grep perseo-core | awk -F ' ' '{print $1}')
...
29-Oct-2018 01:38:48.696 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/usr/local/tomcat/webapps/perseo-core.war] has finished in [3,283] ms
29-Oct-2018 01:38:48.704 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
29-Oct-2018 01:38:48.725 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
29-Oct-2018 01:38:48.746 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3416 ms

CC: @cblanco just in case you know and easy solution for this :)

When the json sent to Perseo is too long, the parsing fails

When the json sent to CEP is too long, the perseo-core fails parsing the json

{"error":"error post to http://10.95.82.36:8080/perseo-core/rules (Unterminated string at 2897 [character 2898 line 1])","data":null}

In addition to a fix in the code to check JSON length and report an error, the "too long" should defined (and documented) as part of the resolution of this bug.

incorrect syntax with script expression

I've tried to use a script expression in a rule, using two proposed esper syntax, but both are seen with incorrect syntax by perseo-core esper engine:
time=2018-03-14T13:13:15.842Z | lvl=ERROR | corr=n/a | trans=n/a | op=refreshCore | comp=perseo-fe | srv=n/a | subsrv=n/a | msg=[object Object] error put to http://perseo-core:8080/perseo-core/rules (Incorrect syntax near 'expression' at line 1 column 35 [context ctxt$testservice$testalain expression String js:trunc(name)[trunc(name); function trunc(n) {return n.substr(n.indexOf(":" + 1));}] select *,"my_rule_update_tto" as ruleName, *,cast(cast(ev.temperature?, String),float) + 1.0 as temp, trunc(cast(ev.id?, String)) as iddev from pattern [every ev=iotEvent(cast(cast(temperature?,String),float)<5.0 and type="thing")]])

time=2018-03-14T13:23:15.875Z | lvl=ERROR | corr=n/a | trans=n/a | op=refreshCore | comp=perseo-fe | srv=n/a | subsrv=n/a | msg=[object Object] error put to http://perseo-core:8080/perseo-core/rules (Incorrect syntax near 'expression' at line 1 column 35 [context ctxt$testservice$testalain expression String js:trunc(name)[name.substr(name.indexOf(":" + 1));] select *,"my_rule_update_tto" as ruleName, *,cast(cast(ev.temperature?, String),float) + 1.0 as temp, trunc(cast(ev.id?, String)) as iddev from pattern [every ev=iotEvent(cast(cast(temperature?,String),float)<5.0 and type="thing")]])

the used esper version (5.5.0) support script expression, any idea of the issue with the syntax ?
or is it a script expression support issue ?

Thanks in advance for help or feedback on this problem

bad entity update in a null value by a update rule (using "null as nulo" in EPL)

a rule like

{
    "name": "entityupdate",
    "text": "select *, 27 as numero, false as bool, null as nulo from iotEvent where type?='WaterTank'",
    "action": {
        "type": "update",
        "parameters": {
            "id": "TestEntity2",
            "type": "TestEntity",
            "attributes": [
                {
                    "name": "numero",
                    "value": "${numero}",
                    "type": "Number"
                },
                {
                    "name": "json",
                    "value": "[${nulo},1,2]",
                    "type": "Object"
                },
                {
                    "name": "nulo",
                    "value": "${nulo}",
                    "type": "Object"
                },
                {
                    "name": "bool",
                    "value": "${bool}",
                    "type": "Boolean"
                }
            ]
        }
    }
}

is updating an entity in a bad way:

{
    "id": "TestEntity2",
    "type": "TestEntity",
    "bool": {
        "type": "Boolean",
        "value": false,
        "metadata": {}
    },
    "json": {
        "type": "Object",
        "value": "[null,1,2]",
        "metadata": {}
    },
    "nulo": {
        "type": "Object",
        "value": "null",
        "metadata": {}
    },
    "numero": {
        "type": "Number",
        "value": 27,
        "metadata": {}
    }
}

This another rule:

{
    "name": "entityupdate",
    "text": "select *, 27 as numero, false as bool, null as nulo from iotEvent where type?='WaterTank'",
    "action": {
        "type": "update",
        "parameters": {
            "id": "TestEntity2",
            "type": "TestEntity",
            "attributes": [
                {
                    "name": "numero",
                    "value": "${numero}",
                    "type": "Number"
                },
                {
                    "name": "json",
                    "value": "[${numero},1,2]",
                    "type": "Object"
                },
                {
                    "name": "nulo",
                    "value": "${nulo}",
                    "type": "Object"
                },
                {
                    "name": "bool",
                    "value": "${bool}",
                    "type": "Boolean"
                }
            ]
        }
    }
}

is updating entity:

{
    "id": "TestEntity2",
    "type": "TestEntity",
    "bool": {
        "type": "Boolean",
        "value": false,
        "metadata": {}
    },
    "json": {
        "type": "Object",
        "value": [
            27,
            1,
            2
        ],
        "metadata": {}
    },
    "nulo": {
        "type": "Object",
        "value": "null",
        "metadata": {}
    },
    "numero": {
        "type": "Number",
        "value": 27,
        "metadata": {}
    }
}

date cast problems with esper

cast(cast(status_datetime?,string),date, dateformat: 'yyyy-MM-dd HH:mm:ss').format("yyyy-MM-dd'T'HH:mm:ss.SSSX") as statusdatetime

time=2022-06-29T09:34:52.668Z | lvl=ERROR | from=::ffff:172.17.0.16 |
corr=85f76744-5ba6-4080-ad44-7c7f5d4434bb; cbnotif=3; node=mSox4tLGxT;
perseocep=132 | trans=b7a07260-91cb-40cf-82ed-e9b0a7abc962 |
srv=smartcity | subsrv=/CEP | op=handleException | comp=perseo-core |
msg=Exception encountered processing deployment-id '91c27686-4208-4b0d-
bb5c-8af42c14f89d' statement ......

java.lang.NullPointerException: null
at
com.espertech.esper.common.internal.view.core.ZeroDepthStreamNoIterate.
insert(ZeroDepthStreamNoIterate.java:41) ~[esper-common-8.4.0.jar:?]
at
com.espertech.esper.common.internal.context.activator.ViewableActivator
Filter$1.matchFound(ViewableActivatorFilter.java:76) ~[esper-common-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processStatementFilterSingle(EPEventServiceImpl.java:796) ~[esper-
runtime-8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processMatches(EPEventServiceImpl.java:675) ~[esper-runtime-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processWrappedEvent(EPEventServiceImpl.java:453) ~[esper-runtime-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
sendEventMap(EPEventServiceImpl.java:395) ~[esper-runtime-8.4.0.jar:?]
at
com.telefonica.iot.perseo.EventsServlet.doPost(EventsServlet.java:91)
~[classes/:?]
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:681) ~[servlet-
api.jar:4.0.FR]
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:764) ~[servlet-
api.jar:4.0.FR]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
ationFilterChain.java:227) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
terChain.java:162) ~[catalina.jar:9.0.64]
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
~[tomcat-websocket.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
ationFilterChain.java:189) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
terChain.java:162) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVal
ve.java:197) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextVal
ve.java:97) ~[catalina.jar:9.0.64]
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticato
rBase.java:541) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav
a:135) ~[catalina.jar:9.0.64]
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.jav
a:92) ~[catalina.jar:9.0.64]
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccess
LogValve.java:687) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve
.java:78) ~[catalina.jar:9.0.64]
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
360) ~[catalina.jar:9.0.64]
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:3
99) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight
.java:65) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPr
otocol.java:890) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoin
t.java:1787) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.
java:49) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolE
xecutor.java:1191) ~[tomcat-util.jar:9.0.64]
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPool
Executor.java:659) ~[tomcat-util.jar:9.0.64]
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThre
ad.java:61) ~[tomcat-util.jar:9.0.64]
at java.lang.Thread.run(Thread.java:829) ~[?:?]

and

java.lang.NumberFormatException: For input string: ".E367E22"
at
jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.
java:2054) ~[?:?]
at
jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
~[?:?]
at java.lang.Double.parseDouble(Double.java:543) ~[?:?]
at java.text.DigitList.getDouble(DigitList.java:169) ~[?:?]
at java.text.DecimalFormat.parse(DecimalFormat.java:2126)
~[?:?]
at
java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2240) ~[?:?]
at
java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1541) ~[?:?]
at java.text.DateFormat.parse(DateFormat.java:393) ~[?:?]
at
com.espertech.esper.common.internal.epl.expression.funcs.ExprCastNode$S
tringToDateWStaticFormatComputer.stringToDateWStaticFormatParseSafe(Exp
rCastNode.java:501) ~[esper-common-8.4.0.jar:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.m20(Sele
ctExprProcessorImpl.java:496) ~[?:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.m19(Sele
ctExprProcessorImpl.java:486) ~[?:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.m1(Selec
tExprProcessorImpl.java:345) ~[?:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.m0(Selec
tExprProcessorImpl.java:327) ~[?:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$SelectExprProcessorImpl.process(
SelectExprProcessorImpl.java:317) ~[?:?]
at
com.espertech.esper.common.internal.epl.resultset.handthru.ResultSetPro
cessorHandThroughUtil.getSelectEventsNoHavingHandThruView(ResultSetProc
essorHandThroughUtil.java:45) ~[esper-common-8.4.0.jar:?]
at
generated.ResultSetProcessorFactoryProvider_3bdadcdf45383c454b0545a4594
5ea037a4d1438_ruleTest64smartcity47CEP$RSP.processViewResult(RSP.java:1
31) ~[?:?]
at
generated.OutputProcessViewFactoryProvider_3bdadcdf45383c454b0545a45945
ea037a4d1438_ruleTest64smartcity47CEP$OPV.update(OPV.java:65) ~[?:?]
at
com.espertech.esper.common.internal.view.core.ZeroDepthStreamNoIterate.
insert(ZeroDepthStreamNoIterate.java:41) ~[esper-common-8.4.0.jar:?]
at
com.espertech.esper.common.internal.context.activator.ViewableActivator
Filter$1.matchFound(ViewableActivatorFilter.java:76) ~[esper-common-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processStatementFilterSingle(EPEventServiceImpl.java:796) ~[esper-
runtime-8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processMatches(EPEventServiceImpl.java:675) ~[esper-runtime-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
processWrappedEvent(EPEventServiceImpl.java:453) ~[esper-runtime-
8.4.0.jar:?]
at
com.espertech.esper.runtime.internal.kernel.service.EPEventServiceImpl.
sendEventMap(EPEventServiceImpl.java:395) ~[esper-runtime-8.4.0.jar:?]
at
com.telefonica.iot.perseo.EventsServlet.doPost(EventsServlet.java:91)
~[classes/:?]
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:681) ~[servlet-
api.jar:4.0.FR]
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:764) ~[servlet-
api.jar:4.0.FR]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
ationFilterChain.java:227) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
terChain.java:162) ~[catalina.jar:9.0.64]
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
~[tomcat-websocket.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
ationFilterChain.java:189) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
terChain.java:162) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVal
ve.java:197) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextVal
ve.java:97) ~[catalina.jar:9.0.64]
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticato
rBase.java:541) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav
a:135) ~[catalina.jar:9.0.64]
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.jav
a:92) ~[catalina.jar:9.0.64]
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccess
LogValve.java:687) ~[catalina.jar:9.0.64]
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve
.java:78) ~[catalina.jar:9.0.64]
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
360) ~[catalina.jar:9.0.64]
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:3
99) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight
.java:65) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPr
otocol.java:890) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoin
t.java:1787) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.
java:49) ~[tomcat-coyote.jar:9.0.64]
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolE
xecutor.java:1191) ~[tomcat-util.jar:9.0.64]
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPool
Executor.java:659) ~[tomcat-util.jar:9.0.64]
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThre
ad.java:61) ~[tomcat-util.jar:9.0.64]
at java.lang.Thread.run(Thread.java:829) ~[?:?]

/etc/perseo-core.properties missed and perseo-fe url as variable for docker

  • This file is missed in perseo-core deploys: /etc/perseo-core.properties
# This file should be copied by deployment process
# into /etc/perseo-core.properties, with the appropiate permissions

# URL for invoking actions when a rule is fired
action.url = http://<PERSEO_FE_URL>/actions/do

# Time in milliseconds (long) to "expire" a "dangling" rule
rule.max_age= 60000
  • The variable PERSEO_FE_URL should be a parameter for docker container

java.io.FileNotFoundException in unit test

Running mvn package causes the following if the file /var/log/perseo/perseo-core.log doesn't exist:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running es.tid.fiware.perseo.GenericListenerTest
update
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /var/log/perseo/perseo-core.log (No existe el fichero o el directorio)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:136)
    at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
    at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
    at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
    at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
    at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:295)
    at org.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:176)
    at org.apache.log4j.xml.DOMConfigurator.findAppenderByReference(DOMConfigurator.java:191)
    at org.apache.log4j.xml.DOMConfigurator.parseChildrenOfLoggerElement(DOMConfigurator.java:523)
    at org.apache.log4j.xml.DOMConfigurator.parseCategory(DOMConfigurator.java:436)
    at org.apache.log4j.xml.DOMConfigurator.parse(DOMConfigurator.java:999)
    at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:867)
    at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:773)
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
    at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
    at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73)
    at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242)
    at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254)
    at es.tid.fiware.perseo.GenericListener.<clinit>(GenericListener.java:36)
    at es.tid.fiware.perseo.GenericListenerTest.testUpdate(GenericListenerTest.java:65)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)

Curiosly, the test runs without fail.

Thi excepction should be avoided. Maybe it is just a matter of adjusting the logger for the unit test target in order not using logging file or a logging file mock.

Astronomic Clock in rules

It's mandatory to allow the usage of astronomic clock concepts in Perseo time rules. Mainly Sunset and Sunrise based on LAT/LOG values.

I suppose this could be achieved by adding the proper Java Library to Esper deployment.

Support fort NGSI-LD

Hi,

we would like to use perseo in conjunction with orion-ld. However, perseo only supports ngsi-v2. When is NGSI-LD support planned?

regards,
rob

set internal time resolution

https://esper.espertech.com/release-8.4.0/reference-esper/html_single/#apiruntime-time-internal

[17.6.1.4. Internal Timer Settings]()

This option can be used to disable the internal timer thread and such have the application supply external time events, as well as to set a timer resolution.

The next code snippet shows how to disable the internal timer thread via the configuration API:

[Configuration config = new Configuration();
config.getRuntime().getThreading().setInternalTimerEnabled(false);]()

This snippet of XML configuration leaves the internal timer enabled (the default) and sets a resolution of 200 milliseconds (the default is 100 milliseconds):

[<esper-configuration xmlns="http://www.espertech.com/schema/esper">
  <runtime>
    <threading>
      <internal-timer enabled="true" msec-resolution="200"/>
    </threading>
  </runtime>
</esper-configuration>]()

We recommend that when disabling the internal timer, applications send an external timer event setting the start time before creating statements, such that statement start time is well-defined.

problem with perseo-fe on receving perseo-core timer:at() fired actions

I've been using perseo project recently and i've tested some of its functionalities through the provided examples in Github . i have deployed perse-core as a docker container and perseo-fe source using npm.
i configured the perseo-fe along with the perseo-core container and i made sure they communicate with each other by using the examples in https://github.com/telefonicaid/perseo-fe/tree/master/examples . But then i began to use a new rule including the timer:at() function so that the perse-core fires an action at a specific time.
see

then i checked the perseo-core logs and made sure it fired an action but there was failure in communication between it and the perseo-fe as follows .

screenshot from 2018-05-15 06 56 39

and i don't know what is the problem with perseo-fe and why it gives this error

i checked the perseo-core separately with a socket server listening on port 6000 and made the core fire towards it instead of 9090 and i made sure the perseo-core fires on time but i also found another problem where it fired only the values involved in the select clause and it didn't fire the json body included in the action field .
So there are two problems
1- the perseo-fe has a problem on receiving actions from the perseo-core with the timer:at() function
2-the perseo-core fires the values of the select clause and doesnot include the json included action field.

I hope there would be a solution for these issues @fgalan @dmoranj .

Checkstyle seems to be broken

(Rasied in #151 (comment))

It seems that the checkstyle pass (based in telefonica_checkstyle.xml ruleset) is not working. Although the pom.xml is supposedly configured to use it, it seems that mvn checkstyle:check uses the default sun_checks.xml ruleset.

I did an attemp of fixing it, but not sure if it's ok. I'm including it below, it could be a good starting point.

diff --git a/pom.xml b/pom.xml
index a210429..d8bcaea 100644
--- a/pom.xml
+++ b/pom.xml
@@ -41,7 +41,8 @@
             **/src/main/java/org/json/**, org/json/**
         </sonar.exclusions>
         <sonar.language>java</sonar.language>
-        <sonar.dynamic>reuseReports</sonar.dynamic>
+	<sonar.dynamic>reuseReports</sonar.dynamic>
+	<checkstyle.config.location>telefonica_checkstyle.xml</checkstyle.config.location>
     </properties>
 
     <build>
@@ -121,6 +122,11 @@
                 <configuration>
                 </configuration>
             </plugin>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-checkstyle-plugin</artifactId>
+                <version>3.0.0</version>
+            </plugin>
         </plugins>
     </build>
     <reporting>
@@ -146,9 +152,7 @@
                 <artifactId>maven-checkstyle-plugin</artifactId>
                 <version>3.0.0</version>
                 <configuration>
-                    <configLocation>telefonica_checkstyle.xml</configLocation>
-                    <includeTestSourceDirectory>false</includeTestSourceDirectory>
-                    <failOnViolation>false</failOnViolation>
+		    <configLocation>telefonica_checkstyle.xml</configLocation>
                 </configuration>
             </plugin>
             <plugin>
diff --git a/telefonica_checkstyle.xml b/telefonica_checkstyle.xml
index 5a6539e..1dfab02 100644
--- a/telefonica_checkstyle.xml
+++ b/telefonica_checkstyle.xml
@@ -196,7 +196,7 @@
         <module name="MagicNumber"/>
         -->
         <module name="MissingSwitchDefault"/>
-        <module name="RedundantThrows"/>
+        <!--module name="RedundantThrows"/-->
         <module name="SimplifyBooleanExpression"/>
         <module name="SimplifyBooleanReturn"/>
 
@@ -229,4 +229,4 @@
         </module>
     </module>
 
-</module>
\ No newline at end of file
+</module>

Ideally, checkstyle should be included in .travis.yml.

[SHOULD] Work towards fulfilling the requirements for a full GE

The following MUST requirements for full GE status are not currently satisfied:

Missing Badges:

  • README Badge - Docs (ReadtheDocs)
  • README Badge - Docker Pulls
  • README Badge - StackOverflow Tag
  • README Badge - Support Channel (if not SO)
  • README Badge - CI Build/Unit Test (Travis)

Missing items within the README

  • README - FIWARE Paragraph present
  • README - QA Section
  • README - How to Deploy text present
  • README - How to Run Tests text present
  • README - Main API Walkthrough text present
  • README - Access to the advanced API and Documentation text present
  • README - Training Section
  • README- Contains description of GE

Missing on ReadtheDocs Documentation

  • DOCS - Uses Chapter CSS
  • DOCS - Chapter Badge

No CREDITS file

An incubated enabler should be working to fulfilling these requirements

  • SHOULD requirement from the TSC

catalina and localhost_access_log files are still created into container

/usr/local/tomcat/logs

catalina.2019-11-29.log localhost_access_log.2020-01-10.txt localhost_access_log.2020-04-17.txt localhost_access_log.2020-07-20.txt localhost_access_log.2020-10-22.txt
catalina.2020-03-16.log localhost_access_log.2020-01-11.txt localhost_access_log.2020-04-18.txt

....

localhost_access_log.2020-01-07.txt localhost_access_log.2020-04-14.txt localhost_access_log.2020-07-17.txt localhost_access_log.2020-10-19.txt manager.2021-01-26.log
localhost_access_log.2020-01-08.txt localhost_access_log.2020-04-15.txt localhost_access_log.2020-07-18.txt localhost_access_log.2020-10-20.txt
localhost_access_log.2020-01-09.txt localhost_access_log.2020-04-16.txt localhost_access_log.2020-07-19.txt localhost_access_log.2020-10-21.txt

ls | wc -l
468

Warn log about "Could not find the filterCallback to be removed within..."

time=2023-05-23T09:52:32.626Z | lvl=WARN | from=::ffff:10.129.2.52 | corr=bec9201c-7b89-45ba-85d7-caf1c703f5f2; node=-7GlhSXN6v; perseocep=816679 | trans=645090e0-08c8-4088-8094-96592f65480d | srv=jcyl | subsrv=/silossal | op=remove | comp=perseo-core | msg=.removeFromNode (24) Could not find the filterCallback to be removed within the supplied node, params=[FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='service'}', filterOperator=EQUAL, filterValue=jcyl}, FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='subservice'}', filterOperator=EQUAL, filterValue=/silossal}, FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='service'}', filterOperator=IS, filterValue=jcyl}, FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='type?'}', filterOperator=EQUAL, filterValue=RoadFrostSensor}, FilterValueSetParamImpl{lookupable='ExprFilterSpecLookupable{expression='.boolean_expression'}', filterOperator=BOOLEAN_EXPRESSION, filterValue=com.espertech.esper.common.internal.filterspec.ExprNodeAdapterSSPlain@a3879}] filterCallback=com.espertech.esper.common.internal.context.util.EPStatementHandleCallbackFilter@61727b45

GET /admin/log operation

Implement the admin API to get current log level. This is:

GET /admin/log

{
   "level": "INFO"
}

In addition, it has to be included in the documented, along with the PUT /admin/log?level=XXX op.

Note that level is returned in UPPPERCASE.

DateTime in perseo

Context

Issue

Since Orion-LD supports NGSI-V2 out-of-the-box we should be able to use the Perseo services without any trouble.

After having the broker-perseo subscription setup and receiving the event in the perseo-core component, I've noticed that the DateTime pseudo-attributes do not get generated.

Incoming message in perseo-core

{
  "noticeId": "5ae4e520-5824-11ec-9ba9-a3d9baac0822",
  "noticeTS": 1638967370866,
  "id": "urn:ngsi-ld:Entity:001",
  "type": "Delivery",
  "isPattern": false,
  "subservice": "/",
  "service": "unknownt",
  "requestedTimeslot__type": "Property",
  "requestedTimeslot": "2014-04-29T13:18:05Z",
...

Expected incoming message in perseo-core to have parsed pseudo-attributes for the DateTime property :

...
    "requestedTimeslot": "2014-04-29T13:18:05Z",
    "requestedTimeslot__ts": 1398777485000,
    "requestedTimeslot__day": 29,
...

The notification was triggered by the following request to the context broker (ngsi-ld interface). Mind the DateTime notation according to "EXAMPLE OF MIGRATION TO NGSI-LD" in NGSI-LD docs

curl --location --request PATCH 'http://orion:1026/ngsi-ld/v1/entities/urn:ngsi-Entity:001/attrs' \
--header 'Content-Type: application/ld+json' \
--data-raw '{
    "requestedTimeslot": {
        "value": {
            "@type": "DateTime",
            "@value": "2014-04-29T13:18:05Z"
        },
        "type": "Property"
    },
}'

Upgrade Esper version

Currently we are using

<dependency>
            <groupId>com.espertech</groupId>
            <artifactId>esper</artifactId>
            <version>4.7.0</version>
</dependency>

This seems to be pretty old. From http://www.espertech.com/esper/esper-documentation, current version (in the moment of writting this) is 7.0.0 and the older supported one seems to be 4.10.0 (dated 2013).

We have to upgrade to a newer version

Changing Java package name

Currently we are using as Java package name:

es.tid.fiware.perseo;

It will be desirable to align this with Sonar naming, thus it could be:

com.telefonica.iot.perseo;

Effort: 1 man day

RDT. Action required?

From [email protected] on 02/08/2023

imagen

You are receving this email because your Read the Docs project is impacted by an upcoming deprecation.
Historically, Read the Docs has installed MkDocs 0.17.3 by default if the project didn't define a specific version of it. Now, this version is a pretty old and we recommend our users to use the latest version of MkDocs: 1.5.1.
To avoid failing builds we kept the old default version of MkDocs. We are changing that behavior to always install the latest available version and we wanted to let you know about that so you can prepare your project.
Starting on October 3rd Read the Docs will install the latest available version of MkDocs. If you want to use a different version, make sure you are pinning your MkDocs dependency.
Here you have a pretty simple example of the section required on the .readthedocs.yaml configuration file:
python:
install:
- requirements: docs/requirements.txt
The content of docs/requirements.txt would be similar to:
mkdocs==1.5.1
We are sending you this email because you are a maintainer of the following projects that are affected by this removal:
fiware-orion
fiware-perseo-core
Read more about this in our Reproducible builds guide for more details.

TaskThread stack trace from tomcat-util.jar

          at java.lang.Thread.run(Thread.java:829) ~[?:?]
            at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-util.jar:9.0.62]
            at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-util.jar:9.0.62]
            at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-util.jar:9.0.62]
            at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-coyote.jar:9.0.62]
            at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743) ~[tomcat-coyote.jar:9.0.62]
            at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890) ~[tomcat-coyote.jar:9.0.62]
            at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat-coyote.jar:9.0.62]
            at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) ~[tomcat-coyote.jar:9.0.62]

docker env var to set log level

Currently there is no way to change log level without use API.
Currently default log level is INFO
A docker en var to set a a different log level would be desirable.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.