Description: Configuring siddhi to receive text type events by reading text files using regex match is not processing, but can see the file has deleted( action after process)

Steps to recreate:

• Configure siddhi app for file source with mode=regex and with text mapper as below

@App:name('SiddhiAppFileRgx') @source(type='file', mode='regex', begin.regex='events-', end.regex='-events', tailing='false', file.uri='/home/kavitha/usr/SP_filein/textTest.txt', action.after.process ='delete', action.after.failure='move', move.after.failure='/home/kavitha/usr/SP_fileout', @map(type='text')) define stream RegxStream1 (symbol string, price float, volume long); define stream RegxStream2 (symbol string, price float, volume long);

from RegxStream1#log("stream is processed.... ") insert into RegxStream2;

• Save a 'textTest' text file in given location with event details as below (e.g.)

events- symbol:"TestABC", price:55.0, volume:1000 -events

• Deploy the siddhi app and check

Description:
https://docs.wso2.com/display/SP400/Distributed+Deployment
In the above documentation, under Configuring the clustered setup,
To configure resource nodes, needed parameters are not there in deployment.yaml file.(<SP_HOME>/conf/worker/deployment.yaml)

when bringing up the server, this error is throwing,

Exception in thread "Timer-0" java.lang.ExceptionInInitializerError
at com.rabbitmq.client.impl.AMQConnection.defaultClientProperties(AMQConnection.java:75)
at com.rabbitmq.client.ConnectionFactory.(ConnectionFactory.java:96)
at org.wso2.extension.siddhi.io.rabbitmq.sink.RabbitMQSink.connect(RabbitMQSink.java:366)
at org.wso2.siddhi.core.stream.output.sink.Sink.connectWithRetry(Sink.java:167)
at org.wso2.siddhi.core.SiddhiAppRuntime.start(SiddhiAppRuntime.java:273)
at org.wso2.carbon.stream.processor.core.internal.StreamProcessorService.deploySiddhiApp(StreamProcessorService.java:71)
at org.wso2.carbon.stream.processor.core.internal.StreamProcessorDeployer.deploySiddhiQLFile(StreamProcessorDeployer.java:80)
at org.wso2.carbon.stream.processor.core.internal.StreamProcessorDeployer.deploy(StreamProcessorDeployer.java:159)
at org.wso2.carbon.deployment.engine.internal.DeploymentEngine.lambda$deployArtifacts$6(DeploymentEngine.java:291)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at org.wso2.carbon.deployment.engine.internal.DeploymentEngine.deployArtifacts(DeploymentEngine.java:282)
at org.wso2.carbon.deployment.engine.internal.RepositoryScanner.sweep(RepositoryScanner.java:114)
at org.wso2.carbon.deployment.engine.internal.RepositoryScanner.scan(RepositoryScanner.java:68)
at org.wso2.carbon.deployment.engine.internal.DeploymentEngine.start(DeploymentEngine.java:121)
at org.wso2.carbon.deployment.engine.internal.DeploymentEngineListenerComponent.onAllRequiredCapabilitiesAvailable(DeploymentEngineListenerComponent.java:216)
at org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager.lambda$notifySatisfiableComponents$38(StartupComponentManager.java:250)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager.notifySatisfiableComponents(StartupComponentManager.java:236)
at org.wso2.carbon.kernel.internal.startupresolver.StartupOrderResolver$1.run(StartupOrderResolver.java:204)

Description:
It would be easier for a developer if he can see the logs in the console of Stream Processor Studio itself rather than having to go and look at the terminal.

When I was adding content to Quick Start Guide, in order to convince a user that the query actually works I directed him/her to log the output stream using newly added log sink. The logs will be shown in the terminal he/she used to start the editor. It really is an inconvenience and it would be much much better if he/she can see the logs in the console itself.

Related Issues:
[Improvement] [Editor] Showing events from a user selected stream in the console #100

Description:
https://docs.wso2.com/display/SP400/Creating+a+Distributed+Siddhi+Application
In the above documentation, given links are broken.The links are highlighted below.
A Siddhi Application is a combination of multiple Siddhi executional elements. A Siddhi executional element can be a Siddhi Query or a Siddhi Partition.

== Steps to Recreate ==

1. Start the editor, open 'http-io-sample' sample and save it.
2. Go to 'Single Event Simulation' , select the saved siddhi application as 'Siddhi App Name', inputStreamhttp as 'Stream Name' and add a valid Time Stamp (optional)
3. Fill necessary values for event attributes which will be listed after selecting the stream.
4. Start the siddhi application by clicking the 'Start' button.
5. Click 'Send' to send the event

== Actual Result ==

Exception thrown in backend for multiple registered message processors.

ERROR {org.wso2.carbon.transport.http.netty.internal.HTTPTransportContextHolder} - More than one message processor has registered and cannot proceed with 'null' message processor ID. ERROR {org.wso2.carbon.transport.http.netty.sender.TargetHandler} - Cannot correlate callback with request callback is null

== Expected Result ==

No exception in backend.

== Steps to Recreate ==

1. Start the editor, go to event simulator and the select 'Feed Simulation'
2. Create a feed simulation with 'CSV file' as the simulation and save.
3. Click 'Edit' of the created simulation and press 'Cancel' to exit without doing any modifications.

== Actual Result ==

Popup will be displayed by asking to confirm where to discard the changes or not even there're no changes.

== Suggestion ==

Better if the pop up won't display if the user haven't done any changes for simulation.

Description: New files are getting deleted when 'file.uri' configured with 'action.after.process' is 'delete'

Steps to recreate:

• Configure siddhi for file source with 'file.uri' and 'action.after.process' set to "delete" as below

@source(type='file', mode='regex', begin.regex='event-', end.regex='-event', dir.uri='/home/kavitha/usr/SP_filein', tailing='false', action.after.process = 'delete', action.after.failure='move', move.after.failure='/home/kavitha/usr/SP_fileout', @map(type='text')) define stream RegxStream1 (symbol string, price float, volume long); define stream RegxStream2 (symbol string, price float, volume long);

• Deploy the siddhi app

Observed: Siddhi processed the event and particular file got deleted from location but, meantime all new files added to particular folder also getting deleted.

Description:
In the siddhi editor the font size is too small to read. If a developer need to increase the font size, the browser should be zoomed.

• It would be better to allow the developer to adjust font size according to the choice.
• Increasing the default font size also will help the developers

Suggested Labels:
Improvements

Suggested Assignees:

Affected Product Version:
4.0.0

Steps to Recreate:

1. Define http sink as below.

@Sink(type='http',publisher.url='http://localhost:8080/testresults', headers='methodName:com.wso2.sp.test.VerifyXML', method='POST',
@map(type='json'))
define stream FooStream (message string,method String,headers String);

2. Monitor headers receiving at publisher.url

Actual Result:
the header defined above will receive as below as escaping the first letter(m) from it.
ethodName=com.wso2.sp.test.VerifyXML

Expected Result:
Defined header should be received as it is.
methodName=com.wso2.sp.test.VerifyXML

Description:
The text box to input values for an attribute in the stream doesn't show when selecting the first stream for the first time

Affected Product Version:
v4.0.0-M14

OS, DB, other environment details and versions:
Ubuntu 16.04 LTS

Steps to reproduce:

1. Start the editor
2. Copy and paste the following query
   @app:name("HelloWorldApp")

define stream CargoStream (weight int);

@info(name='HelloWorldQuery')
from CargoStream
select weight, sum(weight) as totalWeight
insert into OutputStream;

from OutputStream#log("LOGGER")
select *
insert into temp2;

3. Save the .siddhi file
4. Go to Event Simulator
5. Select HelloWorldApp in Siddhi App Name drop down
6. CargoStream is selected by default. (You can even re-select the CargoStream from drop-down)
   The text box to input weight doesn't appear in the first try. When you select another stream it appears. And when you re-select CargoStream after selecting another stream the text box for weight appears.

Description:
https://docs.wso2.com/display/SP400/Distributed+Deployment
In the above documentation, there are some typos and grammatical errors which need to be corrected.

Description:
Editor start-up script needs to be changed to 'studio' as Stream Processor Studio is the product name

Suggested Labels:
Editor, Improvement

== Steps to Recreate ==

1. Create a feed simulation by adding valid values 'Time Interval' and 'TimeStamp Interval' with 'CSV file' as the simulation and save.
2. Go to [SP_HOME]/deployment/simulation-configs and open the simulation file which created early.

== Actual Result ==

Both 'Time Interval' and 'TimeStamp Interval' fields refer to same filed as 'timeInterval' in simulation config.

In File

In UI

== Expected Result ==

Referred fields in simulation config file for both attributes should be map to different attributes.

Description:
when you download and extract the WSO2 stream processor and navigate to SP_HOME /bin and try to execute ./editor it won't start it give's an error saying can't open and gives a part of the SP_HOME absolute path until it encounters a white space character like this

Suggested Labels:

Component/Studio
Type/Bug

Suggested Assignees:

Affected Product Version:
4.0.0
OS, DB, other environment details and versions:
Ubuntu Linux
Steps to reproduce:
Download the WSO2 stream processor zip from the following link
https://github.com/wso2/product-sp/releases
and extract it inside a folder with a name containing white spaces (eg: Stream Processor) and navigate into SP_HOME/bin and try to execute ./editor.sh

it won't start until you remove the white spaces in the SP_HOME path (eg: Chaning Stream Processor to Stream_Processor or StreamProcessor)

Related Issues:

Description:

When I try suggestions in writing Siddhi Queries,
Before selecting the suggestion

After selecting string as the suggestion

Still it shows up an error.

Suggested Labels:
Improvement

Affected Product Version:

4.0.0

== Steps to Recreate ==

1. Create a dashboard by providing a name and url.
2. Open dashboard in design view, go to available listed widgets and drag and drop few widgets to dashboard space.
3. Click 'Open in new window' button of one deployed widget in dashboard space.

== Actual Result ==

It will open the widget in a new window but there's no data displayed

== Expected Result ==

Widget should be display data when open in a new window.

== Steps to Recreate ==

1. Start the editor, go to event simulator and the select 'Feed Simulation'
2. Create a feed simulation by adding valid values for 'Starting Event's Timestamp', 'Ending Event's Timestamp' and 'Time Interval' with 'CSV file' as the simulation and save.

== Actual Result ==

A warning log thrown in backend for not having timestamp attribute or time interval.

WARN {org.wso2.carbon.event.simulator.core.internal.generator.csv.core.CSVEventGenerator} - Either timestamp attribute or time interval is required for CSV simulation of stream 'inputStreamHttp'. Time interval will be set to 1 second for source configuration : {"fileName":"tEstcEv.csv","delimiter":",","timeInterval":"1000","simulationType":"CSV_SIMULATION","streamName":"inputStreamHttp","isOrdered":true,"siddhiAppName":"io-sample"}

== Expected Result ==

No warn log should be printed since the necessary fields were filled accurately. Even though warn message mentioned about missing 'TimeStamp Attribute', there's no such attribute to add for simulation configs.

== Steps to Recreate ==

1. Start the editor, go to event simulator and the select 'Feed Simulation'
2. Create a feed simulation by adding valid values for 'Starting Event's Timestamp' and 'Ending Event's Timestamp' with 'CSV file' as the simulation and save.
3. Click 'Edit' of the created simulation.

== Actual Result ==

There's no date picker to choose date and time for time stamp field.

== Expected Result ==

User should be able to modify 'Starting Event's Timestamp' and 'Ending Event's Timestamp' by picking a date and time from date picker.

Description:
Aliment of the Run or Debug button is wrong

Suggested Labels:
Bug
Studio

Suggested Assignees:
Damith

Affected Product Version:
4.0.0-M14

Description:
An error message thrown when start the worker if Siddhi app is open as a file but already saved

Steps to recreate:

• Configure a valid Siddhi app
• Open above configuration file (it should be saved - no unsaved data should be there)
• Run the worker node

[2017-08-28 16:41:11,045] ERROR {org.wso2.carbon.stream.processor.core.internal.StreamProcessorDeployer} - Error: File extension not supported for file name siddhiapptest1.siddhi~. Support only.siddhi . org.wso2.carbon.stream.processor.core.internal.exception.SiddhiAppDeploymentException: Error: File extension not supported for file name siddhiapptest1.siddhi~. Support only.siddhi . at org.wso2.carbon.stream.processor.core.internal.StreamProcessorDeployer.deploySiddhiQLFile(StreamProcessorDeployer.java:95) at org.wso2.carbon.stream.processor.core.internal.StreamProcessorDeployer.deploy(StreamProcessorDeployer.java:159) at org.wso2.carbon.deployment.engine.internal.DeploymentEngine.lambda$deployArtifacts$0(DeploymentEngine.java:266) at org.wso2.carbon.deployment.engine.internal.DeploymentEngine$$Lambda$152/944746201.accept(Unknown Source) at java.util.ArrayList.forEach(ArrayList.java:1249) at org.wso2.carbon.deployment.engine.internal.DeploymentEngine.deployArtifacts(DeploymentEngine.java:257) at org.wso2.carbon.deployment.engine.internal.RepositoryScanner.sweep(RepositoryScanner.java:110) at org.wso2.carbon.deployment.engine.internal.RepositoryScanner.scan(RepositoryScanner.java:68) at org.wso2.carbon.deployment.engine.internal.DeploymentEngine.start(DeploymentEngine.java:106) at org.wso2.carbon.deployment.engine.internal.DeploymentEngineListenerComponent.onAllRequiredCapabilitiesAvailable(DeploymentEngineListenerComponent.java:212) at org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager.lambda$notifySatisfiableComponents$51(StartupComponentManager.java:238) at org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager$$Lambda$103/783859647.accept(Unknown Source) at java.util.ArrayList.forEach(ArrayList.java:1249) at org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager.notifySatisfiableComponents(StartupComponentManager.java:224) at org.wso2.carbon.kernel.internal.startupresolver.StartupOrderResolver$1.run(StartupOrderResolver.java:204) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505)
Suggestion: Can we improve it to get rid of error message

Description:
I tried the SP editor with firefox. There is an unnecessary button middle of the screen and it will not disappear.

Steps on creating and deploying the MongoDB server should be included in the README file.

== Steps to Recreate ==

1. Create a dashboard by providing a name and url.
2. Open dashboard in design view, go to available listed pages and select a page

== Actual Result ==

There's no option to remove the page.

== Expected Result ==

There should be an option to remove a selected page

Description:
$subject need to be removed in autocompletion

Description:
sink and source is added to the autocompletion but not @store

Description:
Sometimes the tooltip is not visible as its color is similar to the editor background.

Make it a bit lighter.

Description:

When executing the sample in this doc https://docs.wso2.com/display/SP400/Sample+0001%3A+Receiving+and+Publishing+Events+via+Kafka+Transport , some steps are missing.

Suggested Labels:

We need to start zookeeper node using bin/zookeeper-server-start.sh config/zookeeper.properties and kafka server node using bin/kafka-server-start.sh config/server.properties before start a worker node. It should be mentioned in the documentation.

Description:
Cannot put debug points to query in or out inside a partition

Affected Product Version:
WSO2 Stream Processor 4.0.0 Milestone 13 Release

OS, DB, other environment details and versions:
Ubuntu 16.04

Steps to reproduce:
try to add debug points to follwing siddhi app.
-------------SIddhi app------>
@app:name('TestSiddhiApp')
define stream sensorStream (sensorId string, temperature double);
partition with (sensorId of sensorStream)
begin
from sensorStream#window.length(1)
select max(temperature) as maxTemp
insert into DeviceTempStream
end;
----------End of siddhi app---->

== Steps to Recreate ==

1. Start the editor, go to event simulator in Firefox and the select 'Feed Simulation'
2. Try to add a TimeStamp ('Starting Event's Timestamp' or 'Ending Event's Timestamp') by picking a date and time

== Actual Result ==

Unable to pick a time and date from date picker. A 'NaN' string will be displayed in field.

== Expected Result ==

A postive integer for selected time and date should be added in field. This works fine with chrome.

In Firefox (Not Working)

In Chrome (Working)

Description:
Sometimes the editor console does not appear when the console button was pressed, and sometimes the console div space is generated but no console is shown in that.
E.g.

Suggested Labels:
Bug
Studio

Suggested Assignees:
Damith

Affected Product Version:
4.0.0-M14

Description:
When we started to follow the Quick start Guide for Stream Processor, the step five does not work. the URL is missing. therefore we can not continue further. The description which is given in the link is not accurate
https://docs.wso2.com/display/SP400/Quick+Start+Guide

Description:
Not working on Windows, CARBON_HOME environment variable is not set correctly.

Suggested Labels:
SP, Severity/Major, Type/Bug

Affected Product Version:
Alpha2

OS, DB, other environment details and versions:
Windows

Description:

In this documentation https://docs.wso2.com/display/SP400/Supporting+Different+Transports#SupportingDifferentTransports-Kafka

There is no any directory called libs in the <SP_HOME>, therefore it should be <SP_HOME>/lib.

Description:
In Siddhi editor the console size is fixed. Once there is an error it is hard for the user to read the error. It would be better if the editor console size is adjustable.

Description:
When the user is writing a complex query it would help him/her verify the correctness and the workflow of the query if he/she can "somehow" select a stream and then the events from that particular stream can be observed in the console in Stream Processor Studio while the App is running.

Related Issues:
[Improvement] [Editor] Showing logs in the console #99

Description:

• It would be better if point 9 and 10 are changed since developers get an error as unable to connect (through it tries to connect to a server again and again).
• There is a typo at point no. 9. Correct the port number as 9893.

Suggested Assignees:
Rukshani Weerasinha

Description:
Tool tip messages include unnecessary spaces.

Suggested Labels:
Component/Studio, Type/Bug

Suggested Assignees:
@dnwick

Description:
There is a link which can be used to download kafka broker in this documentation which is related to the topic "Receiving and Publishing Events via Kafka Transport", but it does not direct to the correct place.

We can download kafka using https://kafka.apache.org/downloads.

== Steps to Recreate ==

1. Create a dashboard by providing a name and url.
2. Open dashboard in design view, go to available listed pages and select a page

== Actual Result ==

There's no option to make the page as 'Use as landing'

== Expected Result ==

There should be an option to ser as 'use as Landing' for a selected page

Description:
When you start either the editor or the worker, it just hangs.
./bin/worker.sh start

Suggested Labels:

Suggested Assignees:

Affected Product Version:
wso2sp-4.0.0-M16-SNAPSHOT

OS, DB, other environment details and versions:

Steps to reproduce:
Start the server using ./bin/worker.sh start

Related Issues:

Installing the rabbitmq server has several steps and needed to install other needed dependencies as well. While installing these dependencies conflicts occur. README file needed to be more descriptive.

Description:
the button is not shown

Shortcuts and buttons are not shown anywhere

Description:

In this documentation https://docs.wso2.com/display/SP400/Sample+0001%3A+Receiving+and+Publishing+Events+via+Kafka+Transport <DAS_HOME> should be changed to the <SP_HOME>

and

Description:
When we save the app while it's running, it's getting redeployed.

I think we can automatically stop the app before saving and log saying that the app stopping.
Else it will be just noise for the user.

Description:
Configured source for file type with regex mode as below with 'move.after.process' attribute,
@source(type='file', mode='regex', begin.regex='event-', end.regex='-event', dir.uri='/home/user/SP_filein', action.after.process = 'move', move.after.process = '/home/user/SP_fileout', @map(type='text'))

and got an error message like "moveAfterProcess cannot be used when tailing is enabled. Hence stop the siddhi app"

Usability issue: user doesn't get an idea why this error comes since tailing attribute is not present in this config.

Suggestion: Good if error message improved to let the user knows as by default tailing has enabled for regex mode, so that 'MoveAfterProcess' cannot be used

Description:
https://docs.wso2.com/display/SP400/Deployment+Guide
In the above documentation they have mentioned that the group ID and other configurations required for a clustered set up are edited in the <SP_HOME>/conf/deployment.yaml file.
But the path should be <SP_HOME>/conf/worker/deployment.yaml file.
or <SP_HOME>/conf/manager/deployment.yaml file.
I am not sure about the exact path, I hope that it will be correct.

Description:
In this documentation https://docs.wso2.com/display/SP400/Supporting+Different+Transports#SupportingDifferentTransports-Kafka

The step 2.f is not needed. If we do so it generates errors

Description:
max value is not produced ever

When random primitive is used, it's not outputing max value ever
E.g for the above setting cannot see a 3
it was only producing 0,1,2

== Steps to Recreate ==

1. Start the editor, go to event simulator and the select 'Feed Simulation'
2. Create a feed simulation with 'CSV file' as the simulation and save.
3. Click 'Edit' of the created simulation and delete the source by clicking the delete button near 'Source 1 - CSV File'
4. Select 'CSV file' as simulation source and click 'Add New'
5. Try to select 'Siddhi App Name', 'Stream Name' or 'CSV File' from dropdown list

== Actual Result ==

Unable to expand from drop down list to select above fields

== Expected Result ==

Existing Siddhi Apps, streams and csv files should be expand when clicking each drop down list.