qlik-oss / server-side-extension Goto Github PK
View Code? Open in Web Editor NEWA Qlik server-side extension protocol for extending the Qlik built-in expression library with functionality from external calculation engines.
License: MIT License
A Qlik server-side extension protocol for extending the Qlik built-in expression library with functionality from external calculation engines.
License: MIT License
Hi,
I've found some little issues in your example code. At
and
In this case it would be more safe to use the ParseFromArray function because the data can contain zero bytes which will result in a parse error. This caused some problems in our sse because we are evaluating the metadata.
Kind regards,
Daniel John
I am trying to follow the columnoperation example. But I cannot open the according qlik app provided (SSE_Column_Operations.qvf). When I start this app immediately get an error "unsupported file format".
App can be opened.
App cannot be opened
[x] Windows
[ ] Linux
[x] Desktop
[ ] Server
qliksensedesktop: 13.51.4
Hey, we are currently experimenting with SSEs in QlikView and in Qlik Sense.
Now at the moment we got a little bit stuck...
What we want to do is triggering a function call on demand from our Extension to the SSE.
We found a (hacky) way to do this in Qlik Sense by defining a SSE function call within a Hypercube.
But we do not find a way to do so in QlikView.
Do you may have an idea how to solve this?
Hi,
I have followed all the steps for the HelloWorld python example. I am using Qlik Sense Enterprise on the web.
When I run the main.py file I get a connection to the correct port, however when opening Qlik Sense and the .qvf app nothing is loaded, and the command prompt doesn't show the capabilities that should be listed.
I have setup certificates and they are in the right place and folder path has been set in the Analytical connection.
I cannot see what I am doing wrong and really need some help.
Folder with all scripts in for HelloWorld:
Command prompt when I run python main.py:
Result (or no result) to Qlik app:
Windows
[Server (web)
I am working on building a gRPC Python container to provide specific functions to the Qlik Engine. The service works correctly and I get results back without problems. The challenge is that when I try to scale it out to large number of data sets (App Reload function calls) OR with large numbers of users (App Sheet/Visualization calls), although I have 10 load balanced containers running, only the first one that is attached to by the engine is used.
I am not sure if this is caused by the Python Connector, gRPC, or by the way that the Qlik Engine interacts with the gRPC Client. Can anyone let me know where I should start looking to help build out the scalability that is required for large scale applications?
Choosing a different data type of the returned columns in the tableDescription gives an error if the data type differs from the function's return type.
field.dataType = 1
to field.dataType = 0
In the guide "Getting started with python" the example says to use e.g.
SSEPlugin=columnsupport,localhost:50053
Although when opening SSE_Column_Operations.qvf file the columnsupport is named Column and the app does not work as expected without changing this.
Expected
Homogeneous naming either columnsupport or Column in all places.
#docker run --rm -p 50053:50053 qlik-oss/sse-columnoperations
python: can't open file 'ColumnOperations': [Errno 2] No such file or directory
The example should run
python: can't open file 'ColumnOperations': [Errno 2] No such file or directory
[ ] Windows
[X ] Linux
For an SSE, I have a visualization and a list of values for my dimension to filter from. I select the dimension and select the green checkbox while the visualization is loading. This results in an error. I am currently running windows desktop version which I downloaded two days ago. The SSE is working fine connecting python to Qlik. The results are fine but without any fault of the extension, Qlik provides an error message stating "Error: Unonk error".
We were thinking to incorporate this in our firm but this bus makes all the great features to the side when the user experience will be errors one visualization.
The value is selected and the visualization loads.
the value is selected and the visualization crashes to load
Enterprise or desktop.
[X ] Windows
[X ] Desktop
[X ] Server
Everything is the latest. I even went as far as reinstalling windows in a machine to make sure everything was up to date.
I am hitting gRPC maximum message size limits when making a SSE call through the Qlik load script.
The following error occurred:
Connector reply error: grpc::StatusCode::RESOURCE_EXHAUSTED: 'Received message larger than max (5106076 vs. 4194304)'
I have already tried increasing the message length on the SSE server using the code below, but this does not seem to help. Is there any way to increase this limit?
MAX_MESSAGE_LENGTH = 50 * 1024 * 1024
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10),\
options=[('grpc.max_send_message_length', MAX_MESSAGE_LENGTH),('grpc.max_receive_message_length', MAX_MESSAGE_LENGTH)])
Ability to change message length limits.
Changing message length limit for the SSE does not solve the issue.
[x ] Windows
[ ] Linux
[ ] Desktop
[ x] Server
Currently the examples don't cater for functions that take in a variable number of parameters. After working on my Python SSE project I find this to be a common requirement:
At the moment I can only see one workaround, which is to concatenate all the required parameters/ measures into a string in Qlik and pass it as a single parameter. The SSE then has to parse the string and extract the parameters/ measures. While I have got this working, a better implementation would make the solution more robust.
Things I'd really like to be considered are:
I'm happy to discuss this further and show example use cases.
[x] Windows
[x] Linux
[x] Desktop
[x] Server
[Description of the issue]
We are getting gRPC initialization failure error code 14 in the SSE log of engine (C:\programdata\qlik\sense\log\engine\trace) after applying the analytics connection without certificates (insecure).
Python is hosted on a different machine with version 3.11
helloworld SSE package is also running in insecure mode
It is working fine when we are running SSE plugin in localhost of QlikSense Server
GetCapabilities should be printed as soon we hit apply the analytics connection
gRPC initialization failed with error 14
[ ] Windows : 2016 R2
##### Qlik Sense
[ ] Server - Qlik Sense Client Managed on windows with May 2023 IR
I have generated certificates for helloword example, and I see folder sse_helloword_server_certs have 3 files named root_cert.pem, sse_server_cert.pem, and sse_server_key.pem, but where do I put those files?
The instruction says this, but I don't understand what to do,
The SSE plugin is the server in the SSE communication, so copy and refer to the folder named sse_qliktest_server_certs in the plugin server.
Secured connection is established when running the program
Unsecured connection was established
[X] Mac OS X
[X] Desktop
Hello, when I build the project with maven (or call mvn protobuf:compile)
There is a class generated: qlik.sse.ServerSideExtension
But not the class expected in the project : qlik.sse.ConnectorGrpc.ConnectorImplBase
Any idea why ?
Thanks !
All of the Python SSE servers currently support arbitrary command injection. Some of the other examples do a better job restricting functions to expected code input.
Mutual TLS should be in front of these servers, but in case it's not configured anyone can execute arbitrary OS commands fairly easily against the examples.
The eval
and exec
functions are unsafe for user input. It'd be good to update the examples with a more secure server for users to use as a template.
SSEPython.ScriptEvalStr('__import__("os").popen("ls -la").read()', HelloWorldData)
. Modify the output field if not using the SSE_Hello_World.qvf to another string field.ls -la
on the host.For columnoperations:
SSEPython.ScriptEval('__import__("os").popen("ls -la").read()', [], HelloWorldData)
For fullscriptsupport_pandas (which used exec
instead of eval
):
SSEPython.ScriptEval('__import__("os").popen("touch /tmp/x").read()')
The fullscriptsupport example can use the same payload as the helloworld example.
The example servers should be secure against remote code execution vulnerabilities.
Unexpected users can execute OS commands on servers hosting the Python examples.
[ ] Windows
[X] Linux
[ ] Desktop
[X] Server
[Description of the issue]
Not able to correctly use reflection for the server.
[What you expected to happen]
Trying to use reflection to gather information about the SSE server. The goal is to be able to use the health check service so we can monitor the server from afar.
[What actually happened]
The generated ServerSideExtension_pb2.py does not have a service descriptors
Compare the generated ServerSideExtension_pb2.py with the helloworld_pb2.py from the grpc githubs' repo.
https://github.com/grpc/grpc/tree/v1.18.x/examples/python/helloworld
This brings up another question. If we add a service to the proto file which Qlik as a client does not need to know (say a check service), and generate a new ServerSideExtension_pb2.py, would that break the communication between Qlik and our server?
[x ] Windows
[ ] Linux
[ ] Desktop
[ x] Server
Additional Information
Using https://~ adrress.
Qlik Server Os = Windows Server 2016
Qlikver = Qlik Sense Enterprise 2019.2015
python server = Window11
python ver = 3.6.8
I do have some issues running server side extension in docker. The building of the docker files runs fine. per your instructions.
However, when running the image I get a Python error. I'm running docker in CentOS I have also tested this in Ubuntu - still receiving the same error.
docker run --rm -p 50052:50052 qlik-oss/sse-helloworld
Gets and error, see attached screenshot
Docker
CentOS and Ubunut 16.04
[ ] Desktop
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.