Comments (2)
Based on the context provided, it seems you are looking to improve the performance of Elsa Workflows by addressing the issue of redundant data storage, particularly in the WorkflowExecutionLogRecords
table. The SerializedPayload
column in this table stores a large amount of data, which can lead to rapid database growth and potentially impact performance due to the size of the data being stored and transferred.
To address this issue, you are proposing to remove redundant data from the SerializedPayload
of the WorkflowExecutionLogRecords
table. This would involve not storing the entire journal data for each activity, which can include outcomes, exceptions, and other journalization data.
As an alternative, you've considered allowing users to configure what gets stored in the journal, particularly for activities that create outputs. This could be managed through the Persistence Log Record Module, where users can select which outputs to persist.
The impact of this enhancement would be significant storage savings and improved performance due to reduced data transfer to the persistence store. You've provided a benchmark showing that the size of a record could be reduced significantly, from 683 bytes to 505,923 bytes, in the case of an HTTP sample.
To proceed with this enhancement, you would need to evaluate the current implementation and identify where the redundancies occur. Then, you would need to modify the logging mechanism to either remove the redundant data or provide configuration options for users to select what data should be stored.
This change would affect any use case where activities generate outputs that are currently being stored in the SerializedPayload
. Activities like Elsa.FlowSendHttpRequest
, which add parsed content of HTTP calls to the journal, would be particularly affected.
In summary, the proposed enhancement aims to optimize data storage and improve performance by reducing the amount of redundant data stored in the WorkflowExecutionLogRecords
table. This would involve changes to the logging mechanism and potentially adding user-configurable options for data persistence.
This comment was generated by Glime.
from elsa-core.
closed as done with #5266
from elsa-core.
Related Issues (20)
- [BUG] Issues with POST and GET Execute APIs HOT 1
- [ENH] MongoDB storage for alterations HOT 4
- [FEAT] Postgresql adapter for Hangfire
- [ENH] Optimize Workflow Import by Storing Raw JSON to Avoid Unnecessary Serialization
- [ENH] Enable and Disable MassTransit Consumers HOT 1
- Third-Party Integration Activities HOT 1
- [CHORE] Configure CI/CD with Renovate Bot HOT 1
- Make `Newtonsoft.Json` private? HOT 1
- What is the best place to implement clean-up logic for ActivityExecutionContext HOT 1
- ELSA 3.1.3 Getting Error when i call get /workflow-definitions endpoint HOT 2
- [BUG] MassTransit DisableConsumers Fails to Properly Disable Consumers and Disables Necessary Consumers HOT 1
- [CHORE] Elsa 3.0 设计器中 Fork的属性Branches如何设置? HOT 1
- [ENH] Enable External Interaction with Publish Event and Event Activities HOT 1
- [BUG] FailedToConnect activity in SendHttpRequest is not executed HOT 3
- [BUG] Race condition in Bookmark Management HOT 3
- [BUG] Mixed Content block after deployed to kubernetes HOT 2
- [BUG] Duplicate executions when using the default scheduling provider in single-node environment HOT 1
- [ENH] Add File Download Capability to SendHttpRequest Activity HOT 1
- [BUG]An unhandled error has occurred. Reload HOT 2
- [BUG] ELSA 3.2 rc1 with latest studio. Null reference exception when editing JavaScript expressions HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elsa-core.