Comments (6)
- What does the migration do?
- What do we migrate?
- How does the process of migration look like?
from nussknacker.
Are you sure all of these questions are relevant within the scope of this issue? I mean, here we are dealing with boundary of some bounded context where I think we should at least treat actual migration process of the target backend as black box. Nevertheless I will try to answer your questions.
The process of migration of scenario between two environnents involves frontend of source Nussknacker, backend of source Nussknacker and backend of target Nussknacker. This should shed some light on your question about what the migration do. It just migrate the scenario from primaryEnvironment to secondaryEnvironment. I will refer to them as source / target environments. Currently the source backend more or less just forwards the requested data from frontend (containing for example scenario graph) and the target backend is the one who executes the actual migration. The problem is when two backends runs on different versions and the requested data might turn out to be incompatible. Typical issue is newer version of Nussknacker requires more fields as its request data and older (source) Nussknacker cannot be aware of this fact.
There are two natural solutions how to deal with that. We may set up external proxy which will do the actual translation. Or we can version the API manually by storing multiple DTOs. Each of them has its pros and cons: the proxy approach does not require additional DTOs in code but required external module which has to be maintained. The end to end testing also may become more problematic. The pros and cons of manual versioning are basically reverse of the pros/cons of the proxy solution.
Each of these approaches may be scalled in various directions. The proxy module may transform DTOs from other subdomains and their corresponding APIs while manual versioning may track DTOs back in time up to one, or more versions.
I hope I described it without unnecessary technical complexities. Let me know what you think :)
from nussknacker.
Wow, this is super relevant. If I have to review the code I need to know what the feature does and what are the requirements. TBH any feature should start with these.
Additional questions:
- do we want to migrate scenarios from the older version to the newer version only, right?
- let's suppose the current version is 1.14. In this version, we should maintain the current migration request DTO, and the previous version migration request DTO, right? It means that in the next version (1.15) the current migration request from v1.14 becomes the previous migration request DTO and the current one should be defined. Am I right?
from nussknacker.
Yeah, I just thought you want to know the internals of migration. So we are on the same page.
- I assumed bidirectional flow. Altough in the description above I assumed scenario where source has older version but it does not have to be the case.
- Yes, you are right. It may be summarized as:
1.14 | 1.15 | 1.16 |
---|---|---|
A | B | C |
B | C | D |
from nussknacker.
This approach seems to be a little bit problematic.
Let's consider another approach:
the migration request will have:
- version of the scenario (V1)
- the scenario in the format specified by the version (V1)
Then the service that is responsible for converting the scenario from V1 format to V2 (the format of a given instance).
So we have to provide a conversion: V1 -> V2. We could easily maintain conversion between several versions.
Eg.
instance 1: v1.12,
instance 2: v1.14
migration request from instance I1:
POST /migrate
{
"version": "1.12",
"scenario": {
// format specific for the v1.12
}
}
in instance I2 the request is being handled. The instance has the following conversions:
- v1.12 -> v1.13
- v.1.13 -> v1.14
To migrate v1.12 to v1.14 both conversions have to be applied.
When we develop the v1.15 the new conversion will be added (no modifications, just a new thing is added). Some test will keep an eye on the existence of proper conversion (to make sure the new conversion is added for a new version).
WDYT?
from nussknacker.
Ok, if we want to version more API versions then it is better solution. My approach was just a hack (possible because I versioned only one version back in the time). So I propose to define
sealed trait NuVersion {
def prevVersion: Option[NuVersion]
def nextVersion: Option[NuVersion]
}
case object Version1_14 extends NuVersion
case object Version1_15 extends NuVersion
....
And use this to be able to perform transitive DTO translations.
from nussknacker.
Related Issues (20)
- Support for new Flink Source and Sink inferfaces
- Migration of a new scenario throws an error "No scenario X found"
- Fix Either decoder so it works with Either[T,T]
- Build Error: while running C:\NusskNacker\staging\nussknacker-staging\designer\runServer.sh HOT 20
- Map variable nodes don't show field errors anywhere in node edit popup HOT 1
- #this variable and it's content is not suggested in nested projection/filtering expression HOT 1
- Scenario authoring documentation
- Provide Designer and lite-runtiem ARM docker images HOT 1
- Support methods on arrays
- OpenAPI - Omitting nested optional fields in request requires conversion to Unknown of parent record
- Request for Task-Level Parallelism Configuration in Future Nuccassker Release HOT 3
- Compare dialog crashes when clicked when it has been opened in a specific way
- Expand display mechanism of Custom Actions
- Add endpoint for migration between environments
- Topic Name Convention with Azure Schema Registry
- Extend capability to type maps with non string keys
- BasicAuth Realm issue
- Relax validation of model properties in UIProcessValidator
- Make filter rules of categories to be disjoint union instead of intersection
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nussknacker.