Since the model-server now has a dependency on the model-client we can use it to run more extensive validations before accepting a change.

We have to clean and finish components such as the LightModelClient

This is one of the things:

LightModelClient.NodeAdapter.reference.serialize()

• #49

If there are breaking changes in the meta model then we need a way to migrate existing models.

The abstraction level of the REST API between the advanced-model-client and the model-server is mostly a dumb key-value store. The model-server don't understand the content of the stored data, the client does all the work.

A more specific API that knows how to decode the data could do more advanced validations. Is also required for the ModelQL (#13) and other performance optimizations.

Could be separated into a design and an implementation task.

The MandatoryReferenceAccessor currently throws a RuntimeException when a mandatory reference is missing:

https://github.com/modelix/modelix.core/blob/main/metamodel-runtime/src/commonMain/kotlin/org/modelix/metamodel/ReferenceAccessor.kt#L27

When propagating changes to clients, catching RuntimeException is too generic. We should be more specific here, for example by throwing our own, e.g. MandatoryReferenceMissingException.

This is also applicable to other parts in the code, for example:

https://github.com/modelix/modelix.core/blob/main/metamodel-runtime/src/commonMain/kotlin/org/modelix/metamodel/ChildAccessor.kt#L24

Check if this is already supported (write some unit tests).

Grammar Cells in MPS use a parser to deconstruct an expression tree into tokens and to rebuild a new tree that has the same textual representation, but respects the operator precedence rules.

In case of parentheses the editor supports inserting text without immediately creating a node for it (it's an annotation, so while it actually is a node it's one that has no semantic meaning yet). It tries to parse the tokens into a valid expression tree until the users inserts a matching pair of parentheses.

This could replace the currently manually coded data classes to ensure that the specification and the code actually match.

The new high level API v2 provided by the model-server should be specified to have a documentation and contract what this API provides. As it will primarily be based on WebSockets, AsyncAPI has to be used instead of OpenAPI

This is a breaking change. Existing models cannot be read anymore.
This might be a good opportunity to introduce versioning and migration of models.

BulkQuery is used to reduce the number of requests send to the server. This is not yet used in CLTree.visitChanges.

The usage of the plugin is currently a bit ugly. See

Gradle allows to load plugins like this:

plugin {
  id("my.plugin.id")
}

but it has to conform to some naming conventions. When using gradle init to generate a new plugin project then this works as expected. It was done when this plugin was created: https://github.com/modelix/modelix.core/tree/a146e344ae3135b763a3bfd710d00dfdc33e72ba/metamodel-gradle , so this can be used to see how to fix it.

Users want to execute custom code on model changes. We need to add listener support for this feature.

Before the light model server can be used it needs some unit test to ensure the synchronisation works.

Should be a different spec than for the high-level API. Otherwise, this API might become to prominent.

When browsing a model-server repository, one generally wants to be able to browse existing concepts. This could be similar to the logical view in MPS (meaning read & write).

Another example is the google drive browser.

Side transformations and node substitutions can be implemented with the same infrastructure. There are left/right/center transformations.

From the perspective of defining actions it's useful to distinguish between "replacing" a cell and "inserting between" two cells. Distinguishing between left and right transformations as in MPS while defining an action is not useful. You could specify that an action should be available in the gap before or after a node/cell, but triggering it should be possible from both sides of that gap.

To make the difference clear the action locations should be named before/after instead of left/right.

Hi,

I just try to run model-server with ../gradlew run command. But it does not start. I always got an error related with keycloak.

`11:11:11.517 [main] INFO ktor.application - Autoreload is disabled because the development mode is off.
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.modelix.authorization.KtorAuthUtilsKt$installAuthentication$1$3.invoke(KtorAuthUtils.kt:55)
at org.modelix.authorization.KtorAuthUtilsKt$installAuthentication$1$3.invoke(KtorAuthUtils.kt:54)
at io.ktor.server.auth.jwt.JWTAuthKt.jwt(JWTAuth.kt:326)
at org.modelix.authorization.KtorAuthUtilsKt$installAuthentication$1.invoke(KtorAuthUtils.kt:54)
at org.modelix.authorization.KtorAuthUtilsKt$installAuthentication$1.invoke(KtorAuthUtils.kt:41)
at io.ktor.server.auth.Authentication$Companion.install(Authentication.kt:98)
at io.ktor.server.auth.Authentication$Companion.install(Authentication.kt:94)
at io.ktor.server.application.ApplicationPluginKt.install(ApplicationPlugin.kt:98)
at org.modelix.authorization.KtorAuthUtilsKt.installAuthentication(KtorAuthUtils.kt:41)
at org.modelix.model.server.Main$main$ktorServer$1.invoke(Main.kt:247)
at org.modelix.model.server.Main$main$ktorServer$1.invoke(Main.kt:245)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:323)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:312)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartup(ApplicationEngineEnvironmentReloading.kt:340)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:312)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:149)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:279)
at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:185)
at org.modelix.model.server.Main.main(Main.kt:290)
Caused by: java.lang.IllegalArgumentException: Configuration URL can not be null.
at org.keycloak.authorization.client.AuthzClient.(AuthzClient.java:253)
at org.keycloak.authorization.client.AuthzClient.create(AuthzClient.java:94)
at org.modelix.authorization.KeycloakUtils.(KeycloakUtils.kt:35)
... 19 more
[11:11:20] New version is available at ignite.apache.org: 2.14.0
<============-> 94% EXECUTING [1m 21s]

:model-server:run`

Is there any solution ?

Currently the client pulls the full ktor dependency for a HTTP server. It would be helpful to remove/reduce this dependency to another small HTTP server.

High level documentation of the ideas.
This includes ModelQL.

Changing the text of a property cell is the simplest editing gesture. After rendering readonly text this is the first step towards making it editable. It requires placing the caret and handling keyboard and mouse input.

Clients applications of the model-server might have to inform users about things like lost updates. The model-server must make it possible for them to detect such situations.

The model-server does not yet fully implement the ModelQL.

Currently we have to specify the health_check in each compose file manually. If the model-server image was making use of the HEALTHCHECK we could leave that responsibility to modelix.

Example here:
https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator-gradle-plugin/README.adoc#openapigenerate-1

The current prototype high-level v1 REST + WebSocket API is basically unused and should be removed before consumers occur.

The separation just has historical reasons. The CP... classes were serialized using GSON, but after replacing it with a more efficient custom serialization format, the separation is not required anymore.

https://github.com/modelix/modelix.core/tree/main/model-client/src/commonMain/kotlin/org/modelix/model/persistent
https://github.com/modelix/modelix.core/tree/main/model-client/src/commonMain/kotlin/org/modelix/model/lazy

When the light-model-server sends an update after some model change it reruns the whole ModelQL query. For big query results this can be a performance issue.

500: java.lang.RuntimeException: No deserializer found for: mps-model:r:ab82085c-dcbe-4cb8-b44c-5b62f62aa2f7(Common)

java.lang.RuntimeException: No deserializer found for: mps-model:r:ab82085c-dcbe-4cb8-b44c-5b62f62aa2f7(Common)
at org.modelix.model.lazy.INodeReferenceSerializer$Companion.deserialize(INodeReferenceSerializer.kt:45)
at org.modelix.model.lazy.CLTree.getReferenceTarget(CLTree.kt:354)
at org.modelix.model.api.TreePointer.getReferenceTarget(TreePointer.kt:73)
at org.modelix.model.operations.SetReferenceOp.apply(SetReferenceOp.kt:26)
at org.modelix.model.operations.UndoOp$captureIntend$1.invoke(UndoOp.kt:46)
at org.modelix.model.operations.UndoOp$captureIntend$1.invoke(UndoOp.kt:43)
at org.modelix.model.api.TreePointer.computeWrite(TreePointer.kt:32)
at org.modelix.model.operations.UndoOp.captureIntend(UndoOp.kt:43)
at org.modelix.model.operations.UndoOp.captureIntend(UndoOp.kt:29)
at org.modelix.model.operations.RevertToOp$captureIntend$1.invoke(RevertToOp.kt:32)
at org.modelix.model.operations.RevertToOp$captureIntend$1.invoke(RevertToOp.kt:30)
at org.modelix.model.api.TreePointer.computeWrite(TreePointer.kt:32)
at org.modelix.model.operations.RevertToOp.captureIntend(RevertToOp.kt:30)
at org.modelix.model.operations.RevertToOp.captureIntend(RevertToOp.kt:25)
at org.modelix.model.operations.RevertToOp.apply(RevertToOp.kt:18)
at org.modelix.model.operations.OTWriteTransaction.apply(OTWriteTransaction.kt:35)
at org.modelix.model.operations.OTWriteTransactionKt.applyOperation(OTWriteTransaction.kt:158)
at org.modelix.model.server.HistoryHandler$revert$1.invoke(HistoryHandler.kt:67)
at org.modelix.model.server.HistoryHandler$revert$1.invoke(HistoryHandler.kt:66)
at org.modelix.model.api.IBranch$runWriteT$1.invoke(IBranch.kt:30)
at org.modelix.model.api.IBranch$runWriteT$1.invoke(IBranch.kt:30)
at org.modelix.model.api.ContextValue.computeWith(ContextValue.kt:38)
at org.modelix.model.api.PBranch.runWrite(PBranch.kt:53)
at org.modelix.model.operations.OTBranch.runWrite(OTBranch.kt:94)
at org.modelix.model.api.IBranch$DefaultImpls.runWriteT(IBranch.kt:30)
at org.modelix.model.operations.OTBranch.runWriteT(OTBranch.kt:22)
at org.modelix.model.server.HistoryHandler.revert(HistoryHandler.kt:66)
at org.modelix.model.server.HistoryHandler$init$1$3.invokeSuspend(HistoryHandler.kt:56)
at org.modelix.model.server.HistoryHandler$init$1$3.invoke(HistoryHandler.kt)
at org.modelix.model.server.HistoryHandler$init$1$3.invoke(HistoryHandler.kt)
at io.ktor.server.routing.Route$buildPipeline$1$1.invokeSuspend(Route.kt:116)
at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)

The code at

https://github.com/modelix/modelix-samples/blob/feature/light-model-server/rest-api-model-ql/src/main/kotlin/org/modelix/sample/restapimodelql/LightModelClientWrapper.kt#L75

should be migrated to the LightModelClient into a function such as waitForRootNode which blocks.

Also consider adding a client using ktor default client:

https://github.com/modelix/modelix-samples/blob/main/rest-api-model-ql/src/main/kotlin/org/modelix/sample/restapimodelql/LightModelClientWrapper.kt#L39

Deserialize Method in IConceptReferenceSerializer throws RuntimeException if a language is missing.
This results in a long debugging turnover time when multiple languages are missing because you have to add each language individually. Ideally you get a list of all missing languages at once.

(originally opened here modelix/api-gen#16)

Relevant Stacktrace:
java.lang.RuntimeException: No deserializer found for: mps:0e530434-6261-4ef4-98d8-ebfa2f94dde0/5601080448567809140 at org.modelix.model.lazy.IConceptReferenceSerializer$Companion.deserialize(IConceptReferenceSerializer.kt:61) at MPSRemoteNode.getConcept(MPSRemoteNode.kt:20) at org.modelix.mps.apigen.runtime.MPSLanguageRegistry$Companion.getInstance(Language.kt:36) at some.path.web.dashboard.routes.ComponentsKt.softwareComponents(Components.kt:384) at some.path.web.dashboard.routes.ComponentsKt$listSoftwareComponents$1.invokeSuspend(Components.kt:30) at some.path.web.dashboard.routes.ComponentsKt$listSoftwareComponents$1.invoke(Components.kt) at some.path.web.dashboard.routes.ComponentsKt$listSoftwareComponents$1.invoke(Components.kt) at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:248) at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:116) at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:136) at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:78) at io.ktor.routing.Routing.executeResult(Routing.kt:155) at io.ktor.routing.Routing.interceptor(Routing.kt:39) at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:107) at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt) at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt) at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:248) at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:116) at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:136) at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:78) at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:127) at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt) at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt) at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:248) at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:116) at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:136) at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:78) at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:123) at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt) at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt) at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55) at kotlinx.coroutines.BuildersKt__Builders_commonKt.startCoroutineImpl(Builders.common.kt:194) at kotlinx.coroutines.BuildersKt.startCoroutineImpl(Unknown Source) at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:134) at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:56) at kotlinx.coroutines.BuildersKt.launch(Unknown Source) at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:43) at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:34) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda-1$lambda-0(NettyApplicationEngine.kt:251) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:834)

• Typescript generator
• Kotlin generator
• Java generator

There are other option to render templated HTML, eg. https://www.thymeleaf.org/. Lets build a prototype to discuss pro/cons.

Or make sure that they are handled properly. I don't think this is the case yet.

https://github.com/modelix/modelix.core/blob/main/model-client/src/commonMain/kotlin/org/modelix/model/lazy/RepositoryId.kt

The class CPOperationsList stores all the operations that were applied to a version. If there are lots of changes this list can become long and the entry stored on the model-server becomes big. This might be a problem that can be avoided by splitting a long list into a tree like data structure.

Currently, the light-model-client is connected via WebSockets. If the client is not interested in updates the overhead of tracking dependencies and installing listeners can be removed. A simple REST API for reading and writing should be implemented, but with the serialization format.

The implementation in [1] is basically the first version of [2]. [2] is more scalable. An integration of [2] with MPS will allow to incrementally compute results of ModelQL.

[1] https://github.com/JetBrains/MPS-extensions/tree/master/code/shadowmodels/solutions/de.q60.mps.incremental.runtime
[2] https://github.com/modelix/incremental

One reason for performance issues is that nodes are loaded lazily while traversing the model. The purpose of the ModelQL is to load all the required data before working with it. This is also useful for the advanced model client/server.

In case the new high-level v2 API includes REST endpoints, it might be beneficial for clients and for tooling support to generate an OpenAPI specification on the fly as a lot more tooling is available for OpenAPI.