Is your feature request related to a problem? Please describe.
If I use ROW_NUMBER() in SqlQueryable, I always have to CAST it to INT beacuse Rhetos doesn't support BIGINT(Int64).
This workaround will only work if SqlQueryable returns less than 2,147,483,647 rows.

Describe the solution you'd like
Rhetos should support Int64 data types. ConceptKey proposition: LongInteger.

We would like to use a third party assembly in RestService.cs generated code. Is there a way to add a using statement in a generated RestService.cs file?

In our project which is built on top of Rhetos, there is a huge need of having unit tests that cover the crucial business logic/rules. Let's say we have business rules spread over 3 major places inside our application:

1. DSL scripts: this is where major of our business rules live, but unfortunately we couldn't find any proper mechanisms to have them unit tested.

2. .NET class library: some of complex logic is maintained here. It is fine to write unit tests for medium complex logic. However, I found it's really tricky/even impossible to mock a DomRepository, because it's too tightly coupled by design. I think a good solution for this problem should be injecting an interface of DomRepository, instead of a concrete class as it is currently.

3. Also a .NET class library, that extend Rhetos DSL by defining new keywords: we rarely have to implement this so we don't have much experience on how to organize unit tests for it. Some advice would be more than welcomed.

When trying to delete entity that has a UniqueReference entity without CascadeDelete concept applied, insufficient error message is returned. There is no info about entity that references aforementioned entity.

Logged error from CommandsTrace:

<!--2018-09-05 12:53:59.3470 ProcessingEngine CommandsWithClientError-->
<?xml version="1.0" encoding="utf-16"?>
<ExecutionCommandsLogEntry xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Rhetos.Processing">
  <Commands xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays" xmlns:d2p2="Rhetos.Processing.DefaultCommands" i:type="d2p2:Rhetos.Processing.DefaultCommands.SaveEntityCommandInfo_005b_005d">
    <d2p1:anyType i:type="d2p2:Rhetos.Processing.DefaultCommands.SaveEntityCommandInfo">
      <DataToDelete xmlns="http://schemas.datacontract.org/2004/07/Rhetos.Processing.DefaultCommands">
        <d2p1:anyType xmlns:d5p1="MyModule" i:type="d5p1:MyModule.MyEntity">
          <ID xmlns="http://schemas.datacontract.org/2004/07/Rhetos.Dom.DefaultConcepts">672529a4-72b2-422f-9a47-6e975512c1ff</ID>
          <CreatedAt i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <CreatedByID i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <CreatedForID i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <KrajIzvjestajnogRazdoblja i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <NominatedZnsStatusID i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <Oznaka i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <ProjektID i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <RedniBroj i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <SnapshotCreatedAt i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <SnapshotGroupID i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <VidljivInstituciji i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
          <ZnsVrstaZahtjevaID i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/MyModule" />
        </d2p1:anyType>
      </DataToDelete>
      <DataToInsert i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/Rhetos.Processing.DefaultCommands" />
      <DataToUpdate i:nil="true" xmlns="http://schemas.datacontract.org/2004/07/Rhetos.Processing.DefaultCommands" />
      <Entity xmlns="http://schemas.datacontract.org/2004/07/Rhetos.Processing.DefaultCommands">MyModule.MyEntity</Entity>
    </d2p1:anyType>
  </Commands>
  <ExecutionId>039b77bb-917b-4191-ae3f-28af36ce3309</ExecutionId>
  <UserInfo>73316707963,::1 port 60859</UserInfo>
</ExecutionCommandsLogEntry>
<!--2018-09-05 12:53:59.3660 ProcessingEngine CommandsWithClientError-->
<?xml version="1.0" encoding="utf-16"?>
<ExecutionResultLogEntry xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Rhetos.Processing">
  <ExecutionId>039b77bb-917b-4191-ae3f-28af36ce3309</ExecutionId>
  <Result>
    <CommandResults />
    <Success>false</Success>
    <SystemMessage i:nil="true" />
    <UserMessage>[[[It is not allowed to delete a record that is referenced by other records.]]]</UserMessage>
  </Result>
</ExecutionResultLogEntry>

Possible solution:
Include CascadeDelete and FK constraint error handling in UniqueReference concept.

Releases of versions should include prebuilt Rhetos server dlls.
I want to take Rhetos and run it immediately, I don't want to bother with setting up development environment and building binaries from source. It can stay as part of the release, but I feel as it would make a life a lot easier for anyone just trying out the technology to have prebuilt binaries.

Generated or generic GUI that allows CRUD operations on all data structures.

The GUI is intended for rapid application prototyping during development, and administration during production. It should use standard Rhetos permissions, so that any user can access it without security risks.

It can be improved to support other common features: executing action, downloading report, master-detail entities, recomputing persisted data, etc.

Hello,

We would like to add a column to Common.PrincipalHasRole and make it part of existing unique constraint:

CREATE UNIQUE CLUSTERED INDEX [IX_PrincipalHasRole_Principal_Role] ON [Common].[PrincipalHasRole]
(
	[PrincipalID] ASC,
	[RoleID] ASC
)

In the dsl we have:

Module Common
{
    Entity SecurityUnit
    {
        ShortString Name { Unique; }    
    }

    // Extending existing table so when we add role to user, we need to add a security unit as welll
    Entity PrincipalHasRole
    {
        Reference SecurityUnit Common.SecurityUnit;
    }
}

How can we set "Reference SecurityUnit Common.SecurityUnit" to be part of existing unique constraint?

Hi, I am not sure if this is already on the roadmap or not but here goes. I think Rhetos needs a centralized command line tool to execute at least all the common works one user may do when he uses Rhetos. Right now, the scripts and the configurations are so discrete and scattered all around (well, at least from my viewpoint). For a reference/example, take a look here at dotnet CLI tool. Maybe it's an rhetos.exe to be downloaded. Then, we add it to $PATH and excute the commands in cmd or Powershell to help with the development using Rhetos.

Haven't thought thoroughly but here are a few example commands:

• rhetos new [--version]: Create a new template project with Rhetos binaries and a sample Nuget package with one or two .rhe scripts.
• rhetos config view: A command to see all configurations of the current project at the path (We may need to discuss a way to detect a Rhetos project). The configurations can be database connections, and used packages. There can be some switches to extend this command as well. For example, rhetos config view <package_name> to view configurations of a specific package, or rhetos config view all for all packages found within the project.
• rhetos config add <package_name> <package_to_be_added>: This command adds a new dependency to a package. This one can cover the actions needed in RhetosPackages.config and RhetosPackageSources.config. Should also go with rhetos config modify and rhetos config remove, etc.
• rhetos run: Apply all the latest changes of the current project and start a new web server at a random port (quite similar to the way dotnet run or npm start behaves).
• ...

Motivation:

• Roslyn compiler works 2 or 3 times faster then the existing .NET compiler currently used by Rhetos (from .NET Framework 4.5.1). Compiling the generated application is the slowest step of the deployment for some very large enterprise applications developed on Rhetos, so this improvement would significantly increase the deployment time.
• Using the new C# features (C# 6.0 and above) in DSL C# code snippets.
• In the long run, we expect to migrate to .NET Core, and this upgrade may solve part of that goal.

Scope:

• Migrate Rhetos and CommonConcepts package from .NET 4.5.1. to latest version (currently .NET 4.7.2).
• Switch from CSharpCodeProvider to Roselyn compiler.
• Check if there are any backward compatibility issues to be expected with other packages or existing applications, considering the new version of framework and compiler.

Changing "EntityFramework.UseDatabaseNullSemantics" should affect the server at run-time to allow easier testing. Currently a developer must run DeployPackages.exe for the change to be applied although there is no real technological reason for this limitation.

Design considerations:
Instead of generating the fixed code that sets UseDatabaseNullSemantics, the EntityFrameworkContext.Initialize() method should read the value from IConfiguration directly.

Allow application developers to define a common interface that multiple entities can implement as a subtype. Reading data through the interface should return union of all implementations.

A.k.a. composite entity, union, interface.

Notes:

1. An entity should be allowed to implement multiple interfaces (be a subtype of more than one polymorphic entity).
2. Application developer should be allowed to add a new subtype for existing polymorphic entity by adding new plugins, without modifying existing DSL scripts (without modifying the polymorphic entity's definition).
3. Application developer should be allowed to add a new property to the polymorphic entity and to the existing subtypes by adding new plugins, without modifying any of the existing DSL scripts.

Currently this features was usually implemented in applications by manually creating an SqlQueryable that selects UNION ALL from all the subtype entities. That implementation did not allow points 2. and 3. from the notes above.

When exists self reference on entities for which row permissions are to be applied, error occurs when reference is included in checking row permissions.
Workaround is to use extension (SqlQueryable or similar) in which identifier for self reference (or its properties) is declared as value type (Guid).

Describe the bug

• I run CreateAndSetDatabase.exe by command line outside "bin" folder and it can't copy file Template.ConnectionStrings.config.
• It is same with CreateIISExpressSite.exe.

To Reproduce
Run command line outside "bin" folder like:

.\bin\CreateAndSetDatabase.exe "(local") "TestDatabase"

Error message

Unhandled Exception: System.IO.FileNotFoundException: Could not find file 'Template.ConnectionStrings.config'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.InternalCopy(String sourceFileName, String destFileName, Boolean overwrite, Boolean checkHost)
   at CreateAndSetDatabase.Program.Main(String[] args) in C:\My Projects\Rhetos\Source\CreateAndSetDatabase\Program.cs:line 135

Expected behavior
It could copy Template.ConnectionStrings.config for creating ConnectionStrings.config.
I think problem is at here

If a Rhetos application uses packages "Rhetos.ODataGenerator" (v2.2.0) and "Rhetos.LightDMS" (1.6.0), DeployPackages.exe will fail with an exception Could not load file or assembly 'Microsoft.Data.OData, Version=5.6.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies..

Unique constraint error should report more information on the error (entity name, property name, ID) to help user or developer fix the problem. Currently a generic error is returned: "It is not allowed to enter a duplicate record.". This can be a problem when multiple entities are modified in a single command.

Until the problem is solved, a following workaround can help for debugging: In web.config, inside the rules group, add the following lines:

  <logger name="ProcessingEngine" minLevel="Trace" writeTo="TraceLog">
    <filters>
      <when condition="not starts-with('${message}','Command execution error: Rhetos.UserException: It is not allowed to enter a duplicate record.')" action="Ignore" />
    </filters>
  </logger>

Note that the additional logging may affect server performance.

Describe the bug
Some of the generated database schemas have authorization (owner) set to "dbo", but some have a user account that executed a deployment. We have not experienced any issues with these schemas during the application runtime, but it gave us some extra work while maintaining the SQL Server, when were removing the obsolete user accounts.

Expected behavior
I am not sure what conditions result with different schema owners, but the problem could be solved by setting the schema authorization to dbo explicitly.
Instead of CREATE SCHEMA Common, the Module concept should create query CREATE SCHEMA Common AUTHORIZATION dbo.

Upgrading existing applications
If we just change the Module concept, this will result in regenerating all the modules and tables in the database. Data will not be affected, but the upgrade process could take hours or more for the existing applications with very large databases.
This is why we need to additionally create a data-migration script that will make this change in the existing schemas and update the Rhetos.AppliedConcept entries (add "AUTHORIZATION dbo") so the Rhetos framework knows that the change was already made.

Is your feature request related to a problem? Please describe.

Describe the solution you'd like
Including the xml files with the summary of types in the Rhetos nuget package.

Describe alternatives you've considered

Additional context
This would come very handy when implementing a plugin for VSCode so the description of the concpet could be displayed when writing the code.

Describe the bug
When I add Rhetos.LightDMS, or some other nuget, which has DataMigration folder, in my package dependencies, Rhetos doesn't unpack DataMigration folder, only DslScripts is unpacked (in PackagesCache folder). I also checked Rhetos.DataMigrationScript table and it doesn't contain any of the Rhetos.LightDMS migration scripts.
NuGet Version: 4.6.2.5055

To Reproduce
Add new package and in it's .nuspec file add dependency on Rhetos.LightDMS (or some other plugin which has DataMigration folder) and run deploy packages. Data migration scripts from Rhetos.LightDMS won't be executed or unpacked

Expected behavior
DataMigration folder should be unpacked and data migration scripts should be executed.

After several projects dealing with time zone differences, which resulted in quite tricky and fragile solutions, I think that Rhetos itself should support DateTimeOffset data type. There are some good reasons to have DateTimeOffset supported:

1. Database should be portable, so it can be restored on any server without having to care about server's timezone. Instead, timezone differences should be handled by the database engine by reading the timezone offset information stored in DateTimeOffset values.
2. There are so many use cases that need precise timestamps: accounting, logging, ... This need obviously can not be handled by the database server nor the application layer. Of course we can make an agreement that every timestamp value in the database is in a specific timezone (let's say UTC), but this approach requires a lot of unnecessary conversions from one timezone to the other. Also, I think handling timezone conversions in application layer is very error-prone, which we should avoid as much as possible.

Is your feature request related to a problem? Please describe.
It would be nice to share the same C# code betwen multiple code snippets inside the rhe scripts.

Describe the solution you'd like
Creating a low level concept that should just create a method on a repository class that could be used like this "domRepository.Module.Entity.Method()"

Describe alternatives you've considered
An Action could be used but I think that this generates addidtional unneceseray code.
It also exposes the action throw the processing engine so you must take care of the permissions.

Is your feature request related to a problem? Please describe.

When SqlQueryable uses a polymorphic's database view (see example below), developer will sometime define the dependency with SqlDependsOn, but this will generated a bug in the application. The polymorphic is internally split into two objects: DataStructure with the polymorphic's name and the SqlObject with the same name that generated the view in database, so the correct dependency defition should be SqlDependsOnSqlObject.
This can easily cause a bug, because the deployment accidentally work in developer's environment (especially if the Polymorphic existed long before the SqlQueryable), but cause an error later (when deploying to another environment).

Module TestPolymorphic
{
    SqlQueryable Q1 "SELECT ID, Name FROM TestPolymorphic.P1"
    {
        ShortString Name;
        SqlDependsOn TestPolymorphic.P1; // This does not work, but it should.
        // SqlDependsOnSqlObject TestPolymorphic.P1; // This works.
        // AutoDetectSqlDependencies;  // This also works.
    }

    Polymorphic P1
    {
        ShortString Name;
    }
}

Deployment of the script above results with error:

FrameworkException: SqlException has occurred:
Msg 208, Level 16, State 1, Procedure Q1, Line 1
Invalid object name 'TestPolymorphic.P1'.

Describe the solution you'd like
For each SqlQueryable that references a Polymorphic, the system should automatically create an equivalent SqlDependsOnSqlObject.

Describe alternatives you've considered
AutoDetectSqlDependencies seems to be working, but on some modules we want to use explicit dependencies.
'SqlDependsOnSqlObject' also works, but the main problem is that developers often write 'SqlDependsOn not knowing that they made a bug, and the bug can cause the deployment to fail long after the feature development has finished.

Allow application developers to define data read and write permissions for each record, based on some record's property or other related data.

Motivation:

• Allow developers to simply create view-models or DTOs, using only declarative code, without need for writing middleware service or custom web methods.
• The existing REST service exposes only simple objects by default. For example, if developer need to reads or write a master-detail object in a single request, a custom code must be developed, sometimes with custom data serialization withing already serialized web requests.

Scope:

• Implement a Rhetos plugin that exposes new web API or extends the REST API, with GraphQL interface that allows CRUD operations on Rhetos entities and other data structures. We could implement one or both in this options:
• Option A) Generic API - Standard GraphQL implementation.
• Option B) Predefined API – Developer can use GraphQL on backend to specify new REST methods for complex data transfer objects. Currently the complex DTOs are implement by writing custom server Actions or in middleware view-model.

Risks and issues:

• Performance – Affects only option A. Might be reduced by implementing artificial limitations and command timeout.
• Security – GraphQL queries should be interpreted before the Rhetos security layer (ProcessingEngine), and result with one or more ProcessingEngine commands. For example, if multiple entities are requested in a single GraphQL query, a separate internal ProcessingEngine command can be executed for each entity, allowing the existing security system to work on this request.

We have a problem when deploying Rhetos to big dataset because recomputing tables takes up a lot of time (3+ hours) and can break the deploy (TFS build max run time exceed). For now we have been setting NORECOMPUTE on table Common.KeepSynchronizedMetadata in a migration skript. But that is not a good solution for two reasons. Sometimes developers forget to change the guid of the migration skript or are not aware that large recomputation will occur. The other reason is recomputation are skipped on small datasets that so developers need to execute recomputation manually.
Can I suggest a new flag on DeployPackages.exe that skips recomputation so we can switch off that feature when needed.

Hello,

We built a new concept that sits on top of DataStructure.

    //defines this class as a plugin for rhetos
    [Export(typeof(IConceptInfo))]
    //defines the concept keyword in dsl scripts
    [ConceptKeyword("Inherits")]
    public class InheritsInfo : IConceptInfo
    {
        [ConceptKey]
        public DataStructureInfo DataStructure { get; set; }

        public string DataStructureName { get; set; }
    }

How can we apply the same concpet to Compute?

LockProperty concept should work with any kind of filter, either defined by a developer (with ItemFilter, e.g.) or a standard predefined filter such as FilterAll.

The following usage of LockProperty should result will all records in the entity being locked, instead the deployment fails with an error described below.

Module TestLockItems
{
    Entity LockAll
    {
        ShortString Name { LockProperty 'FilterAll' 'All is locked'; }
    }
}

DeployPackages.exe fails with the error:

The best overloaded method match for 'Common.OrmRepositoryBase<Common.Queryable.TestLockItems_Simple3,TestLockItems.Simple3>.Filter(System.Linq.IQueryable<Common.Queryable.TestLockItems_Simple3>, System.Collections.Generic.IEnumerable<System.Guid>)' has some invalid arguments
At line 69614, column 39, file 'C:\My Projects\Rhetos\Source\Rhetos\bin\Generated\ServerDom.Repositories.cs',
after: "... (changedItems != null && changedItems.Length > 0) { var lockedItems =",
before: "_domRepository.TestLockItems.Simple3.Filter(this.Query(changedItems...".

Argument 2: cannot convert from 'Rhetos.Dom.DefaultConcepts.FilterAll' to 'System.Collections.Generic.IEnumerable<System.Guid>'
At line 69614, column 133, file 'C:\My Projects\Rhetos\Source\Rhetos\bin\Generated\ServerDom.Repositories.cs',
after: "...Items.Simple3.Filter(this.Query(changedItems.Select(item => item.ID)),",
before: "new FilterAll()); if (lockedItems.Count() > 0) throw new Rhetos.Use...".

So i have MainRole which inherits MyClaim from RoleGroup and few different mail roles also inherit that RoleGroup.
I need to remove MyClaim from MainRole but editing RoleGroup is not an option because other main roles have to stay the same.

If i insert a row in RolePermission with RoleID = MainRoleID, ClaimID = MyClaimID, IsAuthorized = 0, will that override the permission which MainRole inherits from RoleGroup?

The existing GeneratePasswordResetToken method cannot allow anonymous user access and cannot easily be used in forgot-password scenario. Another web method is needed for anonymous user (who forgot the password) to send the token to previously registered email address. The same issue occurs whether a user is accessing the Rhetos server directly or using a web application that impersonates the user.

Motivation:

• Simplify or automate the documentation generation for the Rhetos web API.
• Provide tools for developers for interactive use of the Rhetos web API and client libraries.

Scope:

• Create a new Rhetos plugin that provides an OpenAPI implementation for Rhetos REST API (https://github.com/Rhetos/RestGenerator).
• Research what OpenAPI tools can be integrated into Rhetos or used together with the Rhetos OpenAPI implementation.

Is your feature request related to a problem? Please describe.
An entity with History { AllProperties; } has automatically generated filter by DateTime, that returns the list of record versions that were active at that point in time. The problem with this filter is that is not efficient when we only need a subset of records; it always reads all records that were active at that time from the database.

Describe the solution you'd like
To allow efficient data loading for user interface, this filter should be implemented to allow additional filters to be added. Most often this would be used to filter by ID, or filter the detail records by parent ID.

The most practical solution would be to modify the existing filter by DateTime (in EntityHistoryAllPropertiesCodeGenerator and EntityHistoryCodeGenerator) to be implemented as a ComposableFilterBy. Additional research needs to be done to see if it is technically possible to implement this in a efficient way.

Alternatively, a more specific filter might be added to allow filtering by ID and filtering by parent reference (direct or indirect) along with the DateTime.

Describe alternatives you've considered
In the existing applications, developers usually implement their own specific filters separately for each situation to read the data at the specific point in time. This works but requires additional coding that could be avoided.

Is your feature request related to a problem? Please describe.
When optimizing row permissions, SamePropertyValue concept can only be used when a detail or extension has redundant property from a direct parent or a base data structure.

In some situations it would be helpful to allow SamePropertyValue an indirect reference to the parent's parent (or any number of levels), without having to persist the property on the parent.

Describe the solution you'd like
SamePropertyValue concept should be modified to allow more then one level of references in the path. For example, SamePropertyValue in the code below should result with optimized row permissions query when reading records from ParentInfo.

Module TestRowPermissionsSameValue
{
    Entity ParentsParent
    {
        ShortString Name;

        RowPermissions
        {
            Allow MyItems 'context => item => item.Name.StartsWith(context.UserInfo.UserName)';
        }
    }

    Entity Parent
    {
        Reference ParentsParent { Detail; }
        ShortString Name;
    }

    SqlQueryable ParentInfo
    "
        SELECT
            p.ID,
            ParentsParentName = pp.Name,
            ParentName = p.Name
        FROM
            TestRowPermissionsSameValue.ParentsParent pp
            INNER JOIN TestRowPermissionsSameValue.Parent p ON p.ParentsParentID = pp.ID
    "
    {
        Extends TestRowPermissionsSameValue.Parent;
        ShortString ParentsParentName { SamePropertyValue 'Base.ParentsParent.Name'; }
        ShortString ParentName;
    }
}

Describe alternatives you've considered
A workaround for the scenario above would be to add a redundant ShortString ParentsParentName to Entity Parent. This is a valid solution because it could be used to additionally optimize the row permission on Entity Parent, but sometimes the tradeoff of persisting a redundant property is not worth it if the Entity Parent does not have the performance issues, while adding the ParentsParentName property to SqlQueryable ParentInfo is trivial, and there is more chance for the view to have performance issues.

When reading data from server with paging, if a request asks for both records list and the total count, this request will usually result with 2 request to the database.

2 SQL queries are needed if the total count is greater then the paging limit, but if there are less records then the paging limit then the total count can be deduced from the list.

http://localhost/Rhetos/rest/Common/Principal/RecordsAndTotalCount?&top=1000&sort=Name

results with these 2 SQL queries:

SELECT TOP (1000) 
    [Project1].[C1] AS [C1], 
    [Project1].[ID] AS [ID], 
    [Project1].[Name] AS [Name]
    FROM ( SELECT 
        [Extent1].[ID] AS [ID], 
        [Extent1].[Name] AS [Name], 
        1 AS [C1]
        FROM [Common].[Principal] AS [Extent1]
    )  AS [Project1]
    ORDER BY [Project1].[Name] ASC
go
SELECT 
    [GroupBy1].[A1] AS [C1]
    FROM ( SELECT 
        COUNT(1) AS [A1]
        FROM [Common].[Principal] AS [Extent1]
    )  AS [GroupBy1]

Describe the solution you'd like
This reading mechanism is implemented in ServerCommandsUtility class, ExecuteReadCommand method. This method should be optimized to avoid querying "count" if it already loaded less records then the paging count.

Describe alternatives you've considered

Additional context

We are trying to run DeployPackages.exe /DatabaseOnly on a remote test application server from our release agent machine. These are two different computers on different domains. The script that is executed looks like
net use x: \\ipadress\path password /user:username
x:\Rhetos\bin\DeployPackages.exe /DatabaseOnly
net use x: /d

The error is:

2018-09-27T14:00:17.1157500Z [Info] DeployPackages: Skipped deleting old generated files (DeployDatabaseOnly).
2018-09-27T14:00:17.1972167Z [Info] DeployPackages: Skipped download packages (DeployDatabaseOnly).
2018-09-27T14:00:17.5031830Z [Trace] DeployPackages: Loading plugins.
2018-09-27T14:00:18.1560397Z [Error] DeployPackages: System.IO.FileLoadException: Could not load file or assembly 'RestService, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. Operation is not supported. (Exception from HRESULT: 0x80131515)
2018-09-27T14:00:18.1563409Z File name: 'RestService, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' ---> System.NotSupportedException: An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more information.
2018-09-27T14:00:18.1566427Z    at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
2018-09-27T14:00:18.1567840Z    at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
2018-09-27T14:00:18.1568920Z    at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
2018-09-27T14:00:18.1569609Z    at System.ComponentModel.Composition.Hosting.AssemblyCatalog.LoadAssembly(String codeBase)
2018-09-27T14:00:18.1570334Z    at System.ComponentModel.Composition.Hosting.AssemblyCatalog..ctor(String codeBase)
2018-09-27T14:00:18.1571084Z    at Rhetos.Extensibility.MefPluginScanner.<>c.<LoadPlugins>b__4_0(String a) in C:\My Projects\Rhetos\Source\Rhetos.Extensibility\MefPluginScanner.cs:line 94
2018-09-27T14:00:18.1571727Z    at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
2018-09-27T14:00:18.1572328Z    at System.Diagnostics.Contracts.Contract.ForAll[T](IEnumerable`1 collection, Predicate`1 predicate)
2018-09-27T14:00:18.1573141Z    at Microsoft.Internal.Requires.NotNullElements[T](IEnumerable`1 values, String parameterName)
2018-09-27T14:00:18.1573876Z    at System.ComponentModel.Composition.Hosting.AggregateCatalog..ctor(IEnumerable`1 catalogs)
2018-09-27T14:00:18.1574724Z    at Rhetos.Extensibility.MefPluginScanner.LoadPlugins(List`1 assemblies) in C:\My Projects\Rhetos\Source\Rhetos.Extensibility\MefPluginScanner.cs:line 95
2018-09-27T14:00:18.1575686Z    at Rhetos.Extensibility.MefPluginScanner.FindPlugins(ContainerBuilder builder, Type pluginInterface) in C:\My Projects\Rhetos\Source\Rhetos.Extensibility\MefPluginScanner.cs:line 55
2018-09-27T14:00:18.1581114Z    at Rhetos.Extensibility.Plugins.FindAndRegisterPlugins[TPluginInterface](ContainerBuilder builder) in C:\My Projects\Rhetos\Source\Rhetos.Extensibility\Plugins.cs:line 72
2018-09-27T14:00:18.1581952Z    at DeployPackages.AutofacModuleConfiguration.Load(ContainerBuilder builder) in C:\My Projects\Rhetos\Source\DeployPackages\AutofacModuleConfiguration.cs:line 55
2018-09-27T14:00:18.1582410Z    at Autofac.Module.Configure(IComponentRegistry componentRegistry)
2018-09-27T14:00:18.1582805Z    at Autofac.ContainerBuilder.Build(IComponentRegistry componentRegistry, Boolean excludeDefaultModules)
2018-09-27T14:00:18.1583329Z    at Autofac.ContainerBuilder.Build(ContainerBuildOptions options)
2018-09-27T14:00:18.1583787Z    at DeployPackages.Program.GenerateApplication(ILogger logger, Arguments arguments) in C:\My Projects\Rhetos\Source\DeployPackages\Program.cs:line 174
2018-09-27T14:00:18.1584342Z    at DeployPackages.Program.Main(String[] args) in C:\My Projects\Rhetos\Source\DeployPackages\Program.cs:line 69

Has anybody run into this issue and how to we solve it? Tried to 'unblock' assembly but it didn't help. Also running DeployPackages.exe /DatabaseOnly on localy on application test server executes with no errors.

Describe the bug
Rhetos automatically detects some changes in source definition for ComputedFrom...KeepSynchronized and that triggers recompute on the next deployment.
The change is not detected (it should be) if a new computed property is added to the target entity, but the source is not changed (for example the property already exists in the source for the computation).

To Reproduce

1. Create a SqlQueryable that returns one row and two columns (two properties).
2. Create and Entity that has only one of those properties and persists the value from this SqlQueryable, by using ComputedFrom and KeepSynchronized.
3. Run DeployPackages.exe, the data in the Entity will automatically be filled.
4. Add the second propery to the entity and change ComputedFrom definition to persist this property's value from the SqlQueryable, without changing the SqlQueryable.
5. Run DeployPackages.exe, the new property in the Entity will have value null, instead of the value from the SqlQueryable. This is a bug.

Expected behavior
After the second deployment (step 5.) the entity should have a record in database with both properties' values set.

The tutorial should be a part of the project's github wiki pages.

Is your feature request related to a problem? Please describe.
Some of DSL concepts (for example Extends and Detail) result with an on-delete-cascade foreign key in the database. This feature is unreliable because if a record is deleted by cascade delete directly in the database, then the business logic implemented in application layer for will not be executed this record.

Describe the solution you'd like
Any code that adds ON DELETE CASCADE to the database should be removed.
The cascade delete functionality is already also implemented in C# code (see ReferenceCascadeDeleteCodeGenerator class), so the on-delete-cascade foreign key in the database is redundant and should never be used in practice. If there are any features that do not handle cascade delete in C#, it is better for delete to fail, instead of automatically deleting the related records in database, bypassing the application's business logic.

Backward compatibility
This feature should be implemented in a backward compatible way to avoid breaking data-migration scripts and other similar feature in the existing applications. On-delete-cascade should still work on older applications.
See Backward compatible feature implementation in Rhetos and CommonConcepts.

Is your feature request related to a problem? Please describe.
Yes. In one of our projects, we have an Angular front end application communicating with a Rhetos backend server. There are 2 scenarios where API versioning matters:

• We have 2 separate teams working on front end and back end development. That lead to the scenario where deployed front end version is not compatible with back deployed back end version (front end is ahead of back end).
• In production environment, a user may have a cached version of the front end application, and it is not 100% compatible with newest deployed Rhetos back end.

Describe the solution you'd like
Rhetos should have a mechanism to manage the API versions. For example, prefix new changes/modules with version number.

Describe alternatives you've considered
I think for now we can manually prefix the module names with the proper version number. That may break the module structure, make modules less maintainable; but it can do the job.

Additional context
None

A detail entity record should not exist without the parent record, so the detail-master reference (ReferenceDetailInfo) should automatically be a required property.

Is your feature request related to a problem? Please describe.
There are some existing low-level concepts that allow developers to directly insert code in the save method of an entity: OnSaveUpdate, OnSaveValidate, AfterSave and Initialization. The only difference is the position (extension point/tag) inside the save method where the code will be injected.
Two of the concepts are missing, to cover all of the typically used extension points inside the save method: ArgumentValidation and OldDataLoaded.

Describe the solution you'd like
Implement two new concepts ArgumentValidation and OldDataLoaded in the same manner as the four existing concepts mentioned above.

Describe alternatives you've considered
Without these two concepts, developer can create a class (IConceptCodeGenerator plugin) that inserts any code snippet to those extension points in the save method, but this approach requires more coding.

I have a problem where Rhetos orders all of my migration scripts by their full name (PackageName.DataMigrationName) and one script depends on another one but their execution order on empty database is not correct because of their names. Is there a way to change the order or add data migration script dependancy?
Note that data migration scripts are in different packages

Example

Folder A

• Script3
• Script4

Folder B

• Script1
• Script2

(Folder A is alphabeticly in front of Folder B)
Rhetos will execute in this order -> Script3, Script4, Script1, Script2
Wanted order is -> Script1, Script2, Script3, Script4

Is your feature request related to a problem? Please describe.
From my point of view, the main motivation would be potential performance enchantments.
The feature request is not tied to a specific problem.

Describe the solution you'd like
An option to build Rhetos with (latest stable) EF Core version.

Ideally, solution would be build as .NET Standard to allow interoperability with plugins written in .NET Framework. Or it should provide a way to load such plugins.

Hi everyone,
I am facing an issue that if I send requests slowly one by one, every request will finish in a timely manner, let say 100ms. But if I send those request faster (80ms/request) then every request will take much more time to finish or even timeout.
I think the issue is because Rhetos accesses database synchronously, not asynchronous.
Has anyone also had this issue before?

Describe the bug
Let's say DataStructure 'A' uses SqlQueryable 'B' as its ComputedFrom source and 'B' is an extension of 'A'. If AllProperties is used on aforementioned ComputedFrom, Rhetos will try to create a ForeignKey on property 'ID' from 'A' to 'A' which caues a SqlException (Introducing FOREIGN KEY constraint '...' on table 'A' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.)

To Reproduce

Module Music
{
	AutodetectSqlDependencies;

	Entity Artist
	{
		ShortString FirstName { Required; MinLength 5; }
		ShortString FamilyName;
		Money ToursRevenue { MinValue 0; }
		Money TotalRevenue { DenyUserEdit; }

		ComputedFrom Music.ComputeArtist
		{
			AllProperties;
			KeepSynchronized;
		}
	}

	SqlQueryable ComputeArtist
	"
		SELECT
			a.ID,
			TotalRevenue = SUM(rs.Revenue) + a.ToursRevenue
		FROM
			Music.Artist a
			LEFT JOIN Music.Record r ON a.ID = r.ArtistID
			LEFT JOIN Music.RecordSales rs ON r.ID = rs.RecordID
		GROUP BY
			a.ID, a.ToursRevenue
	"
	{
		Extends Music.Artist;
		Money TotalRevenue;
	}

	Entity Record
	{
		Reference Artist { Detail; }
	}

	Entity RecordSales
	{
		Reference Record { Detail; }
		ShortString Country;
		Money Revenue;
	}
}

Error message
Introducing FOREIGN KEY constraint '...' on table 'A' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.

Expected behavior
AllProperties should ignore ID if ComputedFrom source is an extension of target.

Motivation:

• Significantly simplify the development and debugging of C# code in Rhetos applications.
• Improve developer’s learning speed with IntelliSense.
• Most important project in the list, but is not yet ready for development.

Scope:

• Implement an extension for VS Code for Rhetos application development (with code snippets, linting, build integration, integrated error reporting and code formatting.
• Better syntax highlighting and IntelliSense support for C# and SQL code.
• Writing C# code snippet with full user-experience of IDE for C# projects, without copy-pasting or using external projects. Using C# compiler in VS Code for Rhetos C# snippets.
• Build and deployment integrated into VS Code.

Next steps:

• Research and brainstorming.
• Research VS Code T4 implementation; its technology is probably very close to what we need.
• Research Language Server Protocol

Notes:

• The existing plugin (https://github.com/Rhetos/RhetosVSCode) contains only syntax highlighting.

HTTPS is standard today, so it default web config should have https enabled.

Hi,

We have deployed rheots with forms authentication in azure app services . The login page shows up but after submitting username and password the server responds with 404 Not found. The application works fine locally on IIS. Request response in image:

On azure portal anonymous access is enabled and App Service Authentication is Off
Has anybody hosted a rhetos application in azure app services? What configuration is needed to make it work?

    SqlQueryable BookInfo
    "
    SELECT
        k.ID,
        CommentCount = COUNT(kom.ID), --this is a "simple" comment
        FullName = o.Name+ ' : ' + k.Title
    FROM
        BookStore.Book k
        LEFT JOIN BookStore.Comment kom ON kom.KnjigaID = k.ID
        LEFT JOIN BookStore.Person o ON o.ID = k.AutorID
        GROUP BY
        k.ID, o.Name, k.Title
        
    "
    {
        Extends BookStore.Book;
        Integer CommentCount ;
        ShortString FullName;
    }

Build will throw exception because of "simple" part of the comment.

Exception message:

DslSyntaxException: At line 204, column 60, file 'BookStore\DslScripts\BookStore.rhe',
after: "...nfo " SELECT k.ID, CommentCount = COUNT(kom.ID), --this is a simple "",
before: "comment" FullName = o.Name+ ' : ' + k.Title FROM BookStore.Book L...".
Previous concept: SqlQueryable BookStore.BookInfo
Property SqlSource (String) =
    SELECT
        k.ID,
        CommentCount = COUNT(kom.ID), --this is a simple
Property Module (ModuleInfo) = BookStore
Property Name (String) = BookInfo
Expected ";" or "{".

Call stack:

[Error] DeployPackages: Rhetos.Dsl.DslSyntaxException: At line 204, column 60, file 'BookStore\DslScripts\BookStore.rhe',
after: "...nfo " SELECT k.ID, CommentCount= COUNT(kom.ID), --this is a simple "",
before: "comment" FullName = o.Name+ ' : ' + k.Title FROM BookStore.Book k L...".
Previous concept: SqlQueryable BookStore.BookInfo
Property SqlSource (String) =
    SELECT
        k.ID,
        CommentCount= COUNT(kom.ID), --this is a simple
Property Module (ModuleInfo) = BookStore
Property Name (String) = BookInfo
Expected ";" or "{".
   at Rhetos.Dsl.DslParser.UpdateContextForNextConcept(TokenReader tokenReader, Stack`1 context, IConceptInfo conceptInfo) in \Rhetos\Source\Rhetos.Dsl\DslParser.cs:line 206
   at Rhetos.Dsl.DslParser.ExtractConcepts(IEnumerable`1 conceptParsers) in \Rhetos\Source\Rhetos.Dsl\DslParser.cs:line 109
   at Rhetos.Dsl.DslParser.get_ParsedConcepts() in \Rhetos\Source\Rhetos.Dsl\DslParser.cs:line 53
   at Rhetos.Dsl.DslModel.Initialize() in \Rhetos\Source\Rhetos.Dsl\DslModel.cs:line 114
   at Rhetos.Dsl.DslModel.get_Concepts() in \Rhetos\Source\Rhetos.Dsl\DslModel.cs:line 75
   at Rhetos.Deployment.ApplicationGenerator.ExecuteGenerators(Boolean deployDatabaseOnly) in \Rhetos\Source\Rhetos.Deployment\ApplicationGenerator.cs:line 79
   at DeployPackages.Program.GenerateApplication(ILogger logger, Arguments arguments) in \Rhetos\Source\DeployPackages\Program.cs:line 183
   at DeployPackages.Program.Main(String[] args) in i\Rhetos\Source\DeployPackages\Program.cs:line 69
Info:

Currently, the properties of the polymorphic are automatically copied to the subtype, if an explicit mapping is not defined. For example, the following snippet from documentation will generated the EventDate property on the Entity BorrowMoney.

Unfortunately this behavior makes it easy for developers to forget to implement the subtype property and generates bug with empty subtype properties with null values.

Module Demo
{
    Polymorphic MoneyTransaction
    {
        DateTime EventDate;
        Money Amount;
    }

    Entity BorrowMoney
    {
        ShortString FromWhom;
        Money Amount;

        Is Demo.MoneyTransaction;
        // The EventDate will be automatically added. The Amount is automatically mapped.
    }

Expected behavior
Deploying the code above should result with a DslSyntaxException exception "The property Money Demo.MoneyTransaction.Amount is not implemented in the polymorphic subtype Entity Demo.BorrowMoney. Please add the property implementation to the subtype.".

The working script should look like this:

    Entity BorrowMoney
    {
        ShortString FromWhom;
        DateTime EventDate;
        Money Amount;

        Is Demo.MoneyTransaction;
       // The EventDate and Amount are automatically mapped.
    }

Notes

• To allow for backward compatibility, the automatic creation of properties should be left in the code, activated by the bool switch in web.config appSettings: "CommonConcepts.Legacy.AutoGeneratePolymorphicProperty". This switch should be added to the web.config with the value "False", but when reading it the default value should be True (if missing), so that the existing applications use the old behavior.
• Update the documentation.

Allow application developers to seamlessly deploy zip packages and source packages. This should simplify transition from the development environment to the production environment.

Currently, application developers use ApplyPackages or improvised batch scripts for deploying source packages in development environment, and ExtractPackages (mostly) with DeployPackages for production environment.

Simplify current development and deployment process by solve DeployPackages deficiencies:

• Deploy multiple zip and source (folder) package at the same time.
• Stop using Rhetos server's folder during package deployment as the source for plugins and DSL scripts. Content of the deployed packages may be kept in Rhetos folder exclusively for backup or logging purpose, not to be misused as a development working folder.
• Allow for a developer's source folder (containing DSL scripts) to be directly deployed to the Rhetos server in a single step.