Run Grate from code about grate HOT 5 CLOSED

Thanks for the reply. We run it as part of AWS an Cloudformation deployment (IaC for AWS basically) where we first create the database (AWS RDS Postgres), then run a dotnet lambda in a CustomResource where grate (or roundhouse) would run and then other lambdas will run that needs the database and schema to be there. We used System.Diagnostics.Process.Start in the past but had some problems with error reporting, but should be possible to solve.

from grate.

Hi, @mk0sojo - thanks for reaching out. I know RoundhousE had libraries as well. I'm not familiar with everyone's workflows around this, of course, but personally, I would prefer a pre-build binary, and call it from the command-line, instead of a library, for this purpose. I haven't taken on the complexity of having separate projects for each and every DB provider for grate, as RoundhousE had, as I personally thought it was a rather exotic use case to call it from code.

But, of course, I might be wrong :)

Is it very unconvenient to call grate using System.Diagnostics.Process.Start in your use-case? https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.process.start?view=net-8.0#system-diagnostics-process-start(system-string-system-string)

from grate.

Thanks for the explanation. I do see that running from code would make collecting errors, etc easier. I just think, one takes on a lot of technical dependencies when using a library that has many 3-rd party dependencies too, and, maybe .NET version incompatibilities arise as well. I think I'd prefer to not go down the route of having a public class library API for grate as well as the command-line interface.

We do use Serilog for logging, which has loads of sinks to log to. Would enabling logging to any of the Serilog sinks be of any help (some kind of more structured format, e.g. XML, database, etc)? There's a huge list of sinks supported by Serilog here: https://github.com/serilog/serilog/wiki/Provided-Sinks

from grate.

Thanks, we got it working, but had to do a change to the Grate source code. In DefaultConfiguration.cs, DefaultOutputPath uses

Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData, Environment.SpecialFolderOption.Create)

Because we run this in a Linux lambda, Environment.SpecialFolder.LocalApplicationData is not writable, we had to change this to

Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData, Environment.SpecialFolderOption.DoNotVerify)

This had to be done even if we override the output directory using the -o flag to the CLI. Is this something that can be changed? I can create PR.

Just for reference, we had to run the following script on the Postgres DB before migrating:

-- Table was lowercase in RoundHouse, but Pascal case in Grate
ALTER TABLE roundhouse.scriptsrun
  RENAME TO "ScriptsRun";

-- Table was lowercase in RoundHouse, but Pascal case in Grate
ALTER TABLE roundhouse.scriptsrunerrors
  RENAME TO "ScriptsRunErrors";

-- Table was lowercase in RoundHouse, but Pascal case in Grate
ALTER TABLE roundhouse.version
  RENAME TO "Version";

-- Make sure ID sequences have the correct values
SELECT setval('roundhouse.scriptsrun_id_seq', max(id)) FROM roundhouse."ScriptsRun";

SELECT setval('roundhouse.scriptsrunerrors_id_seq', max(id)) FROM roundhouse."ScriptsRunErrors";

SELECT setval('roundhouse.version_id_seq', max(id)) FROM roundhouse."Version";

and then add --schema roundhouse (lower case) when calling the grate CLI,

from grate.

Thanks for letting me know about the issues. The ones with table casing and the one with demanding a writable LocalAppData folder should be solved in the next release:
#256
#344

from grate.