bbkr / uprooted Goto Github PK

UpRooted::Schema::Column.is-nullable, UpRooted::Writer.use-schema-prefix, etc.

Less confusing, because in old layout one my expect to provide actual data like UpRooted::Writer.new( schema-prefix => 'foo' ).

Places to check:

• POD
• code comments
• error messages
• test descriptions
• readmes

Consistent naming pattern:

• If something refers to UpRooted::* object instance it should go by full name.
• If something refers to resource like table in database it should be named as table, not Table (because there is no Table namespace or object anywhere)

Many potential users who want to use UpRooted may be discouraged by installation of Raku, studying package management and installing needed C libraries on their system.

Docker image with Raku, DBI, C libraries and UpRooted preinstalled will be a good start for those who want to simply get the job done without deep diving into Raku ecosystem.

To directly copy UpRooted::Reader data to another database.

This is nice to have feature but complicates Writer a lot - next row of data must have ability to terminate previous set of rows and leaving writing after last row must terminate everything..

It will be implemented if it gets 8 upvotes here :)

It should accept Channel from Reader.
On this Channel there will be Table followed by set of rows from this Table. When all Tables are read Channel will be closed.

This flow will Writer to prepare for new data set before actual data will arrive, for example open new file, prepare new printf template, prepare new SQL statement, etc.

Decide if horse riddle is fatal and not allow Tree to be constructed.
Or maybe it should accept such Path but cause error/warning on Reader level?

Instead of having separate "factory" actor:

use UpRooted::Cartographer::MySQL;

my $connection = DBIish.connect( 'mysql', host => ..., port => ..., ... );
my $schema = UpRooted::Cartographer::MySQL.new( :$connection ).schema( );

Just extend base Schema:

use Schema::MySQL;

my $connection = DBIish.connect( 'mysql', host => ..., port => ..., ... );
my $schema = Schema::MySQL.new( :$connection );

https://www.postgresql.org/docs/current/information-schema.html

Decide which database use. Percona is probably most popular today on server environments but MariaDB diverged more from vanilla MySQL. Or maybe use all of them?

This will prevent unwanted descending into data that should not be dumped.

If lucky this will reuse almost whole logic from Writer::MySQLFile except quoting.
Probably Writer::Helper::SQLQuery could be created to be shared by both.
Which will also reuse later VALUES batching code.

It should accept Tree as contract and iterate over Paths in their order.
Each Path should be processed by specific Reader implementation and produce set or rows.

Reader should expose channel, which will receive Table, row, row, row, Table, row, row, row, row, row... pattern.
Channel should be closed when last Path is processed.
Each set of new root Table conditions should open new Channel.

Tree can be represented as set of Paths from root Table to every leaf Table (intermediate Tables are also considered leafs). This is also contract for Reader.

It must have circuit breaker when Path detected loop.

Avoid copy-pasting code between MySQL and PostgreSQL implementations wherever possible. For example:

• Information Schema is read using the same logic and in the same order (only queries are different for each engine)
• Reader is exactly the same.
• Writer to another database handle is exactly the same.
• Writer to SQL file can use common Fully Qualified Names caching.

If successful we will get PostgreSQL implementations almost for free, only tests will be needed.

To complete proof-of-concept chain from Schema to Writer MySQL seems like simplest choice.

Dumping to *.sql file or cloning to another database are probably two most common dev tasks.

To complete proof-of-concept chain from Schema to Writer MySQL seems like simplest choice.

my $writer = UpRooted::Writer::MySQLFile.new( :!use-schema-name );
$writer.write( $reader, id => 1 );

This looks bad for example, named param, positional param and slurpy param in one class.
Not to mention future write method options may have conflicts with conditions if more params are added.

Should be named params all the way:

my $writer = UpRooted::Writer::MySQLFile.new( :!use-schema-name );
$writer.write( :$reader, conditions => { 'id' => 1 } );

This was noble idea to provide "safe" common type that UpRooted::Writer can rely on.

However this is faulty because of following reasons:

• Database type mappings are full of edge cases. MySQL does not have boolean types, Postgres does. MySQL does not have array types, Postgres does. PostgreSQL has two Raku drivers with slightly different type mappings. Some other databases may have weird types. Tracking and trying to establish simple common types for UpRooted::Column is very difficult.
• It will lead to unnecessary double casting in UpRooted::Readers in many cases.
• Some UpRooted::Writers may benefit from complex types. Like for example Writer::JSON (if implemented) will handle PostgreSQL array types without any issues. It is best for Writer to decide how to handle data from database driver.
• We should not limit user. For example if user decides to write custom Converter for DBDish (let's say for spatial data) and custom UpRooted::Writer that can store this type he should be able to do so without looking at real column type.

To write data from Reader directly into PostgreSQL database.