ccsd / canvas-data-embulk-configs Goto Github PK

This repo provides a tag of the current CD version supported, ex v4.2.4
I'm evaluating the most appropriate way to retag or update the tags for this repo when we make mid-version updates from our own use.

Should we trash and recreate the tag with the newest configs/commit, keeping the tag version inline with CD version, as I've been doing, or append a revision number such as v4.2.4 r1?

My hesitation is that r1 isn't something a user would see on Canvas Data Portal documentation.

Apparently 4.2.5 was released without any release notes because it only updated description fields associated with the requests table (and, of course, the version number).

There were no changes to the data for this update which is why this was not included in the
release notes. This update to the schema was a change to update documentation on the requests tables and users should see no changes in behavior or data. I hope this helps clarify this change. If you have any questions on this please let me know and I will be happy to help.

I did a diff of the current schema.json vs. a copy of my schema.json from v4.2.4 and the only differences are some column level description and the table level description, so it's probably about as trivial of an update as possible.

Any thoughts or plans to support Canvas Data v2? Looks like a batch of new tables are planned.

https://docs.google.com/spreadsheets/d/1axjhLwPY4N16SAf61-X8xj3kSg_Ey-PXvcsUTtojRoo/edit#gid=0

TBH I'm not sure when it's planned for release. I thought it was last month (March-April 2021) but I can't find that information any more.

L

https://community.canvaslms.com/docs/DOC-17623-canvas-data-release-notes-2019-08-26
assignment_dim
The muted column in this table is no longer being populated in Canvas Data and requires user attention. This column will be removed on 18 January 2020.

The output destination for quiz_dim.description was incorrectly named body for all databases.
resolved with: d14a12a

This field can't be converted to a local default timezone from UTC without a timestamp.
The field is stored in VARCHAR which makes some queries using this field problematic.
You can store the field using Embulk Output SQL type 'date', but there will be no conversion.

Best solution would be to load the table to staging and convert that value in production locally...
or just use timestamp and convert to create a new value from timestamp.

Example: #4 (comment)