🐛 [firestore-bigquery-export] Error initializing raw change log table about extensions HOT 7 OPEN

I don't really remember (the use of this extension has been in place for over 3 years)

tbh, does it matter? clustering by JSON type fields is not at all possible in BQ. our two JSON fields (data, old_data) are therefore not part of the defined clustered fields.

the error does not make sense to me. it seems somehow internally the data/old_data fields are used in such a way that is incompatible with JSON type.

what also strikes me as odd, is the fact the errors seems to be randomly appearing, if it really was related to JSON types fields then I'd expect this error to occur for every firestore>bigquery event. but this is not the case.

from extensions.

I'll have a look back at the release commits and see if anything could have caused this to start happening. Thanks for providing all this info by the way!

I'll see if we can get this issue prioritised

from extensions.

This is strange to me, as we don't use the datatype JSON as far as I know, data is saved in a raw string.

Do you have any more info on your Clustering you're able to provide?

from extensions.

Hi @cabljac,

thanks for your quick reply;

schema:

(part of) table info;

from extensions.

Was this schema solely generated by the extension?

from extensions.

could this error be related to the fact we updated the extension to version 0.1.51?

I'm unsure when we updated this extension however it must have been recently as this version was released on June 19th

from extensions.

@cabljac

wondering if you have an update on this?

from extensions.