error csv upload with pgx driver (encode date column) about usql HOT 14 CLOSED

Can you try casting the column to a specific data type in the SELECT statement? The copy command doesn't try to convert data types on purpose, to keep the implementation simple. This is called out here: https://github.com/xo/usql#datatype-compatibilty-and-casting

from usql.

Seems I can't cast columns with the CSV source.

usql --set="dsn=${PGX_DSN}" --command "\copy csvq://. :dsn 'SELECT col1::text, col2 FROM test' 'test'"
# error: [L:1 C:8] constant COL1::TEXT is not defined

from usql.

How about a regular SELECT CAST(col1 AS date)? If you're inserting it into a date column, you can't use text.

from usql.

Note, you have to use the csvq driver's syntax, not PostgreSQL's.

from usql.

If a driver can't return data in a format another driver would understand, that's a limitation of the driver, not usql.

from usql.

I checked the csvq ref doc:

usql --set="dsn=${PGX_DSN}" --command "\copy csvq://. :dsn 'SELECT STRING(col1) AS col1, col2 FROM test' 'test'"
# error: ERROR: COPY from stdin failed: unable to encode (*interface {})(0xc001611d90) into binary format for date (OID 1082): cannot find encode plan (SQLSTATE 57014)

Without success.

from usql.

You should cast it to a date, not string/text/varchar.

from usql.

I understand, but csvq don't support date type. Cast to DATETIME also don't work.
Can you show a correct query based on my example?

from usql.

So you'd have to copy the data into another table and do the conversion in PostgreSQL as a next step. But I'd recommend using PostgreSQL native tools for reading the CSV.

from usql.

I just wanted to report a problem when using the pgx driver when postgres is ok.

from usql.

Thanks, but we're not planning on adding type conversion logic in \copy.

from usql.

@artemklevtsov Please understand there's minimal things that we can do for this. usql currently has 44 different SQL drivers that it supports, and there are bound to be difficulties with this.

As such, there are other drivers that support CSV -- specifically DuckDB and the ADODB drivers (Windows). Additionally, I believe there are also ways to import CSV into SQLite3 databases, so that might be a viable solution as well.

from usql.

Also -- PostgreSQL natively supports importing CSVs using their COPY instruction. If you expose the file to a place where the server can see it, you should be able to use that command cleanly with usql.

from usql.

Thanks for the replies.
I tried to switch from the psql. I want to use one cli tool for all operations for the most dbs.
Upload directly on server is not possible in my cases.
Now I can use usql witn postgres driver anyway.

from usql.