mindsdb / datasources Goto Github PK

I have noticed that the information printed when a datasource is not found is referencing mindsdb_native[extra_data_sources], even though it is deprecated in favour of mindsdb.

eg.

Should this be changed? I can work on this if it needs a change.

Initial issue is mindsdb/mindsdb#1326. When user upload big file as ds we need make some processing for this file: get columns list and row count. At this moment we do it by accessing to dataframe: https://github.com/mindsdb/mindsdb/blob/stable/mindsdb/interfaces/datastore/datastore.py#L301
When file is big this step can take a lot of time. Minutes. But we really do not need create dataframe, we need just read first line of file and get count of rows in it. This is actual and for sql-datasources too: instead of fetch all rows from table, we can do only 'select count(1) from table', and 'select * from table limit 1' to get columns names.

Current atlas tests is not passing because test server is down.

Should be possible connection with CA, user cert and key

need add secure_connect_bundle and protocol_version

I have missed some underscores in the init info.

Reference:

I will open a PR that fixes this.

The hive ds has a typo in the module name. It specifies timescale_ds instead of hive_ds.

Reference:

At this moment if dataframe returned by query() is empty, then columns list will be empty too. Such behaviour is inappropriate for sql api, we need get columns list even if dataframe is empty. We need somehow get columns list, or change driver for CH.