davbre / mira Goto Github PK

When "\t" is speficied as the delimiter in datapackage.json it fails to save in the datapackage_resources tables. This is because it fails the ActiveRecord validation:

validates :delimiter, presence: true

Some options

• https://github.com/blueimp/jQuery-File-Upload
• http://hayageek.com/docs/jquery-upload-file.php

Currently everything is indexed by default. When there are large text columns, this type message is returned during the data import:

PG::ProgramLimitExceeded: ERROR: index row size 3432 exceeds maximum 2712 for index "index_xy36_69_on_provenance_text"
HINT: Values larger than 1/3 of a buffer page cannot be indexed.
Consider a function index of an MD5 hash of the value, or use full text indexing.

One option would be to add an additional key-value pair for each field indicating whether or not it should be indexed, e.g. "index: false". Otherwise the max length could be added to the field's "constraints" object, and this information then used to decide to index of not. Inclined to go with first option.

Blank strings are being stored as either "" or NULL depending on the form of the upload. Want to always set to NULL when empty.

This bug was introduced in the last 6 weeks. For example, the refugee deaths dataset was failing to import as it has a column "cause of death". This was not the case previously.

Hello,

thanks for mira. Looks really promising.
Was trying it out but ran into a problem.
After uploading a datapackage.json which is valid according to the validator.
But when I upload a csv file the worker fails with:

Job ProcessCsvUpload (id=8) FAILED (0 prior attempts) with ActiveRecord::UnknownAttributeError: unknown attribute 'DATE' for Xy2_2.

Here is the excerpt from the datapackage.json

"schema": {
        "fields": [
          {
            "name": "DATE",
            "type": "datetime"
          }
....

the api from mira for datapackage/fields returns

[
{
"name": "DATE",
"order": 1,
"big_integer": null,
"add_index": true,
"format": null,
"type": "datetime"
},
...

Any help would be appreciated.
Thanks.

Relates to old version of jquery. See here.

Will update jquery-rails gem, so that jquery3 is used as well as update for the view responsible for rendering tables in admin mode.

When deleting a datasource, the corresponding database table is not being emptied.

Was trying this with our schema from the Carnegie Museums of Pittsburgh (datapackage.json). Unfortunately, it does not validate.

We have two different files that both use the same schema, so we are using the schemas proprty to use a single schema definition for both files.

However, when trying to upload the datapackage.json file, I get the following error:

Datapackage ["Resource 'schema' must be a Hash.", "Path: cmoa.csv.", "Resource 'schema' must be a Hash.", "Path: teenie.csv."]```