Code Monkey home page Code Monkey logo

Comments (5)

tylertreat avatar tylertreat commented on August 16, 2024

This sounds like a similar issue. There should be an errors property on the job.

from bigquery-python.

ampkeegan avatar ampkeegan commented on August 16, 2024

I'm not seeing errors or insertErrors on my job.

I have another piece of code which does have errors on occasion:

job = bqClient.client.push_rows(
            to_bq_jobs[path]['dataset'],
            to_bq_jobs[path]['table'],
            rowsToAdd[rowCount:topCount],
            #insert_id_key=to_bq_jobs[path]['id_to_match']
            )
        #print(str(job))
        if 'insertErrors' in job:

And that lets me print out the errors. However on this job, there isn't an error or insertError key in the job dict.

The status dict has 'state': "running", there isn't any error listed.

I tried changing to job_id, _results = bqClient.client.wait_for_job(job), which still throws the same error and _results doesn't have anything.

from bigquery-python.

tylertreat avatar tylertreat commented on August 16, 2024

Can you dump the contents of the dict that gets returned?

from bigquery-python.

ampkeegan avatar ampkeegan commented on August 16, 2024

I've been testing with JSON newlimited import which also fails, but it also throws a similar error.

Reason:invalid. Message:Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the error stream for more details.
{
    "configuration": {
        "jobType": "LOAD",
        "load": {
            "destinationTable": {
                "datasetId": "redacted",
                "projectId": "redacted",
                "tableId": "stages_copy"
            },
            "schema": {
                "fields": [
                    {
                        "mode": "NULLABLE",
                        "name": "Stage_id",
                        "type": "INTEGER"
                    },
                    {
                        "mode": "NULLABLE",
                        "name": "Stage_Order",
                        "type": "INTEGER"
                    },
                    {
                        "mode": "NULLABLE",
                        "name": "Stage_Name",
                        "type": "STRING"
                    },
                    {
                        "mode": "NULLABLE",
                        "name": "Stage_Pipeline_id",
                        "type": "INTEGER"
                    }
                ]
            },
            "sourceFormat": "NEWLINE_DELIMITED_JSON",
            "sourceUris": [
                "gs://redacted/stages.json"
            ]
        }
    },
    "etag": "\"redacted/29pKp--d60WMuqds86QyFCCo47Q\"",
    "id": "redacted",
    "jobReference": {
        "jobId": "redacted",
        "location": "US",
        "projectId": "redacted"
    },
    "kind": "bigquery#job",
    "selfLink": "https://www.googleapis.com/bigquery/v2/projects/redacted?location=US",
    "statistics": {
        "creationTime": "1539023656058",
        "startTime": "1539023656543"
    },
    "status": {
        "state": "RUNNING"
    },
    "user_email": "redacted"
}

`

from bigquery-python.

bencaine1 avatar bencaine1 commented on August 16, 2024

Our workaround is to surround all our job.result() calls with a try/except that prints out job.errors, but it would be really nice if the errors were just printed out so that we didn't have to do that!

from bigquery-python.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.