BigQuery - Where can I find the error stream?

google-cloud-platform google-bigquery
Ary Jazz picture Ary Jazz · Aug 30, 2018 · Viewed 16.2k times · Source

I have uploaded a CSV file with 300K rows from GCS to BigQuery, and received the following error:

error_image

Where can I find the error stream?

I've changed the create table configuration to allow 4000 errors and it worked, so it must be a problem with the 3894 rows in the message, but this error message does not tell me much about which rows or why.

Thanks

Answer

Ary Jazz picture Ary Jazz · Sep 5, 2018

I'm finally managed to see the error stream by running the following command in the terminal:

bq --format=prettyjson show -j <JobID>

It returns a JSON with more details. In my case it was:

"message": "Error while reading data, error message: Could not parse '16.66666666666667' as int for field Course_Percentage (position 46) starting at location 1717164"

Related questions

What's the difference between BigQuery and BigTable?

Is there any reason why someone would use BigTable instead of BigQuery? Both seem to support Read and Write operations with the latter offering also advanced 'Query' operations. I need to develop an affiliate network (thus I need to track …

Read More
Random Sampling in Google BigQuery

I just discovered that the RAND() function, while undocumented, works in BigQuery. I was able to generate a (seemingly) random sample of 10 words from the Shakespeare dataset using: SELECT word FROM (SELECT rand() as random,word FROM [publicdata:samples.shakespeare] …

Read More
How do I identify the Google Cloud Storage URI from my Google Developers Console?

When I attempt load data into BigQuery from Google Cloud Storage it asks for the Google Cloud Storage URI (gs://). I have reviewed all of your online support as well as stackoverflow and cannot find a way to identify the …

Read More