Code Monkey home page Code Monkey logo

bigquery-python-tutorial's Introduction

Google BigQuery Python SDK Tutorial

Python Google Cloud BigQuery Google Cloud Storage GitHub Last Commit GitHub Issues GitHub Stars GitHub Forks

Google BigQuery Python SDK

Source for the accompanying tutorial: https://hackersandslackers.com/google-bigquery-python/

Getting Started

Get set up locally in two steps:

Environment Variables

Replace the values in .env.example with your values and rename this file to .env:

  • GCP_PROJECT_ID: ID of your Google Cloud Project.
  • GOOGLE_APPLICATION_CREDENTIALS: Filepath to JSON containing Google Cloud credentials as service key.
  • GCP_BUCKET_NAME: Name of an existing Google Cloud Storage bucket (used to upload CSV dataset).
  • GCP_BIGQUERY_DATASET_ID: Desired BigQuery dataset ID (can be new or existing).
  • GCP_BIGQUERY_TABLE_ID: Desired BigQuery table name/ID (should be new table name).

Remember never to commit secrets saved in .env files to Github.

Installation

Get up and running with make deploy:

$ git clone https://github.com/hackersandslackers/bigquery-python-tutorial.git
$ cd bigquery-python-tutorial
$ make deploy

Hackers and Slackers tutorials are free of charge. If you found this tutorial helpful, a small donation would be greatly appreciated to keep us in business. All proceeds go towards coffee, and all coffee goes towards more content.

bigquery-python-tutorial's People

Contributors

renovate-bot avatar toddbirchard avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

whatdoesdatasay aarondlc shrinivas-io maxcodextc akamil-etsy ethicalsecurity-agency

bigquery-python-tutorial's Issues

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

ERROR: ERROR: Could not find a version that matches pprint
No versions found
Was <redacted> reachable?
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches pprint
[pipenv.exceptions.ResolutionFailure]:       No versions found
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches pprint
No versions found
Was <redacted> reachable?

['Traceback (most recent call last):\n', '  File "/usr/local/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pipenv/utils.py", line 501, in create_spinner\n    yield sp\n', '  File "/usr/local/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pipenv/utils.py", line 649, in venv_resolve_deps\n    c = resolve(cmd, sp)\n', '  File "/usr/local/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pipenv/utils.py", line 539, in resolve\n    sys.exit(c.return_code)\n', 'SystemExit: 1\n']

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • Update dependency pytest to v6.2.5
  • Update dependency flake8 to v3.9.2
  • Update dependency google-cloud-bigquery to v2.34.4
  • Update dependency google-cloud-storage to v1.44.0
  • Update dependency google-cloud-storage to v1.44.0
  • Update dependency google-crc32c to v1.5.0
  • Update dependency google-resumable-media to v1.3.3
  • Update dependency googleapis-common-protos to v1.63.0
  • Update dependency isort to v5.13.2
  • Update dependency loguru to v0.7.2
  • Update dependency loguru to v0.7.2
  • Update dependency mypy to v0.991
  • Update dependency proto-plus to v1.23.0
  • Update dependency pyasn1 to v0.6.0
  • Update dependency pyasn1-modules to v0.4.0
  • Update dependency pycparser to v2.22
  • Update dependency python-dotenv to v0.21.1
  • Update dependency pytz to v2021.3
  • Update dependency rsa to v4.9
  • Update dependency six to v1.16.0
  • Update dependency win32-setctime to v1.1.0
  • Update dependency cachetools to v5
  • Update dependency chardet to v5
  • Update dependency flake8 to v7
  • Update dependency google-api-core to v2
  • Update dependency google-auth to v2
  • Update dependency google-cloud-bigquery to v3
  • Update dependency google-cloud-bigquery to v3
  • Update dependency google-cloud-core to v2
  • Update dependency google-cloud-storage to v2
  • Update dependency google-cloud-storage to v2
  • Update dependency google-resumable-media to v2
  • Update dependency mypy to v1
  • Update dependency packaging to v24
  • Update dependency pyparsing to v3
  • Update dependency pytest to v8
  • Update dependency python-dotenv to v1
  • Update dependency python-dotenv to v1
  • Update dependency pytz to v2024
  • 🔐 Create all rate-limited PRs at once 🔐

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Warning

Renovate failed to look up the following dependencies: Failed to look up pypi package python.

Files affected: pyproject.toml

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

pep621
pyproject.toml
  • poetry >=0.12
pip_requirements
requirements.txt
  • cachetools ==4.2.1
  • certifi ==2020.12.5
  • cffi ==1.14.5
  • chardet ==4.0.0
  • colorama ==0.4.4
  • google-api-core ==1.26.0
  • google-auth ==1.26.1
  • google-cloud-bigquery ==2.6.1
  • google-cloud-core ==1.6.0
  • google-cloud-storage ==1.36.0
  • google-crc32c ==1.1.2
  • google-resumable-media ==1.2.0
  • googleapis-common-protos ==1.52.0
  • grpcio ==1.35.0
  • idna ==2.10
  • loguru ==0.5.3
  • packaging ==20.9
  • proto-plus ==1.13.0
  • protobuf ==3.14.0
  • pyasn1-modules ==0.2.8
  • pyasn1 ==0.4.8
  • pycparser ==2.20
  • pyparsing ==2.4.7
  • python-dotenv ==0.15.0
  • pytz ==2021.1
  • requests ==2.25.1
  • rsa ==4.7.1
  • six ==1.15.0
  • urllib3 ==1.26.3
  • win32-setctime ==1.0.3
poetry
pyproject.toml
  • Python ^3.8
  • google-cloud-storage *
  • google-cloud-bigquery *
  • loguru *
  • python-dotenv *
  • pytest *
  • mypy *
  • isort ^5.6.4
  • flake8 ^3.8.4
  • Check this box to trigger a request for Renovate to run again on this repository

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Creating virtualenv biquery-python-tutorial-Nzke21qj-py3.9 in /home/dependabot/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

  PackageNotFound

  Package loguru (0.5.3) not found.

  at /usr/local/.pyenv/versions/3.9.1/lib/python3.9/site-packages/poetry/repositories/pool.py:144 in package
      140│                     self._packages.append(package)
      141│ 
      142│                     return package
      143│ 
    → 144│         raise PackageNotFound("Package {} ({}) not found.".format(name, version))
      145│ 
      146│     def find_packages(
      147│         self, dependency,
      148│     ):

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Creating virtualenv biquery-python-tutorial-O57jrGmq-py3.9 in /home/dependabot/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

  SolverProblemError

  Because biquery-python-tutorial depends on pytest (*) which doesn't exist, version solving failed.

  at /usr/local/.pyenv/versions/3.9.4/lib/python3.9/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│ 
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

ERROR: ERROR: Could not find a version that matches pprint
No versions found
Was <redacted> reachable?
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/.pyenv/versions/3.7.7/lib/python3.7/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/.pyenv/versions/3.7.7/lib/python3.7/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/.pyenv/versions/3.7.7/lib/python3.7/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches pprint
[pipenv.exceptions.ResolutionFailure]:       No versions found
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches pprint
No versions found
Was <redacted> reachable?

['Traceback (most recent call last):\n', '  File "/usr/local/.pyenv/versions/3.7.7/lib/python3.7/site-packages/pipenv/utils.py", line 501, in create_spinner\n    yield sp\n', '  File "/usr/local/.pyenv/versions/3.7.7/lib/python3.7/site-packages/pipenv/utils.py", line 649, in venv_resolve_deps\n    c = resolve(cmd, sp)\n', '  File "/usr/local/.pyenv/versions/3.7.7/lib/python3.7/site-packages/pipenv/utils.py", line 539, in resolve\n    sys.exit(c.return_code)\n', 'SystemExit: 1\n']

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.