Topic: pyarrow Goto Github

👇 Here are 49 public repositories matching this topic...

agutiernc / data-eng-zoomcamp
pyarrow,Data Engineering Zoomcamp 2024
User: agutiernc
data-ingestion data-warehouse docker pandas pipelines postgresql pyarrow python sql terraform mage-ai jupyter-notebook etl-pipeline dbt dlt ci-cd apache-kafka apache-spark google-cloud-platform etl

benyaminzojaji / mongodb_tutorial
pyarrow,MongoDB tutorial repository
User: benyaminzojaji
mongodb pyarrow python tutorial

dr-saad-la / pyarrow-tuts
pyarrow,Pyarrow Tutorials
User: dr-saad-la
programming pyarrow python3 tutorials

ericpaul075 / ocp8-big-data-project-deployed-in-aws-cloud
pyarrow,Define a big data architecture and perform distributed machine learning calculations on an EMR cluster using AWS
User: ericpaul075
aws-s3 aws-ec2 aws-emr-clusters big-data keras-models pyspark python tensorflow feature-extraction boto3 pca pillow pyarrow hadoop

jakubpluta / flowsome
pyarrow,Minimal framework for building and executing data workflows on a single machine
User: jakubpluta
data pipelines polars pyarrow python

lykmapipo / nyc-tlc-trip-data
pyarrow,Python scripts to download, process, and analyze the New York City Taxi and Limousine Commission (TLC) Trip Record Data dataset
User: lykmapipo
data data-engineering data-extraction data-transformation etl lykmapipo metadata nyc python geopandas pandas pyarrow s3 apache-arrow jupyterlab joblib fsspec apache-spark nyc-taxi-dataset

narwhals-dev / narwhals
pyarrow,Lightweight and extensible compatibility layer between dataframe libraries!
Organization: narwhals-dev
Home Page: https://narwhals-dev.github.io/narwhals/
cudf dask ibis modin pandas polars pyarrow vaex

randomfractals / chicago-crimes
pyarrow,Exploring Chicago crimes dataset with Jupyter notebooks, DuckDB, Malloy and new Panel/PyScript data and dashboard tools.
User: randomfractals
chicago crimes duckdb julia jupyter-notebooks large-csv malloy malloydata parquet polars pyarrow

svjack / pyarrowexpressioncasttoolkit
pyarrow,A small cast tookit class drived from _ParquetDatasetV2 to support cast in filters argument
User: svjack
pyarrow arrow pandas partitions dataset log tookit dtype conditions

tdiprima / arrow-and-parquet
pyarrow,
User: tdiprima
apache-arrow apache-parquet javascript python pandas pyarrow

uber / petastorm
pyarrow,Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Organization: uber
tensorflow pytorch deep-learning machine-learning sysml pyspark pyarrow parquet parquet-files

vipinc007 / parquetviewer
pyarrow,A web application for viewing Apache Parquet files . This is a Python + Flask application
User: vipinc007
parquet-files parquet-viewer pyarrow pandas flask-application python3

wheretrue / biobear
pyarrow,Work with bioinformatic files using Arrow, Polars, and/or DuckDB
Organization: wheretrue
Home Page: https://www.wheretrue.dev/docs/exon/biobear/
bioinformatics biology rust-bio samtools arrow python biopython duckdb polars pyarrow

xbrianh / xdlake
pyarrow,A loose implementation of the deltalake protocol, written in Python on top of pyarrow, focused on extensibility, customizability, and distributed data.
User: xbrianh
Home Page: https://github.com/xbrianh/xdlake
databricks delta-lake deltalake deltatables hive parquet pyarrow python spark