xianwill Goto Github PK

arrow

Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and interprocess communication. Languages currently supported include C, C++, Java, JavaScript, Python, and Ruby.

arrow-rs

Official Rust implementation of Apache Arrow

data-geonames

delta

An open-source storage layer that brings scalable, ACID transactions to Apache Spark™ and big data workloads.

delta-rs

A native Rust library for Delta Lake

dragnet

Just the facts -- web page content extraction

hotdog

Hotdog is a syslog-to-Kafka forwarder which aims to get log entries into Apache Kafka as quickly as possible.

json-schema

JSON Schema validator for java, based on the org.json API

justjson

JSON helper library for Android

kafka-delta-ingest

A highly efficient daemon for streaming data from Kafka into Delta Lake

kafkajs

A modern Apache Kafka client for node.js

kplay

node-url-poller

python-readability

fast python port of arc90's readability tool, updated to match latest readability.js!

readability

A standalone version of the readability lib

rust-dataframe

A Rust DataFrame implementation, built on Apache Arrow

scribd.github.io

The Scribd technology site, where we share the challenges in building the world's largest library

slipstream

A tool for doing on-the-fly message validation for Kafka.

spark-boilerplate

A boilerplate for spark projects with docker support for local development and scripts for emr support.