Code Monkey home page Code Monkey logo
Ulixee photo

ulixee Goto Github PK

repos: 25.0 gists: 0.0

Name: Ulixee

Type: Organization

Bio: Ulixee turns every website into an open API. It's built on a decentralized blockchain, meaning there are no logins, no throttle limits, and no restrictions.

Location: Atlanta, GA

Blog: https://ulixee.org

Ulixee

Ulixee is a scraping engine with a built-in deployment unit that enables out-of-the-box querying across a horizontal deployment.

This repository is the development home to several of the tools that make it easy to build and manage these scripts, including Ulixee Desktop, Cloud and Datastore.

Projects

  • Hero /hero. The Automated Browser Engine built for scraping. (repository home - https://github.com/ulixee/hero).
  • Datastore /datastore. Packaged "database" containing API access to crawler functions and extractor functions.
  • Cloud /cloud. Run Ulixee tooling on a remote machine.
  • Stream /stream. Query, transform and compose Datastores running on any machine.
  • Desktop /desktop. Supercharge scraper script development using a Hero Replay toolset, remote Datastore viewer and Error troubleshooter.

Tooling

Try out Ulixee Desktop! The Alpha release is available for download under Assets.

Docker

We publish a Docker image of the latest Ulixee Cloud to:

  • Github Container Registry: docker pull ghcr.io/ulixee/ulixee-cloud && docker tag ghcr.io/ulixee/ulixee-cloud ulixee/ulixe-cloud
  • DockerHub: docker pull ulixee/ulixee-cloud

To use the image, we have a run.sh script that will run with a non-root user on your choice of port. All environmental configurations are listed here.

Developer Environment

This project serves as a Monorepo for developing the Ulixee Datastore, Desktop, Hero and Cloud. To install this project, you'll need to:

  1. Clone with --recursive so that submodules are initialized.
  2. Run yarn build:all from the main repository.

Learn more about Ulixee at ulixee.org.

Contributing

See How to Contribute for ways to get started.

This project has a Code of Conduct. By interacting with this repository, organization, or community you agree to abide by its terms.

We'd love your help in making Ulixee a better set of tools. Please don't hesitate to send a pull request.

License

MIT

Ulixee's Projects

browser-profile-data icon browser-profile-data

Analysis of Browser Headers, DOM, TLS, TCP, etc with Headed<->Headless comparisons and more.

chrome-versions icon chrome-versions

Versions of Chrome for Mac, Windows and Linux with auto-updating turned off.

hero icon hero

The web browser built for scraping

js-libp2p-crypto-secp256k1 icon js-libp2p-crypto-secp256k1

js-libp2p-crypto-compatible implementation of cryptographic signature generation/verification using the secp256k1 elliptic curve

net icon net

Type-checked connections from client to "core" with various transport layers

noderdom icon noderdom

Convert WebIDL files to W3C-spec compliant builds for NodeJs

pg-server icon pg-server

Postgres DB server emulator, proxy or honeypot

remark-slug icon remark-slug

plugin to add anchors to headings using GitHub’s algorithm

secret-agent icon secret-agent

The web scraper that's nearly impossible to block - now called @ulixee/hero

shared icon shared

Common utilities used by the Ulixee and Unblocked repositories

unblocked icon unblocked

A suite of tools for protecting the web's open knowledge.

utls icon utls

Fork of the Go standard TLS library, providing low-level access to the ClientHello for mimicry purposes.

vm2 icon vm2

Advanced vm/sandbox for Node.js

vue-runner icon vue-runner

Easily launch and run any *.vue file as a standalone app

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.