Code Monkey home page Code Monkey logo

omnivore's Introduction

Omnivore

GitHub Workflow Status Discord Mastodon Follow Twitter Follow GitHub

Omnivore Logo

Omnivore is a complete, open source read-it-later solution for people who like text.

We built Omnivore because we love reading and we want it to be more social. Join us!

Every single part is fully open source! Fork it, extend it, or deploy it to your own server.

We also have a free hosted version of Omnivore at omnivore.app -- try it now!

web-screenshot-listview

Join us on Discord! ๐Ÿ’ฌ

We're building our community on Discord. Join us!

Read more about Omnivore on our blog. https://blog.omnivore.app/p/getting-started-with-omnivore

Shoutouts ๐ŸŽ‰

Omnivore takes advantage of some great open source software:

  • TypeScript - Most of our backend and frontend are written in TypeScript.
  • Next.js - Our frontend is a Next.JS app and is hosted on Vercel.
  • SWR - We do all our data fetching on the web using SWR.
  • Stitches - We use Stitches on the frontend to style our components.
  • Mozilla Readability - We use Mozilla's Readability library to make pages easier to read.
  • Swift GraphQL - We generate our GraphQL queries on iOS using Swift GraphQL.
  • Apollo GraphQL - We generate our GraphQL queries on Android using Apollo GraphQL.
  • Radix - We use Radix UI's components on our frontend.
  • And many more awesome libraries, just checkout our package files to see what we are using.

Importing Libraries

Check out our docs for information on importing your data from other apps.

How to setup local development ๐Ÿ’ป

The easiest way to get started with local development is to use docker compose up. This will start a postgres container, our web frontend, an API server, and our content fetching microservice.

Requirements for development

Omnivore is written in TypeScript and JavaScript.

  • Node -- currently we are using Node.js v18.16
  • Chromium -- see below for installation info

Running the web and API services

1. Start docker compose

git clone https://github.com/omnivore-app/omnivore
cd omnivore
docker compose up

This will start postgres, initialize the database, and start the web and api services.

2. Open the browser

Open http://localhost:3000 and confirm Omnivore is running

3. Login with the test account

During database setup docker compose creates an account [email protected], password: demo_password.

Go to http://localhost:3000/ in your browser and choose Continue with Email to login.

Frontend Development

If you want to work on just the frontend of Omnivore you can run the backend services with docker compose and the frontend locally:

docker compose up api content-fetch
cd packages/web
cp .env.template .env.local
yarn dev

You will need to configure some values in the new .env.local file. These are the values for running the web service directly on your host machine and running api and content-fetch within docker:

NEXT_PUBLIC_BASE_URL=http://localhost:3000
NEXT_PUBLIC_HIGHLIGHTS_BASE_URL=http://localhost:3000
NEXT_PUBLIC_LOCAL_BASE_URL=http://localhost:3000
NEXT_PUBLIC_SERVER_BASE_URL=http://localhost:4000
NEXT_PUBLIC_LOCAL_SERVER_BASE_URL=http://localhost:4000

Running the puppeteer-parse service outside of Docker

To save pages you need to run the puppeteer-parse service.

1. Install and configure Chromium

brew install chromium --no-quarantine
export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
export CHROMIUM_PATH=`which chromium`

2. Navigate to the service directory, setup your env file, and install dependencies

cd packages/puppeteer-parse
cp .env.example .env
yarn

3. Start the service

yarn start

This will start the puppeteer-parse service on port 9090.

In your browser go to http://localhost:3000/home, click the Add Link button, and enter a URL such as https://blog.omnivore.app/p/getting-started-with-omnivore.

You should see a Chromium window open and navigate to your link. When the service is done fetching your content you will see it in your library.

How to deploy to your own server

Omnivore was originally designed to be deployed on GCP and takes advantage of some of GCP's PaaS features. We are working to make Omnivore more portable so you can easily run the service on your own infrastructure. You can track progress here: omnivore-app#25

To deploy Omnivore on your own hardware you will need to deploy three dockerized services and configure access to a postgres service. To handle PDF documents you will need to configure access to a Google Cloud Storage bucket.

  • packages/api - the backend API service
  • packages/web - the web frontend (can easily be deployed to vercel)
  • packages/puppeteer-parse - the content fetching service (can easily be deployed as an AWS lambda or GCP Cloud Function)

Additionally, you will need to run our database migrations to initialize your database. These are dockerized and can be run with the packages/db service.

License

Omnivore and our extensions to Readability.js are under the AGPL-3.0 license.

omnivore's People

Contributors

jacksonh avatar sywhb avatar satindar avatar dependabot[bot] avatar rupin90 avatar gitstart avatar stefanosansone avatar podginator avatar sixten avatar remychantenay avatar holton-jiang avatar se-jaeger avatar gannonlawlor avatar 340rahul avatar aaachen avatar herealext avatar egrajeda avatar jeffbowen avatar consultorseovalencia avatar laurentkempe avatar peterdavehello avatar gitstart-omnivore avatar cdev147 avatar kristol07 avatar ismailghedamsi avatar echoriiku avatar suravshresth avatar shanemaglangit avatar prof18 avatar magsmagnoli avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.