Code Monkey home page Code Monkey logo

reddit-booru's Introduction

RedditBooru


What is it?

RedditBooru is an image board site driven by content posted to subreddits.

Advanced Searching

Keyword search can be done and, while not as powerful as reddit's search, it's a goodly bit faster (I like to think). But, what RedditBooru provides above reddit or karmadecay is reverse image searching. This allows users to keep content reposting to a minimum. The search algorithm isn't perfect, but it works most of the time.

Image Hosting

Content hosting through RedditBooru ensures that what's uploaded is exactly what the end users will receive, no middle-man processing, scaling, or JPEGifying in between. Additionally, the original source can be directly attributed to an image so that proper props can be given. Galleries can also be created and modified thanks to login support via reddit OAuth.

Reddit Voting

Taking advantage of the aforementioned reddit OAuth, RedditBooru allows users to vote on images right through the site's interface. Combined with the fullscreen viewer, this makes browsing many subreddits worth of images very simple (and hopefully a nicer experience).

Are there any new features in the works?

RedditBooru is never finished! Here are some ideas that are being worked on/in the thought bucket:

  • Better support for mobile devices (and maybe native apps)
  • Subreddit statistics pages
  • Image/gallery view statistics for users
  • Leaderboards

I'd like to help. What can I do?

If you're here, you know the code is completely open for anybody to hack on. Found a bug or have a great idea? Open a ticket and we'll see what can happen. Want to write that code yourself? Make a pull request! Obviously, I can't guarantee that all requests will make it to the site (gotta make sure all works, and that features won't bring the server to its knees), but I'm very much open to people contributing. For help setting up your development environment, check out the next section.

Development Setup

If you want to get RedditBooru up and running for yourself, follow these steps.

NOTE - As RedditBooru prepares to move towards a more modular, microservice structure, this is all highly subject to change. Currently it's a mess, and it needs to be made not a mess.

Pre-requisites

You will need the following to run RedditBooru:

  • A webserver that can do URL rewriting. I personally use nginx, but anything should be fine
  • PHP 7+ (it's known working on PHP 7.4). You'll also need the following modules:
    • Memcache
    • Imagick
    • GD2
    • MySQL
    • curl
    • json
    • cli (for running cron jobs)
  • composer for installing PHP packages
  • MySQL/MariaDB 5.0+
  • memcache
  • nodejs for services and front-end build pipeline

Initial Setup

Assuming you have all of the above:

# Get the code
git clone https://github.com/dxprog/reddit-booru.git
cd reddit-booru

# Create the backend cache directory and make it writeable
mkdir cache
chmod 777 cache/

# Install composer packages
composer install

# Install node dependencies
npm install

# Assuming you have an empty database set up, create the schema
mysql -u YOUR_USER -p -D reddit-booru < schema.sql

# Build front end files
npm run build

Configuration

RedditBooru can be setup to run multiple frontends with a single backend (though some work is still needed). Given that, there are two configuration files:

  • config.php: This is the backend config which configures "global" things like database, AWS/S3 credentials, etc. Checkout config.sample.php for more info.
  • app-config.php: The configuration file for a frontend, things like images URL, image storage location, etc. Checkout app-config.sample.php for more info.

Paths

You can happily run the entirety of RedditBooru from the same directory. If you want split up backend and front-end, here's what you need to know:

Backend (not accessible by web server)
  • api/
  • cache/ - Writable directory for temporary image cache. This can be symlinked to the thumbnail cache on the front end.
  • config/
  • controller/
  • cron/
  • lib/
  • node_modules/ - You can copy this folder directory, or copy package.json and run npm install
  • services/
  • vendor/ - You can copy this folder directly, or copy composer.json and run composer install
  • views/
  • config.php
Frontend (point your web server here)
  • static/
  • app-config.php
  • index.php
  • upload.php - Deprecated. To be removed soon

URL Rewrites

RedditBooru expects the entire URL path to be passed in as the query string parameter _q and handles additional query string paremeters/routing itself. Example of nginx setup:

location / {
  try_files $uri $uri/ /index.php?_q=$uri;
}

Adding a Source

Sadly, this is a very manual process currently, requiring direct entry into the database. It's not difficult, just tedious. Here's what you need to know about the fields:

  • source_name - The name of the subreddit (including the r/). Ex: r/pics
  • source_baseurl - (DEPRECATED) Full URL path to the subreddt. Ex: http://www.reddit.com/r/pics
  • source_type - Type of source. The only currently supported option is subreddit. This will probably be deprecated and removed in the future.
  • source_enabled - Whether this source is enabled. 1 for enabled, 0 for disabled. Disabled sources will not be displayed nor processed.
  • source_subdomain (OPTIONAL) The subdomain where this can be directly accessed. Ex: pics -> pics.redditbooru.com
  • source_generate_report - Whether the stats bot will generate a monthly report for this subreddit. 1 for enabled, 0 for disabled.
  • source_content_rating - (DEPRECATED) Not used anymore and will be removed in the future.
  • source_repost_check - Whether to have the bot check for reposts and report/comment on them. 0 is disabled, any other number is the number of seconds in which reposting is not allowed. Ex: 86400 -> don't allow reposts for one day.

Getting Content

Content is inserted in one of two ways:

cron/cron.php

To manually update content, run the following from the backend directory:

php cron/cron.php --source=SOURCE_ID

This will scan that source and update/insert the latest 100 items. This can be run as a cron job for continual updates.

Service

The set-it-and-forget-it node script to continually updates all enabled sources. Using either screen (or some other detached-like terminal) or something like forever, run:

node services/task-runner.js

This service does currently have a hard coded URL to the API that returns all sources. Locate http://redditbooru.com/sources/ in task-runner.js and replace with the appropriate path. THIS IS GOING TO BE REPLACED WITH EXTERNAL SERVICE NODES VERY SOON!

Additional Services

There are some additional services rolled up into the main RedditBooru codebase. It's all very hacked together, so please excuse the mess.

SauceNao

The SauceNao service is a node script that manages requests for getting image sources via SauceNao without going over API limits. To get it running:

# SauceNao API key
export SVC_SAUCENAO_KEY = SAUCENAO_API_KEY

# Port to run the service on. SAUCENAO_PORT in config.php should be this same value. Someday, it'll use the env var too.
export SVC_SAUCENAO_PORT = 4444

# Run it. Use screen/forever/init.d script to persist
node services/saucenao-service.js

cron/clean.php (DEPRECATED)

Used to remove old cached files. It's recommended that you use find on a cron job instead.

ai-tan (DEPRECATED)

"ai-tan" provides a couple of features on two different scripts:

  • cron/repost_check.php - checks new posts to see if they're reposts (as dictated by the source's source_repost_check)
  • cron/ai-tan.php - generates and posts montly report of a source's activity (as dictated by the source's source_generate_report)

These scripts rely on an old PHP based reddit framework of mine that's terrible and outdated. Setting this up is non-trivial and won't be covered here as they are actively being replaced by better, stand-alone modules.

reddit-booru's People

Contributors

chiefnoah avatar cocoliliace avatar doomtay avatar dxprog avatar kdhuynh avatar mikairi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

reddit-booru's Issues

Uploaded high-resolution images eventually redirect to compressed 1080p JPEGs

Once I upload an image, I can usually directly link to it without loss of quality or resolution. After a certain amount of time, however, visiting that direct link will redirect me to a compressed JPEG file 1080 pixels long on the short edge.

An example being I regularly upload 5K PNG images such as this one. Upon upload completion it measures 3200 x 5120 in resolution. When I visit this link a few days later, this JPEG image as described above is returned, measuring 1080 x 1728 in resolution instead.

This happens on and off on almost every other upload.

Filter dropdown might not fit in viewport

Because of the increasing amount of subreddits RB supports, the filter dropdown is now so big that the checkboxes cannot all fit inside the screen that I use, but there is no way to scroll down the list.

I'm using a 13" MacBook Pro and using the browser that takes up the least amount of vertical space for controls (Safari) and maximum window height:
ae106d44-f747-4186-9170-cb60472aed8b

I normally use Chrome, and because the controls takes up more vertical space thus reducing the page height, the refresh button is inaccessible for me most of the time.

I'd suggest redesigning the dropdown, or reducing the font size.

Many sources breaks pagination (memcache key size limit)

If a large number of sources are configured, pagination breaks because memcache (and in particular the pecl-memcache implementation) has a key size limit of 250 characters. When a key exceeds this length it is silently truncated by pecl-memcache, which causes, in particular, first the timestamp then eventually the list of sources to be truncated.

A possible solution might be to use a hash of the current key.

For reference I'm experiencing this issue with about 75 sources configured, but depending on the source ids and the other parameters the problems might start occurring with less than that.

Drag and Drop doesn't give options anymore

Previously when I dragged an image file from my PC to redditbooru's main page it gave me an option to either "Search" or "Upload" and now that's gone, was this intentional ?

Add about page

There's one for animebracket, would probably be a good idea to have one here as well. Moreover, it'll be an ideal place to plug your patreon and link the github.

Image search results don't load on mobile

Is this viewport size dependent? Seems like on mobile the image search results just show the given image and the "seems like have been posted on ..." box, and the rest of the results don't load at all, when inspected I found they are not even created in DOM.

Duplicate results

The results of this image shows the same result four times. The creation time is the same, but the cdn.awwni.me urls all differ. I'm guessing it got accidentally uploaded as an album and tagged four times?

[Feature] Batch upload

Via multiple files drag drop or (of possible) selecting multiple files in the dialog.

Reverse image doesn't fully search

Reverse image search does not fully search all subreddits depending on subreddits ticked. If this is an intentional feature, would probably help to store temp cookies on Show NSFW and subreddit ticked options in order to help multiple image searches.

If those settings are saved, it might be an issue stemming with the chrome add-on overwriting existing cookies, as that is my way of usually searching for something.

SQL Schema

Setting up a (development) copy of this project would be much easier if there was an SQL schema file containing a skeleton database (schema and any basic required records).

Large vertical images in galleries

Currently in a gallery, if the height of an image is larger than the height of the viewport, it is not scaled, meaning that the user have to scroll to view the entire image, and the image can never be viewed in its entirety on the screen.

redditbooru-before



I propose scaling those images in CSS, or other methods, such that the user can see the entire image at once, without scrolling.

redditbooru-after

Install / Dependency documentation

Setting up a (development) copy of this project would be easier if there were a list of dependencies and, if possible, some basic install documentation (what services are expected to be available and where they're configured, as well as details of cron setup / example crontab).

Support for PHP7

Can you update your script to work on PHP7? cause php5.4 is no longer updated and can be easily hacked

Resizing galleries removes gallery images

Resizing a gallery page (e.g. http://redditbooru.com/gallery/o7bc/hibiki-cosplay-collection/) causes the images to disappear. This is also triggered when a user saves an image from the gallery page and a download bar causes the viewport to resize, or on Android Google Chrome when a momentum scroll stops.

On window resize, the ImageView resize code is called and replaces the #images.gallery containing the gallery images with a #images div containing an empty collection and a .moreRow. resize() assumes that the contents of #images is a plain ImageView, and doesn't resize galleries appropriately.

Clean up old ramps

Remove logic for anything ramped to 100% (user dropdown, source finder, mobile site).

Sorting in user pages

It would be nice if we can sort posts in user pages by "top", in addition to "new".

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.