Code Monkey home page Code Monkey logo

chorus's Introduction

chorus

Clone Hero-friendly Organized Repository of User-provided Songs

Link to web app here

Introduction

The current state of Guitar Hero/Rock Band/Clone Hero custom songs aggregation is rather unsatisfying, scattered and newcomer-averse: this is an attempt to make it more searchable and user-friendly.

A good part of the sources are based on the official CH charts spreadsheet with a ton of songs from talented charters, including conversions from C3, as well as the official GH/RB setlists. Props to everyone who has added to the list as well! There are currently more than 20,000 charts indexed, and the number is still growing!

Disclaimer: I am not responsible of the charts and songs that are indexed by chorus (except the charts that are attributed to "Paturages"). If you have an issue with any of those, please contact the charters and/or the source owners (links to sources are provided for every song).

How it works

chorus periodically imports links to songs from a list of mostly Google Drive folders (manually provided in the sources folder). It crawls the provided folders, downloads the .chart/.mid and song.ini files (or entire archives) to parse them for information, formally saves the links (not the song bundles themselves) and their metadata in a database and discards the downloaded items. The web app simply reads and searches from the data that is saved in the database.

The search engine makes use of PostgreSQL's ts_vectors and trigrams (via the pg_trgm extension): it operates on the concatenation of artist/band, song, charter and source names. It seems to be working pretty well for now!

How to add your drive

Follow the instructions described here.

Local installation and development

Frontend

npm install dependencies, npm run start:frontend to boot the dev environment.

Full stack

(This might or might not work on Windows. Please reach out to me if you encounter problems.)

Configure a PostgreSQL database and get your Google API client ID and secret. Copy the conf.example folder to a conf folder and fill in the credential files.

npm install the node dependencies. npm run import to fill your database with all the drives. npm start to start the dev server for the web app.

npm run import <short_name> to run just one particular import script.

Fair warning: the very first run will take more than 12 hours, so make sure to run the script as a background task. The following runs should only take about an hour, depending on the amount of new charts. If you're not willing to go through the 12+ hours, feel free to ask me for a database dump on Discord (Paturages#9405) or elsewhere.

API documentation

/api/count

Yields the total amount of indexed charts

/api/random

Yields 20 charts picked at random

/api/latest

Grabs the 20 most recent charts

Query parameters

  • from (Number): Offset from which the 20 charts are fetched
/api/search

Searches according to a query string, yields 20 results

Query parameters

  • query (String): The search terms. It can either be generic words or an "advanced query" string composed of:
    • name="some name": Song name
    • artist="some artist": Artist/band name
    • album="some album": Album name
    • genre="some genre": Song genre
    • charter="some charter": Charter name (as documented in their song.ini/notes.chart)
    • tier_band, tier_guitar, tier_bass, tier_rhythm, tier_drums, tier_vocals, tier_keys, tier_guitarghl, tier_bassghl: Difficulty tier as defined in song.ini by the diff_* entries (number from 0 to 6, usually). For instance, tier_guitar=lt3 will look for tiers that are less than 3, tier_guitar=gt3 will look for tiers higher than 3.
    • diff_guitar, diff_bass, diff_rhythm, diff_drums, diff_vocals, diff_keys, diff_guitarghl, diff_bassghl: Which difficulty parts (easy, medium, hard, expert) are available. It is a 4-wide bitmap (1 bit per difficulty part): 1 is easy, 2 is medium, 4 is hard, 8 is expert. Add numbers together to make multi-part queries.
    • hasForced, hasOpen, hasTap, hasSections, hasStarPower, hasSoloSections, hasStems, hasVideo: self explanatory, 0 to query for absence, 1 to query for presence.
  • from (Number): Offset from which the 20 charts are fetched

/api/search, api/random and /api/latest both yield JSON in the following format:

  • roles (Object): Key is lowercase charter name, value is their "caption", a.k.a. what's gonna show up when they get hovered.
  • songs (Array of Objects): song entities containing:
    • name, artist, album, genre, charter, year
    • hasForced, hasOpen (Object containing parts that have them), hasTap, hasSections, hasStarPower, hasSoloSections, hasStems, hasVideo, isPack, is120 (true if chart only has one BPM marker and it is 120)
    • length, effectiveLength (Number, in seconds; effective length is the duration between the first and the last note)
    • tier_* entries (song.ini difficulty tiering)
    • diff_* entries (bitmaps of difficulty parts, see a bit above for explanation)
    • uploadedAt, lastModified ("last modified" is the timestamp of the latest file modification. It is null when it couldn't be found (e.g. GDrive folders and charts that could not have been scanned only have uploadedAt))
    • hashes: MD5 checksums
      • file: MD5 of the chart itself
      • Other entries are parts that contain checksums per difficulty part
    • noteCounts: How many notes are in a part, per difficulty part
    • link: Download link. It either directs to
      • If it is a GDrive archive, the "view" page, where you have to explicitly click on "download"
      • If it is a GDrive folder, the folder itself
      • Else, it will most likely be a direct download link
    • directLinks (experimental): These links should allow you to directly trigger a download if you access/wget them.
    • sources (Array): Where it was pulled from

Roadmap and random ideas

(you can definitely help if you want to!)

My roadmap is actually getting considerable enough for it to warrant its own GitHub project page. Check it out!

chorus's People

Contributors

afaul25 avatar azsry avatar cha0sbuster avatar dependabot[bot] avatar diamondrivlz avatar ffjdksl avatar joeyballentine avatar laka3000 avatar mangolatango avatar markyard44 avatar mintorment avatar muskit avatar naonemeu avatar oktoberfesttheenthusiast avatar pattty-kakes avatar paturages avatar pedox avatar spartax18 avatar tonybaroneee avatar triggerthedragon avatar tundral5z avatar twofacefear avatar uncouthpanda avatar vaderdragon avatar vanessamagick avatar vjcharthing avatar wagsii avatar welldoneshellfish avatar yolomcswagginz avatar zstolfi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

chorus's Issues

Miles - Electronic Boutique: update charter to HaleyHalcyon

Hi, this is HaleyHalcyon looking herself up on Chorus like a f***ing loser. Could you correct the charter of Miles - Electronic Boutique from Akira Sora (old alias) to HaleyHalcyon (current alias as of Anti Hero 2)? Thanks!

P.S. What do you think of changing the charter names on my older charts as "Akira Sora"?

Server down?

Every time I try to go to chorus, it either:

A: Gives me a bad gateway screen
B: Says there are no songs in the database

Why does not this work for me?

I already left the link of my G.Drive in public, but when searching for my nickname in the lobby of the Chorus or at least the songs that I created, none appears.

I'd like someone to respond instead of deleting my topics, that's chit.

download button not working

whenever I try to download a song, instead of the google drive or the pop up coming up, it says that the server isn't responding with a bullshit explaination from Cloudfare. WTF?!
screenshot 3

Query by ID

I'm currently working on building a chart downloader/manager using Chorus, and I'd like to be able to query on a chart's specific ID.

band and song search issue

I've noticed just recently when you search for a band and you get through the first page of songs and click on more songs it doesn't show more from the band rather shows Bloodhound gang, ewan dobson etc etc. Never had this issue before, just a heads up.

Direct download links

  • Get webContentLinks for GDrive files or a list of webContentLinks to download for folders

Extra ideas that might get done later:

  • Attempt to craft an in-browser downloader (separate page that takes in a list of links or chart IDs) that either downloads files via AJAX and zips the results or just triggers every download
  • This will probably get us closer to in-game song downloads (CHLauncher?) or desktop downloader apps

Rush song problem

I'm trying to download some Rush songs, but whenever I try it says I can't connect even though I'm connected to the internet and there are no other problems. Some songs are ok to download but it seems Neversoft's and Harmonix's charts are the specific ones I can't, and they're the only charters I can see.

Add "warnings" section

e.g. no audio, 120 BPM, have to rename .chart+audio (i.e. GH3 format), maybe pointers to download a whole GDrive folder, no main guitar part...

RockGamer

Why doesn't a single link from RockGamers content of RockBandNetwork songs work?

Edit: Nothing from RockBandNetwork titled songs have a link that works. it just says "Waiting for something to respond..." I own a Acer Nitro 5 Gaming Laptop with 1 Gig of internet and no internet issues even.... except this. Am I missing something here?

Edit x2: Songs I search for are He Sleeps in a Grove (Amberian Dawn) and Pandemonium (Chaotrope)

Rock Band songs not working

Every time I try to download a song from Rock Band, the zip has nothing and my computer it's either in an unknown format or it's damaged. Can you please restore all of the Rock Band songs?

song

Metallica - St. Anger have notes but any sound in the game please check and fix thank u :)

Clone Hero keys

Hello there,
I'm Tân, I'm Vietnamese => I don't have much money => I don't want to use money to buy a guitar for clone hero. So, I put the keys like this: 1,2,3,4,5, arrow up and arrows down.
For some reasons, I can't press 4,5 and another key. I can't do these combos:
1,4,5 2,4,5 3,4,5
Please help, I've tried 3 different keyboards and it's still not working.
Yours clone hero,
Tân
Ps: I'm vietnamese => Not the best at English

Song Removal

Hello!

All of my charts were just removed from chorus. My name is still in the drive, and that link is the one that should be linked to all of my charts. I would enjoy having my charts back up on chorus. If you need anything from me, let me know.

Search term listing all with that word

When you search for something, it's not a one-and-done search only THIS. It includes anything that has that term in it. It's the most noticeable in the genre search tag, since in other tags they can be specified. If you do 'genre="Pop"' it will bring up all genres containing pop, like Pop Rock, Europop, and J-Pop. There's no way to exclude other terms, it seems like, either, like a special character to only bring up Pop.

Not sure if this is a bug or intentional, but I figure I'd bring it to your attention. Sorry if it's not clear!

Fix song.ini encoding issues

Try to use iconv-lite to parse these files.

The encoding issues come from the fact that most of these song.ini files were edited on Notepad for Windows (or other editors that don't save in utf8).

Now the real zinger is: what if the file is actually in utf8 as it should be? We should first check if the file is valid in utf8 (check for � characters I guess?). If not, run iconv-lite with the proper Windows encoding.

(currently working on the exact same issue at work, on a different project: the alternative encoding we should try with iconv is latin1)

Song lengths are invalid when notes in a chart occur after the end of the song

Exhibit: An Endless Sporadic - Magic Machine (jdurand, Vortex Hero)

Theoretical solution: rely by default on the song.ini's song_length entry, fall back to the length calculation from the chart if non-existent (calculating from the song itself would be too much hassle bandwidth-wise).

As a bonus, this would allow to detect notes that occur after the end of a song!

[Idea] Relevance meter or query highlight when searching for a chart

For example, I look up "bathory" in chorus, and there's no charts of anything called "bathory" in this website yet (as of Oct 25th 2018) - I instead get a lot of other charts that don't have enough relevance to what I was looking for.

A relevance meter or a results highlight for search queries would be nice to have in situations like this. For example:

Search query: train 45

Relevance Artist Song title
IIIIIIIIII Jim Vancleve Train 45
IIIII Ozzy Osbourne Crazy Train
IIIII Rod Stewart Downtown Train

Some direct download archives contain ch.dat's or files begin with a backslash

So, recently I wrote a small fishell script to download songs from chorus, which you can find here.

In doing this I noticed some minor problems. Most notably, the downloads are absolutely massive. 50MB~ for a single song isn't abnormal from what I've seen. (Is this normal?) I suspect this is something to do with the ch.dat's that are contained in the download possibly.

Additionally, I noticed that some songs (mostly ones in offical games) have all their files prefixed by a leading backslash. (Possibly a Windows vs Linux issue? Not sure.) This causes the song to be bad until manually renaming every file to remove the backslash.

I also noticed that rather than GDrive links like most songs, the official games are hosted on chorus directly. Did you zip them from your local copy? That'd explain the ch.dat's being included.

[Idea] Add a song preview link/embed on charts on Chorus

Maybe some kind of script could upload the ogg or mp3 to an audio streaming service and fetch an embed from there, or just recognize the song itself without uploading anything.

And I dont mean a chart preview, but just the song.

Missing guitar.ogg

Hello,

I send this message to report a problem with the songs Moonrise by L-Train (and friends). They do not work on Clone Hero because the file guitar.ogg is missing. And the problem is for all the titles of the song (Harmony, Prelude....).

Hope it'll be fixe very soon.

Best regards.

Support other hosting solutions (i.e. write drivers for them), such as Dropbox, Mediafire, MEGA...

Helpers: a good first step would be to investigate the APIs and look for routes that allow us to:

  • Get the list of files/folders from a link that source contributors would provide (the link should direct to their charts directory)
  • Get wgetable download links to either archives or song.ini, .chart and .mid files

If you're brave and want to help with the next steps (creating the driver that integrates these APIs), you could read and document yourself on the existing drivers (google-drive.js, caddy.js), but I'll most likely implement these myself.

No background customizing option

Hey, so after April Fool's you added an option to customize the background, pretty cool. But since the site's last update, there's no way to change the background anymore (although the one I used before is still there).

Are you planning on taking out that feature? Please don't, I like it.

requesting some changes

Hello, could you please change my nickname from "rehannoor75" to "Fenix12" ? because im currently changing my in-game name and some people know me as "Fenix12"

rehannoor75's charts :: https://drive.google.com/drive/folders/1D4-_uFXyeIeGYYMRTDoEpAvcvJGRWPh2

Searching for something returns a blank page

Not really sure when this bug came out since I haven't used the page for a long time, but when you try to search for something, you don't get results, actually you don't even get the "no results" text.
I tested this on chrome and edge with the same results on both, also tried reloading the page but that didn't fix the problem.
image
image

Site is down?

It keeps directing me to a cloudflare snapshot, what's going on?

Chart curation system

As charters and chart reviewers, quality control is one, if not the main focus point for the rhythm gaming community. The idea is to reward and promote people who are putting real effort into making their stuff good.

  • Discord login: to let charters and/or chart reviewers manually curate, moderate and review charts. Consider opening up reviews for non-charters to a lesser extent (anonymous /5 rating).
  • Charters highlighting: Add fancy colors to charter names according to an internal whitelist (that will eventually be tied to CH Discord roles). Possibly add bias towards more skilled charters in the search results.

My songs in the Chorus

It seems like it did not help if I edited the text by putting the Google Drive link away from the audience, none of my songs appear on the Chorus start page and not even in the search, I think that was not enough or something very important is missing. not have worked out. Who knows how to help me here.

Search by Chart Checksum

Since CH stores the song score data using the chart checksum, it would be great if there was a way to search chorus for a chart given the checksum. Chorus already stores this checksum data for each chart. Currently it doesn't use that field for determining search results (AFAICT).

For example, c1c0cf3fa09fe036f75e8d60524edb4d is the checksum for Grim Birthday's chart of Something Human (the first result on https://chorus.fightthe.pw/search?query=Something%20Human), but when I search for that checksum, Something Human isn't even in the results (https://chorus.fightthe.pw/search?query=c1c0cf3fa09fe036f75e8d60524edb4d). This could probably be a search factor that either matches or doesn't match (We probably wouldn't need Something Human to be a result if the user searches for a substring of the hash, eg: 8d60524).

Related to #23, but that issue is for grouping the charts, whereas this would be to show the charts when the checksum is searched for.

Harmonix's charts won't download..

When i try to download Harmonix's charts, they won't download at all. When i download other charter's charts, they download with no problems.

Rock Band songs not working....again

As soon as I posted the error yesterday, the songs returned to normal. But once again, they no longer work when downloading...It has the same issue as before. Please fix this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.