Code Monkey home page Code Monkey logo

hitfactor.info's Introduction

Howler Monkey Classification System - HMCS

Independent Frontend/Webapp to bring better analysis and transparency into the USPSA Classification System. For a full drama background, see practical.shooting.insights on Instagram. For a quick-ish TLDR: see DRAMA.md Currently read-only, uploads planned (See issue #1)

Project Goals

  1. Analyze existing historical data and provide recommended HHF for all classifiers/divisions.
  2. Implement classification calculation and serve as an alternative shooter's classification source for expired members / shooters in protesting clubs.
  3. Implement alternative classification methods (percentiles, fluctuating, etc) for measuring shooter's skill
  4. Provide documentation and insight into classification system and present it to USPSA BOD after malicious actors are removed from the management.
  5. Integrate into USPSA.org website

Running

Locally - All Data
npm i
npm start

(Takes minute+ to hydrate the data and become responsive on M1Pro)

Locally - Quick Mode

For faster turn around when developing API, use:

npm i
npm run quick

(Hydrates in less than 5 seconds)

In Production

Dockerfile is only used by Koyeb. To run in prod mode, use:

npm i
npm run prod

Note: npm i is required, because it uses vite build as a post-install step and serves frontend files from the node itself, instead of running two processes concurrently.

Technical Stack
  • Main language: JavaScript (ES13), TypeScript when needed
  • Monorepo, Node/Fastify Backend, React (vite-swc) Frontend.
  • Backend serves API and static files: (build of React Frontend, downloadables, etc)
  • All data is loaded on node process during startup; no database (yet)
Folder Structure
  • scripts/ -- standalone scripts
  • data/ -- imported (partially processed / split) data, mostly used by backend
  • shared/ -- source code imported by both front- and backend
  • api/ -- backend
  • web/ -- frontend
  • package.json -- monorepo wrapper scripts

For more info, see READMEs in each root folder

Importing Data from USPSA

First you need to obtain and set ENV variables for API keys

  • ZenRows: use https://www.zenrows.com, set as ZENROWS_API_KEY
  • USPSA: you need Uspsa-Api header from mobile app, set as USPSA_API_KEY

Then run npm run import Usually takes a couple of hours on a pretty mediocre connection. Classifications and classifiers each have 258+ slices of 256 requests, each successful request is marked with a dot in the stdout. ZenRows still have (although rare) failures, usually manually refetching the slice with the failure by manually editing import.js is good enough.

Make sure to pay attention to git diff when comitting imports, ideally it should only have additions.

hitfactor.info's People

Contributors

codehowlermonkey avatar

Stargazers

 avatar Pietro Zarri avatar aniket avatar Tim avatar Noah Yetter avatar Jim Doran avatar

Watchers

 avatar

Forkers

pzsette qxvlada

hitfactor.info's Issues

ShooterCell: hover to show quick info

Could be a good way to quickly look up something about the shooter that isn't available in the ShooterTable or RunsTable.

Maybe we can even fit graph in there, and definitely some version of a card, to show other divisions.

Could be web only using existing endpoints api/shooters/A99999

Shooter(s): recommended class

should also include:

  1. Rec. Percent in classifier scores table
  2. Rec. GMs / Ms/ As in classifiers table
  3. "Rec HHF" and "CurHHF - RecHFF" columns in the classifiers table
  4. Rec. Percent & Rec. Class in Shooters Table
  5. Same as 3. for shooters info

Fix Tooltips

Tooltip component from prime react library is unreliable and has weird targeting.

Replace it everywhere where we have tooltips with something else

Fetch expired members

When classifier scores were fetched we filtered out the inactive (as of Jan 2024) members.
The fetch for classifications of members was done after that only against active members.

This resulted in some scores show up with X classification and "Expired / Not Found".

We need to refetch expired members and possibly support active/inactive expiration status in addition to what we have now.

Probably is blocked by #2

Classification Page: Inconsistencies Tab

  1. Add hq/cur/rec letters to api/shooters/:division
  2. Filter out where letters don't match
  3. Probably do different modes: hq-vs-cur, hq-vs-rec, I think cur-vs-rec isn't needed as both are synthetic
  4. review that classification percentage differences that we need for all modes are already there
  5. api can return all needed data and not be aware of the mode
  6. frontend can group by GMs/Ms by using multisort
  7. additional mode/toggle can be all/demotions/bumps

OG idea from howler monkey discord server, if you're from there and have something to add -- please leave a comment.

Consistent URLs for filters/sort/page/etc

Right now, things like filters, pagination, and sort aren't reflected in the URL.

It's not super hard to support this, but it requires significant effort for not-so-much of a reward (quick linking for sending your analysis data sources to someone else for discussion / review).

Parking this as a known issue / TODO / improvement

Add last import date to the UI

Probably can just use a simple fs.stat on one of the files updated by the import script.

Or even make import script write the date on start / completion.

Clubs Table

Mostly this is useful for catching cheaters / reviewing clubs that have shooters with sus scores.

See #22

Fix/improve LO/CO data

LO/CO is a synthetic division made out of a merge between LO and CO data.

It was hacked up as quick as possible, simply to make app not crash.
We need to review debug everything we can with LO/CO div and make sure it doesn't crash and also uses proper data.

Right now all HHFs for it are LO HHFs, and instead we should pick the highest from CO and LO. And maybe show both

Optimize In-Memory JSON Array/Objects Issues

Currently everything is stored and pre-hydrated on server start in-memory, often duplicating the same data multiple times for easier access (e.g. classifier runs by shooter or by division/classifier).

This causes 2 main problems:

  1. Server requires quite a lot of RAM to run
  2. It takes a noticeable time to hydrate the data, multiple minutes on M1 Pro Macbook, resulting in serious loss of productivity, when constantly changing the API and reloading the server.

We need to find a way to optimize both of these things, but the second one is more important, at least until we get to really high RAM usage, which will force us to change hosting (right now app idles at slightly above 4GB with legacy scores, and around 2 without).

Some ideas how to achieve this:

  1. Sequelize + sqlite in memory, with proper DB tables design and basically rewrite of the whole model layer. HUGE EFFORT, HIGH CHANCE OF REGRESSIONS, if you can do this -- you're basically a legit senior backend dev.
  2. RxDB, has in memory, is basically nosql, but has schemas, maybe can be hacked together moderately easy-ish. Higher Middle Dev Level
  3. Just make GB+ JSON files work, JSONStream, await-sync, etc. Anything that will simply save shit to disk and cache it there. Won't help with memory, probably will make it worse. But should make development faster.
  4. Any other ideas?

Shooters Table

top20 to infinity and beyond, use data from classifications at first and nothing else

clicking anything in that table goes to #3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.