pdgonzalez872 / atp_data_elixir Goto Github PK

eprof was failing. Handling it on the Elixir side.

No need to use flow. At least, try this strategy out. Sometimes we get errors. Would be cool to be able to retry if there is an error, specially a :timeout error

I understand the TODOs now. Could create a task that is run to capture whatever data is there and persist that, instead of waiting to successfully fetch everything.

Deploy to:

• Heroku
• Gigalixir
• Google Cloud

I don't like that we are prone and can't recover from timeouts when doing the fetching.

If there is a timeout, move on. Have it not stop the fetching mechanism. Great way to implement OTP.

https://github.com/pdgonzalez872/atp_data_elixir/blob/master/lib/atp_data_elixir/earnings_aggregator.ex#L11-L15

This code is running one query for the player list and another one for each player to load the earnings, so it's possibly running thousands of queries.

You could call Repo.preload on the list in order to issue just two queries, one for the list of players and another one for all their earnings.

Features:

• Line chart that displays the player/prize money revenue
• Materialize css

Create a EarningsAggregator that returns a list of tuples, sorted by earnings from the db.

Persist results in local db, have it display some cool charts maybe. (charts.js)

Eventually, deploy this to prod.

Players, Earnings

Player has many Earnings, Earnings belong to Player

The free tier on gigalixir will block past 10,000 rows.

So, instead of persisting the json with the EarningsAggregator, we can do it in the main task, without saving anything to the db.

create a latest_earnings table and store the list of tuples as json

Add doctests, try the coverage tool, maybe even use CI here for free