tyruiop / syncretism Goto Github PK

It appears in both datops-crawler and datops-compute. Such code is likely to appear mode in the future (for greeks or other financial calculations). It would make sense to refactor it all in a separate library (e.g. datops-financial).

Currently, one needs the chain ID (i.e 'AAPL210730C00145000'). Would it be possible to either have an endpoint that returns all of those chain numbers based on the ticker, expiration and call/put, or be able to call historical based on those parameters.

The filter list is becoming quite long and will keep getting longer.

Solution would be to only show a short list of default filters, and have a way for the user to add more filters by selecting them among a list of available filters.

If we create a PWA it will allow the user to

1. Make less requests to the BE
2. Work offline
3. Keep their preferences over time.

Note that the "state" should be able to be downloaded and then uploaded to restore settings.

For each option, keep track of the volume average and add a filter allowing to search by volume variance.

The volume average calculation should be done at the same time as monthly yield update.

1. Looking for Ticker search to find cheap / long dated options for known ticker.
2. Looking for Vol:OI ration in the search list.
3. Looking for Calls val:Put Vol ratio in the search list.
4. Denote "N" as like events for major News impacting options /share.
5. Looking for "DTE" days to expiry in the search result to know how many days this option will expire.
6. Sort by Volume or OI in result page (with an arrow next to Volume).
7. Alerting on % change in Volume or price of the option (below 20 days moving average) would be good.

On the frontend, the user should be able to click on a column's header to sort by it (like on the old frontend)

Currently all the data is saved in date/TICKER.txt.gz (which are compressed 1 edn per line files) as we crawl.
We need a way to generate and store time series based on said data.

• Normalize & align existing data (so all time series are the same, e.g. two measurements daily, at the same time, interpolate if needed)
• Save corresponding vector somewhere
• Update vectors at the end of each day

This is crucial to be able to calculate the greeks, and compare options among themselves.

Would like to have % IV changes to see option buys or option sells.

+IV % changes --- option buys.
-IV% change ---Option sells.

Currently all options are crawled at the same rate, however some options have little to no variation over extended periods of time while others change rapidly during the day.

• We need a heuristic to detect when a [ticker, expiration date] pair becomes active and high
• We need the crawler to still provide coverage guarantees (e.g. make sure that all such pairs are crawled at least once a day).

When the user clicks on the contract symbol, he should be able to see charts plotting the historical data

It is currently hardcoded in the config file of the crawler (and I think in the compute library), it should be automatically gathered.

See https://fred.stlouisfed.org/series/DGS10

Tracked options should be refreshed (e.g. once per hour for now).

Not sure if this is resolved..

When i tried last night volume filter throws volume + more option with out of band volume filtered.

The Home view should display, for the moment, tracked options.

In the future we might want to include general statistics about the market, this issue is quite open for now.

The crawler is gathering almost all required data to calculate the greeks as we crawl (the only one missing is the risk free interest rate, which can be hardcoded).

So let's add this calculation (tentatively) to the crawler. If it proves to be too demanding (given the crawler's speed requirement), we will move it to regular computation in the datops-compute module.

Greeks' formulas can be found here https://www.macroption.com/black-scholes-formula/

Two options:

• use localStorage to save these (simpler, no need for login/pass)
• create a proper user space (more difficult, but probably useful in the future & can be accessed from different devices).

Do a form of excel lite, where one can:

• select which columns appear on FE (to reduce the clutter)
• create new columns based on the values of the other columns
• save these custom columns.

Now that some tests have been written, it would be nice if we got some CI going on.

Users should be able to search by how much an option's feature deviates from its historical average (20d & 100d).

The standalone client should allow a variety of things:

• Connect to the main Syncretism server & match all features from the frontend
• Integrate an event driven backtesting system
• Allow simple integration with external APIs (TD/IB)
• Use Syncretism's global market watch to trigger more refined data extraction from TD/IB
• Allow user to define patterns and be alerted when identified in live data.
• Communicate with smartphone companion apps.
• Allow simple control of a local instance of the crawler and of the backend
• Possibility to publish user defined alerts on the Syncretism server and for other clients to subscribe to them.

Once the search engine becones a PWA, we should allow the user to select and track some options in particular, not to have to search them again every time.

Also potentially implement paging instead of appending to same table

Currently done by hand, need to create a crontab for it.

• add DB dump link somewhere.

Using a statement like

SELECT timeseries.contractSymbol, AVG(timeseries.delta) FROM timeseries LEFT JOIN live ON timeseries.contractsymbol = live.contractsymbol WHERE live.expiration > 1627721273 GROUP BY timeseries.contractSymbol;

at the end of each days, update the live table with the average value of (most) columns as we go, to enable search by average.

Add two averages: 20days & 100 days averages for each column, for now.

This will allow identifying drops & peaks in movements of each contract.

There are currently no tests whatsoever in the codebase.

I need to write tests at least for

• datops-crawler
• datops-compute
  And reach a good test coverage.

e.g. be able to search options with a given filter and see their immediate successor on the ladder, e.g. strike 90 -> 95 for calls, or 90 -> 85 for puts

• datops-compute should calculate ladders at the end of each day (only for options that require it).
• Make a way for frontend to show this (sort by contractsymbol if the filter is active?)

Once the data is generate and available, create an API to be able to request this data.

Allow for filtering of data set by IV range as part of the query.

Hey is this still crawling and updating? We are no longer getting data from the historical endpoint:

OpenBB-finance/OpenBBTerminal#1857

The frontend is quite broken on smartphones, the worst being the Home view and the charts.

We need to fix the CSS (and possibly the chart generation).

Dear Team,

Thanks for your effort.

In order to secure queries & improve performances:

• Use FIND_IN_SET where applicable
• Rewrite with ? like it is done for batch queries, for all queries
• Allow for results pagination

See https://stackoverflow.com/questions/13116042/how-to-pass-a-variable-to-a-in-clause

User should be allowed to set a timer for a given filter (e.g. 1 / hour) and have it be automatically refreshed following that timer (only during market hours).

Solution:
Allow multiple "listing" views in the options tab, each with its own associated fitler & timer.