dpopadic / foostrat Goto Github PK

Having calculated 3 factors so far (goal superiority, home advantage, form), a single factor for all competitions currently considered (~50) has ca 3.5 million rows:

Planned are ca. 30-50 factors. This would lead to at least a 400 million rows factor library. This will be an issue from a computational perspective. Alternatively, a factor library by competition (eg. Premier League) could be created. Needs investigation.

to schedule updates, the relevant project paths must be correctly specified:

write a file test_task.py..

# test task ----------------------
import os
from datetime import datetime
# relevant working directory..
# if in project: proj_path = os.path.join(os.getcwd(), '')
proj_path = '/Users/dariopopadic/PycharmProjects/fooStrat/'
fl = proj_path + 'append.txt'
myFile = open(fl, 'a+')
myFile.write('\nAccessed on ' + str(datetime.now()))
myFile.close()

write the job using crontab module in python:

# test schedule ----------------------
from crontab import CronTab # python -m pip install python-crontab
# access the system..
cron = CronTab(user=True)
# create new cron job..
job = cron.new(command='python /Users/dariopopadic/PycharmProjects/fooStrat/test_task.py', comment = 'foostrat')
# schedule for every minute..
job.minute.every(1)
# write job to cron tab..
cron.write()

# analytics:
# view all cron jobs..
for job in cron:
    print(job)
# job.enable(False)
# job.is_enabled()
# remove all jobs..
# cron.remove_all()
# in terminal: list / remove all crontab jobs: crontab -l / crontab -r
# update the job..
for job in cron:
    if job.comment == 'foostrat':
        job.minute.every(10)
        cron.write()

kmeans and others from scratch: https://www.johnwittenauer.net/machine-learning-exercises-in-python-part-7/

matplotlib load after installation fails with PyCharm:

import matplotlib.pyplot as plt

tried matpltlib==3.1.1 but didn't resolve

reverse engineer market odds to determine the most important factors

failing to process the following data: flib_argentina_copa_de_la_liga_profesional.pkl

fix the data source download as it's no longer valid

Rolling functions double checks.

looking at Japanese league in separation we get the following factor data for goal superiority factor with k=3 (lagged):

However, when performing calculations across leagues we get the following:

To setup necessary certificates (default certificates) for urllib request (http data download), do:

1. python -m pip install certifi
2. in terminal: /Applications/Python\ 3.7/Install\ Certificates.command

controls need to be constructed for the factors so when new data or methodology/syntax changes and where this results in different values, warning messages or errors need to be thrown!

Transform fooStrat to a pkg. Possible module structure:

fooStrat

• dman: ret_xl_cols
• stands: comp_league_standing
• flib: fgoalsup
• futils: neutralise_scores
• strates: comp_mispriced
• utils: anti_join