mozilla / mofo-allhands-event-app-data-processor Goto Github PK

2.0 23.0 5.0 53 KB

a Python script that fetches data from a Google spreadsheet, transforms to JSON, then commits a data file to a GitHub repo.

License: MIT License

Python 100.00%

mofo-allhands-event-app-data-processor's Introduction

MoFo All-hands Schedule App Data Processor

This script is designed as a data processor helper for mofo-allhands repo. If you want to use this script as is make sure your Google Spreasheet is based on the Google spreadsheet template we designed.

This Schedule App Data Processor script handles the following tasks

takes data from a Google spreadsheet,
and converts it to a JSON array where each object has keys corresponding to the spreadsheet column names,
then stores that JSON locally and/or automatically commits it to a GitHub repo (your schedule app repo, e.g., [MoFo All-hands schedule app](https://github.com/m ozilla/mofo- allhands)).

To run the script

Create a Python virtual environment and install all the required packages. See create a Python virtual environment docs for instructions.
Get API creds from GitHub and Google. See authentication docs for instructions.
Share your Google Spreadsheet with the Google Service Account that you created in Step 2.
Set values for environment variables and run source your-file-name.env
Running update_schedule() will execute these methods in succession. You can trigger it from the command line: python update_schedule.py

mofo-allhands-event-app-data-processor's People

Contributors

Stargazers

Watchers

Forkers

mmmavis flamingspaz

mofo-allhands-event-app-data-processor's Issues

Add .sample-env

Rename update_mozfest_schedule.py

Rename update_mozfest_schedule.py to update_schedule.py

Rename repo: mozfest-schedule-loader ---> schedule-app-data-processor

related issue:

mozilla/mozfest-event-app#171

Consider including "start time" as part of timeblock's "key" value

key of each timeblock object has to be unique. Let's consider including start time as part of the value so we can ensure there are no multiple timeblock objects sharing the same key

handle data for sessions marked as "All Saturday," "All Sunday," "All Weekend"

Trying to figure out the best way to display these in the schedule app. Current plan is to identify sessions marked as "All Weekend" and clone the data, assigning one copy to "All Saturday" and another to "All Sunday." That way the session will show up at the top of both days.

Adjust update_mozfest_schedule.py

Adjust update_mozfest_schedule.py so that it can parse data from the current Google Spreadsheet template: https://docs.google.com/spreadsheets/d/1zHFoJ0A36f_aRv60OwyL3kpDnOJhtAyOBasN9iZ8qBU/edit#gid=1272990187

allow commits to multiple repo branches

Facilitators are listed in reverse order from spreadsheet

on import, skip rows that do not have a githubIssueNumber

This will handle a couple things:

avoid importing rows like line 2, which is just usage notes for people editing spreadsheet
make sure we don't have broken sessions in the schedule app, which uses that ID number in a few ways

Make sure bool env vars are converted into the correct type

COMMIT_JSON_TO_GITHUB
PROMPT_BEFORE_COMMIT_TO_GITHUB

Make sure the values of these 2 env vars are converted to boolean (i.e., from True/False string to real boolean value)

12-hour format for 'start' property

@ryanpitts

Is this possible to format the start property so that it's in 12-hour format? So instead of "start": "14:15" we format it as "start": "2:15 PM"?

Abstract "Etherpad slug" field to "Notes URL"

for more detail see mozilla/mozfest-event-app#200

concatenate facilitator_1, facilitator_2, etc., into `facilitators` key in JSON

... or at least that's how the schedule app's js templates and search functions are currently set up.

e.g. https://github.com/mozilla/mozfest-schedule-app/blob/gh-pages/index.html#L87

That could be reconfigured depending on how we want to display these people's names on session detail pages. For instance, if they're supposed to be displayed as a bullet list, I could see just changing the templates there and leaving the JSON like it is now. @mmmavis any thoughts?

Turn a few configs into env vars with fallback values

TARGET_BRANCHES (default value: ['gh-pages'])
GOOGLE_SPREADSHEET_KEY (default value: empty string)
FETCH_MULTIPLE_WORKSHEETS (default value: True)
MAKE_LOCAL_JSON (default value: True)
COMMIT_JSON_TO_GITHUB (default value: False)

Load WORKSHEETS_TO_FETCH and WORKSHEETS_TO_SKIP from env vars

Update README.md

remember to mention

env vars
sample .env files (can start from copying this file)
share spreadsheet with the Google Service account
rename .sample-env to sample.env
((... to be added)

and

break README.md into several files
put these files on the (to-be-created) docs dir