Kolibri is an open source educational platform to distribute content to areas with
little or no internet connectivity. Educational content is created and edited on Kolibri Studio,
which is a platform for organizing content to import from the Kolibri applications. The purpose
of this project is to create a chef, or a program that scrapes a content source and puts it
into a format that can be imported into Kolibri Studio. This project will read a
given source's content and parse and organize that content into a folder + csv structure,
which will then be imported into Kolibri Studio. (example can be found under examples
directory.
The expected compressed file size after running is 900Mb
./souschef.py
or
./run.sh
-
Install Python 3 if you don't have it already.
-
Install pip if you don't have it already.
-
Create a Python virtual environment for this project (optional, but recommended):
- Install the virtualenv package:
pip install virtualenv
- The next steps depends if you're using UNIX (Mac/Linux) or Windows:
- For UNIX systems:
- Create a virtual env called
venv
in the current directory using the following command:virtualenv -p python3 venv
- Activate the virtualenv called
venv
by running:source venv/bin/activate
. Your command prompt will change to indicate you're working insidevenv
.
- Create a virtual env called
- For Windows systems:
- Create a virtual env called
venv
in the current directory using the following command:virtualenv -p C:/Python36/python.exe venv
. You may need to adjust the-p
argument depending on where your version of Python is located. - Activate the virtualenv called
venv
by running:.\venv\Scripts\activate
- Create a virtual env called
- For UNIX systems:
- Install the virtualenv package:
-
Run
pip install -r requirements.txt
to install the required python libraries.
A sous chef is responsible for scraping content from a source and putting it into a folder
and csv structure (see example examples/Sample Channel.zip
)
A sous chef skeleton script has been started for you, see souschef.py
.
Sous chef instructions can be found here
For more examples, see examples/openstax_souschef.py
(json) and examples/wikipedia_souschef.py
(html)
Please make sure your final chef matches the following standards.
- Does the resulting folder structure match the expected topic tree?
- Are the Channel.csv and Content.csv files valid (no missing files, data formatted correctly, etc.)?
- Does the code work (no infinite loops, exceptions thrown, etc.)?
- Are the source_ids determined consistently (not based on a changing url path, in same location every run, etc.)?
- Is there documentation on how to run it (including extra parameters to use)?
- Are there no obvious runtime or memory inefficiencies in the code?
- Are the functions succinct?
- Are there comments where needed?
- Are the git commits easy to understand?
- Is there no unnecessary nested
if
orfor
loops? - Are variables named descriptively (e.g.
path
vsp
)?
- Is the code compatible with Python 3?
- Does the code use common standard library functions where needed?
- Does the code use common python idioms where needed (with/open, try/except, etc.)?