mitodl / xbundle Goto Github PK

View Code? Open in Web Editor NEW

1.0 32.0 5.0 856 KB

xbundle tool for edX courses

License: Other

Python 34.12% HTML 2.62% JavaScript 63.25%

xbundle's Introduction

xbundle

xbundle converts back and forth between OLX and "xbundle" style XML formats. The xbundle format is a single XML file.

The OLX format is defined in this documentation.

Installation

python setup.py install

This will install xbundle and the xbundle_convert command-line tool.

Using xbundle in your code

To convert from xbundle to OLX

from xbundle import XBundle

bundle = XBundle()
# get input_path and output_path from user input
bundle.load(input_path)
bundle.export_to_directory(output_path)

To convert from OLX to xbundle

from xbundle import XBundle

bundle = XBundle()
# get input_path and output_path from user input
bundle.import_from_directory(input_path)
bundle.save(output_path)

Using the command-line tool

xbundle_convert convert /path/to/course /path/to/output.xml

xbundle_convert convert /path/to/output.xml /path/to/course

Run tests

xbundle_convert test

xbundle's People

Contributors

Stargazers

Watchers

Forkers

kwangkim amir-qayyum-khan mitsei robhaj

xbundle's Issues

fails to completely convert 2.01

Trying to convert the course at https://github.mit.edu/mitx/content-mit-201-2015_Fall

xbundle is failing to include the content within verticals, for nearly all verticals. For example:

https://github.mit.edu/mitx/content-mit-201-2015_Fall/blob/master/chapter/Week_1.xml#L2-L4

is bundled as:

<course org="MITx" course="2.01r_2" url_name_orig="2015_Fall" semester="2015_Fall">
    <chapter display_name="Week 1" url_name_orig="Week_1">
      <sequential url_name="seq_WO1" display_name="Weekly Overview">
        <vertical url_name="WO1"/>
      </sequential>

-- ignoring the contents of https://github.mit.edu/mitx/content-mit-201-2015_Fall/blob/master/vertical/WO1.xml

This may be the same root cause of this bug mitodl/lore#502

Add info on how to run tests to the README

@bdero thought it might be docker-compose run xbundle tox to do it.

Switch to argparse for command help and processing

Add travis.yml and coveralls.io

Switch to using entry point instead of script

duplicate xbundle.py file

There's a duplicate of the outdated xbundle.py in the root of the repo.

Improve test coverage to 90%+ and add test runner

Improve coverage, separate tests into their own folder.

xbundle -> OLX breaks on unicode characters

This is like #14, but when converting in the other direction.

content-mit-801r
Converting edX xml directory '/home/shawn/temp/courses/content-mit-801r' to xbundle file 'content-mit-801r.xml'
Oops, failed to add file /home/shawn/temp/courses/content-mit-801r/about/overview.html, error=All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters
done
Converting xbundle file 'content-mit-801r.xml' to edX xml directory '/tmp/extracted_content-mit-801r'
course id = 8.01
Traceback (most recent call last):
  File "/home/shawn/projects/mitodl/xbundle/bin/xbundle_convert", line 69, in <module>
    main()
  File "/home/shawn/projects/mitodl/xbundle/bin/xbundle_convert", line 56, in main
    bundle.export_to_directory(outfn)
  File "/home/shawn/projects/mitodl/xbundle/xbundle/__init__.py", line 452, in export_to_directory
    self.export_meta_to_directory()
  File "/home/shawn/projects/mitodl/xbundle/xbundle/__init__.py", line 471, in export_meta_to_directory
    open(join(dir, fn), 'w').write(k.text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 1904: ordinal not in range(128)

Example:

Newton’s Laws of Motion" contains a \u2019, instead of an apostrophe.

Test round-trip conversion with 8.MReVx

Achieve 90% coverage

make it python 3 compatible

0.3.0 Release

Includes python 3 compatibility and python 2 unicode bug fix.

Maybe also fix to #34

conversion breaks on unicode characters

Specifically the wrong character in "Newton'" here:

https://github.mit.edu/mitx/content-mit-801r/blob/master/policies/2014_Fall/policy.json

This file contains a unicode single quote instead of an apostrophe.

Result:

ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

Create Dockerfile for testing

This is mostly a developer convenience to have a reproduceable testing environment for xbundle

needs pylint love

Current score is 2.x out of 10.

Remove commented out code

Make xmllint dependency optional

Test round-trip conversion with 8.MechCx

Test round-trip conversion with 3.091

3.091 (particularly the exam version) is an outlier among course structures. However, there is a good chance that we will want to re-use its problems using a tool like LORE.

After some very brief testing with 3.091, converting to an xbundle seems OK, but converting the bundle back to OLX does not. This needs further testing.

Note that the 3.091 course repository is not open: https://github.mit.edu/mitx/content-mit-3091r-exam

conversion breaks when JSON file has spaces in the name

For example, this file:

https://github.mit.edu/mitx/content-mit-801cc/blob/master/policies/2014_Fall/policy%20backup.json