Code Monkey home page Code Monkey logo

edx-val's Introduction

edx-val (Video Abstraction Layer)

edx-val is a django app that creates and retrieves metadata for videos and subtitles. When creating video entries, they can be assigned to preset profiles such as 'high_quality' or 'mobile_only'. When requesting a video, the client does not need to know which profile to retrieve, but only the edx_video_id of that video. Since all the different profiles for that particular video is returned, the client can decide which profiles they want to use.

Example: Retrieve all profiles for a video with `edx_video_id`="example"

>>> get_video_info("example")
{
    'url' : '/edxval/videos/example',
    'edx_video_id': u'example',
    'duration': 111.0,
    'client_video_id': u'The example video',
    'encoded_videos': [
        {
            'url': u'http://www.example.com/example_mobile_video.mp4',
            'file_size': 25556,
            'bitrate': 9600,
            'profile': u'mobile'
        },
        {
            'url': u'http://www.example.com/example_desktop_video.mp4',
            'file_size': 43096734,
            'bitrate': 64000,
            'profile': u'desktop'
        }
    ]
}

Developing

First, create a virtual environment:

virtualenv venvs/val
source venvs/val/bin/activate

To run tests:

make test

edx-val's People

Contributors

ali-d-akbar avatar awais786 avatar ayub-khan avatar azarembok avatar brianhw avatar cpennington avatar crice100 avatar dawoudsheraz avatar dianakhuang avatar edx-requirements-bot avatar feanil avatar gjanesch avatar gsong avatar iamjazzar avatar iamsobanjaved avatar iloveagent57 avatar irfanuddinahmad avatar jibsheet avatar jinder1s avatar jmbowman avatar mraarif avatar muhammad-ammar avatar nasthagiri avatar nedbat avatar noraiz-anwar avatar qubad786 avatar singuliere avatar usamasadiq avatar waheedahmed avatar zainab-amir avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

edx-val's Issues

When there is a file read error, it blocks exporting transcripts

for example:

[2019-01-04 14:28:38,337: ERROR/Worker-2] Failed export to xml
    Traceback (most recent call last):
      File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/git_export_utils.py", line 137, in export_to_git
        root_dir, course_dir)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_exporter.py", line 346, in export_course_to_xml
        CourseExportManager(modulestore, contentstore, course_key, root_dir, course_dir).export()
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_exporter.py", line 169, in export
        courselike.add_xml_to_node(root)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 602, in add_xml_to_node
        super(XmlParserMixin, self).add_xml_to_node(node)  # pylint: disable=bad-super-call
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1114, in add_xml_to_node
        xml_string = self.export_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 591, in export_to_xml
        super(XmlDescriptor, self).add_xml_to_node(node)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 454, in add_xml_to_node
        xml_object = self.definition_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/course_module.py", line 1055, in definition_to_xml
        xml_object = super(CourseDescriptor, self).definition_to_xml(resource_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/seq_module.py", line 674, in definition_to_xml
        self.runtime.add_block_as_child_node(child, xml_object)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1520, in add_block_as_child_node
        block.add_xml_to_node(child)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 602, in add_xml_to_node
        super(XmlParserMixin, self).add_xml_to_node(node)  # pylint: disable=bad-super-call
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1114, in add_xml_to_node
        xml_string = self.export_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 591, in export_to_xml
        super(XmlDescriptor, self).add_xml_to_node(node)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 454, in add_xml_to_node
        xml_object = self.definition_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/seq_module.py", line 674, in definition_to_xml
        self.runtime.add_block_as_child_node(child, xml_object)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1520, in add_block_as_child_node
        block.add_xml_to_node(child)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 602, in add_xml_to_node
        super(XmlParserMixin, self).add_xml_to_node(node)  # pylint: disable=bad-super-call
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1114, in add_xml_to_node
        xml_string = self.export_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 591, in export_to_xml
        super(XmlDescriptor, self).add_xml_to_node(node)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 454, in add_xml_to_node
        xml_object = self.definition_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/seq_module.py", line 674, in definition_to_xml
        self.runtime.add_block_as_child_node(child, xml_object)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1520, in add_block_as_child_node
        block.add_xml_to_node(child)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 454, in add_xml_to_node
        xml_object = self.definition_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/vertical_block.py", line 160, in definition_to_xml
        self.runtime.add_block_as_child_node(child, xml_object)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1520, in add_block_as_child_node
        block.add_xml_to_node(child)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 602, in add_xml_to_node
        super(XmlParserMixin, self).add_xml_to_node(node)  # pylint: disable=bad-super-call
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1114, in add_xml_to_node
        xml_string = self.export_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 591, in export_to_xml
        super(XmlDescriptor, self).add_xml_to_node(node)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/xml_module.py", line 454, in add_xml_to_node
        xml_object = self.definition_to_xml(self.runtime.export_fs)
      File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/video_module/video_module.py", line 717, in definition_to_xml
        course_id=unicode(self.runtime.course_id.for_branch(None))
      File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edxval/api.py", line 864, in export_to_xml
        return create_transcripts_xml(video_id, video_el, resource_fs, static_dir)
      File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edxval/api.py", line 924, in create_transcripts_xml
        static_dir=combine(u'course', static_dir)  # File system should not start from /draft directory.
      File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edxval/api.py", line 882, in create_transcript_file
        transcript_data = get_video_transcript_data(video_id, language_code)
      File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edxval/api.py", line 292, in get_video_transcript_data
        return dict(file_name=video_transcript.filename, content=video_transcript.transcript.file.read())
      File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/db/models/fields/files.py", line 51, in _get_file
        self._file = self.storage.open(self.name, 'rb')
      File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/files/storage.py", line 38, in open
        return self._open(name, mode)
      File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/storages/backends/s3boto.py", line 386, in _open
        raise IOError('File does not exist: %s' % name)
    IOError: File does not exist: video-transcripts/34027979d3504b32a0cee193bf6f3227.sjson

Upstream "catch IOExceptions and raise custom TranscriptsGenerationException"

We originally merged PR #2 to address an issue with export to git, mitodl/edx-platform#104.

Before we had a chance to deploy and test, however, the issue resolved itself (probably due to a configuration change and/or from moving static files to a new location).

That said, this still seems like a valuable code change and a better fit for the original intention of the code. So, we probably want to keep it. And if we keep it, we should submit it upstream to edX.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.