Code Monkey home page Code Monkey logo

pycollab's Introduction

PyCollab

Python Blackboard Collaborate script to download recording based on Blackboard Learn Course ID, or Blackboard Learn Course UUID, Moodle plugin session ID, Moodle LTI Tool. and get:

  • Recording Report in a CSV file (Recording_ID, Storage Size, Duration, Creation Date, Duration) on /reports folder
  • Downloads local folder that will receive the MP4 video if the recording have chats, those will be downloaded too. Recording and Chat csv files will be on /downloads folder
  • Command line attributes depending on the scenario: Learn Course,Blackboard Learn Course UUID, Moodle plugin session ID, Moodle LTI Tool.
  • This script is NOT supported on Behind the Blackboard.

1. Python Instalation

You need to have installed from Python 3.7 to Python 3.8.6

Will depent on Computer operation system how Python is referenced on command line tool:
Mac OS:

  • (HomeBrew) brew install python3
  • normally python3 is the alias to run python scripts when install python 3.7+
Windows OS:
  • Python 3.9+ still do not support PEP 157 package instalation, so must be installed Python 3.8.6 (x86_64 or x86 installer depending on your computer processor)
  • Normally python is the alias to run python scripts when install python from version 3.7 to Python 3.8.6


2. Install script requirements

MacOS:

pip install -r requerimientos.txt

Windows:

python -m pip install -r requerimientos.txt

3. Add Blackboard Collaborate and Learn Credentials

  • Scenario 1: Learn to Collaborate search for recordings
  • Scenario 2: Moodle to Collaborate search for recordings
  • Scenario 3: Collab Admin Institutional Reports

if you have the scenario 1 you need to insert both Learn and Collaborate credentials
If you have the scenario 2 or scenario 3, yo only need to insert Collaborate credentials, but DO NOT REMOVE the key:value association of Learn credentials section from the Config.py file.
edit content of Config.py file

Note: Backboard Credentials / Integration on Learn

In order to get the Learn credentials, go to developer.blackboard.com and register from there to grab the Learn credentials for their application, it is also imperative to remind them that they are creating an application based on your code, so they need to register as a developer.Then on your Blackboard Learn environment, as admin role user, go to Rest API Integration a create an integration using data provided from developer.blackboard.com before.

Check Learn Integration

Note: Backboard Credentials for Collaborate

Now, for Collaborate production they CAN and MUST create a ticket on behind the blackboard requesting their credentials.

4. Modify external file depending on scenario

edit learn_courses.txt file
edit learn_uuids.txt file
edit moodle_lti_id.txt file
edit moodle_plugin_sessions.txt file

for Scenario 3
downlaod the Collaborate Admin Institutional Attendance Report and put on the root directoriy
downlaod Collaborate Admin Institutional Recordings Report and put on the root directoriy

5. Run the script

  • Collab.py -h
  • CollabMoodle.py -h
  • CollabReport.py -h
  • CollabMinutes.py -h
  • CollabRecordings.py -h
  • CollabRecordingsDownload.py -h

Scenario 1

Search recording from Blackboard Learn to Collaborate

  • if you have the Blackboard Learn course id(s) as input data on the file learn_courses.txt, where -w is a value of weeks back for as starting point of searching for recordings: Note: if the recording have chats, those will be downloaded too.
  • python3 Collab.py -f learn_courses.txt -w 10   
    
  • if you have the Blackboard Learn UUID(s) as input data on the file learn_uuids.txt, where -w is a value of weeks back for as starting point of searching for recordings: Note: if the recording have chats, those will be downloaded too.
  • python3 Collab.py -e learn_uuids.txt -w 10   
    

    Report Learn-Collaborate

  • If you need to know about recording storage size, duration and recording ID before download any recording you can create a report, where -f is point to learn_courses.txt file that have Blackboard Learn courses ID listed by row, and -w is a value of weeks back for as starting point of searching for recordings. Note:If the recording has public_access disabled, the recording won't be readed by script and will 403 report will be generated with list recording_id.
  • python3 CollabReport.py -f learn_courses.txt -w 10
    

    Scenario 2

    Search recording from Moodle to Collaborate

    If you have the Moodle with Plugin Integration

  • Add one per row,the sessionuid column values from mdl_collaborate table from Moodle database on the file moodle_plugin_sessions.txt.
  • -w is a value of weeks back for as starting point of searching for recordings
  • If the recording have chats, those will be downloaded too.
  • Then run the command:

    python3 CollabMoodle.py -s moodle_plugin_sessions.txt -w 10   
    

    If you have the Moodle with LTI Integration

  • Add one per row, the id column values from mdl_lit table from Moodle database on the file moodle_lti_id.txt, check that course column value from from mdl_lit table match with id value from mdl_course table.
  • -w is a value of weeks back for as starting point of searching for recordings
  • If the recording have chats, those will be downloaded too.
  • Then run the command:

    python3 CollabMoodle.py -l moodle_lti_id.txt -w 10   
    

    Scenario 3

    Collaborate Admin Institutional Reports

    Report Minutes on Callaborate from AttendeeReport

  • if you need to identify the amount of minutes spend in Collaborate session,first you need to download the Collaborate Admin Institutional Attendance Report according to your zone (ca:Canada, us: USA), then use the script CollabMinutes.py with the parameter -f that point to the file previously downloaded.
  • python3 CollabMinutes.py -f AttendeeReport.csv   
    


    Report RecordingId from Collaborate RecordingsReport

  • if you need to get information about recordingId, storage size, first you need to download the Collaborate Admin Institutional Recording Report according to your zone (ca:Canada, us: USA), then use the script CollabRecordings.py with the parameter -f that point to the file previously downloaded. Note:If the recording has public_access disabled, the recording won't be readed by script and will 403 report will be generated with list recording_id.
  • python3 CollabRecordings.py -f RecordingsReport.csv   
    


    Download RecordingId from Collaborate RecordingsReport

  • if you need to download recordings, first you need to download the Collaborate Admin Institutional Recording Report according to your zone (ca:Canada, us: USA), then use the script CollabRecordingsDownload.py with the parameter -f that point to the file previously downloaded. Note:If the recording has public_access disabled, the recording won't be readed by script and will 403 report will be generated with list recording_id.
  • python3 CollabRecordingsDownload.py -f RecordingsReport.csv   
    


    Delete recordings

    Considerations

    About deletion process

    • Use this feature at your own risk
    • It is soft-deletion, this means, deletion process will remove recording after 30 days from deletion day
    • When deleted from API, It is not taken in count for total calculation of storage for Collaborate Entitlements
    • On reporting only takes in count Blackboard Learn-Collab in Blackboard Data, there is no way to caculate Collab Scheduler total storage using API
    • It is not possible to recover a deleted recording using Collaborate’s API, for that cases will need to create a Blackboard Support ticket for it
    python3 CollabDeleteRecordings.py -f recordingsid_list.txt   
    

    Where recordingids_list.txt is a text-plain o-column-list with all recordingsId that will be deleted from Blackboard Collaborate instance.

      ____    _                  _      _                                  _ 
     |  _ \  | |                | |    | |                                | |
     | |_) | | |   __ _    ___  | | __ | |__     ___     __ _   _ __    __| |
     |  _ <  | |  / _` |  / __| | |/ / | '_ \   / _ \   / _` | | '__|  / _` |
     | |_) | | | | (_| | | (__  |   <  | |_) | | (_) | | (_| | | |    | (_| |
     |____/  |_|  \__,_|  \___| |_|\_\ |_.__/   \___/   \__,_| |_|     \__,_|
                                                                                 
    

    Tutorials

    Downloading - English

    Downloading - Español

    pycollab's People

    Contributors

    sfc-gh-ccuarez avatar

    Stargazers

     avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

    Watchers

     avatar  avatar  avatar  avatar  avatar  avatar  avatar

    pycollab's Issues

    auth:setToken ERROR: Response 400

    Hi,

    I've found other error, this seems to be related to when the Learn REST API application has reached its maximum requests quota:

    [auth:setToken()] ERROR: <Response [400]>
    [auth:setToken()] ERROR: <Response [400]>
    Traceback (most recent call last):
      File "/usr/local/opt/PyCollab/controladores/AuthControlador.py", line 57, in getToken
        token = self.cache['token']
    TypeError: 'NoneType' object is not subscriptable
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "Collab.py", line 16, in <module>
        course_uuids.append(webService.getUUID(curso))
      File "/usr/local/opt/PyCollab/webService.py", line 58, in getUUID
        credencial = {'Authorization' : 'Bearer ' + self.getToken()}
      File "/usr/local/opt/PyCollab/webService.py", line 35, in getToken
        return(self.sesion.getToken())
      File "/usr/local/opt/PyCollab/controladores/AuthControlador.py", line 67, in getToken
        return self.cache['token']
    TypeError: 'NoneType' object is not subscriptable
    

    Guessing it would be helpful to handle this error with a friendly message to help the person executing the script to request an increase in their quota.

    Thanks!

    RecordingID in Scheduler Report

    Hi,

    I have already downloaded the recordings through the scheduler using the scheduler report. Now I want to delete the recordings.
    What is the RecordingID in the report?
    SessionIdentifier
    SessionInstanceIdentifier
    ExternalContextID
    RecordingLink

    Thanks

    Support on READ.me

    Please add a support title on READ.me letting users know that this is NOT supported on behind the blackboard.

    Adicionar búsqueda por id de la sesión para los que usan mooodle plugin collaborate

    El código del proyecto PyCollab es muy útil, pero en el caso de nuestra universidad tenemos open LMS con plugin para collaborate ultra no LTI, la búsqueda por nombre de la sesión nos arroja varias grabaciones debido que no se maneja un nombre único de la sesión, nuestros usuarios suelen usar nombres genéricos somo por ejemplo: "Sesión 1".

    python CollabMoodle.py -s moodle_plugin_sessions.txt -w 100

    ADJUNTO1

    Ability to pull recordings based on recording ID

    A really nice feature would be the ability to pull recordings based on their Recording ID and not just the UUID that is associated to a course. Some recordings are not in a course and the only way to get that is through the recording report that has to be manually created... But if this can be modified to pull recording ID, all recordings can be downloaded with their ID's only and no report or course names are needed.

    UUID name duplicate with video different

    Hi guys! I'm trying download video course uuid, however, when the download is finished, the name uuid its the same to all videos download. What's can is happening ?

    Download recordings from Moodle Plugin

    Some information from the APIS that I can use or what other data I can obtain from the session to be able to identify which course ID the recording belongs to, because the downloads are made by the session name and in all the courses of our LMS it has the Collaborate class names with similar name so cannot be differentiated.

    If you can support with some information. Thanks.

    More information on read.me and

    Hello Carlos,

    We need to add a couple of things on the read.me:

    First, we need to make sure that our clients understand the following:

    In order to get the Learn credentials, they do not to open a case on behind the blackboard nor email [email protected].

    They need to go to developer.blackboard.com and register from there to grab the Learn credentials for their application, it is also imperative to remind them that they are creating an application based on your code, so they need to register as a developer.

    Now, for Collaborate production they CAN and MUST create a ticket on behind the blackboard requesting their credentials.

    So, can you please add this in:

    1. Read.me
    2. config.py (as comments?)

    Appreciate your time and help!

    Charmap Error in downloadRecordingsUUID

    Hi there !!

    I'm getting the following error in some video downloads with PyCollab.
    The download starts, but it fails in some point.
    I don't know if it is a problem with PyCollab or some lib you are using.

    Traceback (most recent call last):
    File "D:\Repositorio_externos\PyCollab\CollabRecordingsDownload.py", line 35, in
    ut.downloadRecordingsUUID(recording_lista)
    File "D:\Repositorio_externos\PyCollab\Utilidades.py", line 299, in downloadRecordingsUUID
    print(fullpath + filename)
    File "C:\Python38\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
    UnicodeEncodeError: 'charmap' codec can't encode character '\u0327' in position 93: character maps to

    403 Errors

    Hi,

    I am getting 403 errors unless I flip the Access on the recording to Public. The default currently for Collab Ultra is to only have course members have access to recordings.
    Is there a work around for this?

    Thanks

    KeyError: 'storageSize'

    Hi!

    Thanks for the script, I'm receiving this error when trying to execute a medium set of courses (>500) with 1 (one) week of recordings parameter:

    Traceback (most recent call last):
      File "Collab.py", line 26, in <module>
        grabaciones = ut.listaGrabaciones(grabacionesJson)
      File "/usr/local/opt/PyCollab/Utilidades.py", line 18, in listaGrabaciones
        recordinglist.append({"recording_id" : recordings['results'][x]['id'], "recording_name" : recordings['results'][x]['name'], "duration":recordings['results'][x]['duration'], "storageSize":recordings['results'][x]['storageSize'],"created": recordings['results'][x]['created']})
    KeyError: 'storageSize'
    
    

    Thanks for any assistance =D

    UUID option to grab recordings appear not working

    When grabbing a recording using the course name, it is able to grab the UUID and than grab the recordings...

    But when using the same UUID and same weeks, using the UUID file the recordings are not found.

    UTF-8 support for CollabRecordingsDownload.py

    Hello,

    When i load a Collaborate report using CollabRecordingsDownload.py, session name with french accent are broken ("é" become "é".
    Capture d’écran 2023-03-02 à 10 23 43

    I try everything in the code, but can't find how to fix that, can you help me ?

    Thanks

    Destination Folder

    This looks really great! Has anyone translated it to english? What folder do the videos and data download to? Can you run the script on PC and specify a destination folder on the local area network? I don't see anything on python web searching about PEP 157 package instalation. But I see something on PEP 517. Is that a typo?

    Error to running script Collab

    Python version testing: 3.9
    Python version testing: 3.8
    Python version testing: 3.7

    Steps To Reproduce

    1. pip install -r requerimientos.txt
    2. edit files learn_courses.txt
    3. python3 Collab.py -f learn_courses.txt

    The current behavior

    Course Recordings from 2020-09-22T02:30:42Z
    [auth:setToken()] ERROR: <Response [400]>
    [auth:setToken()] ERROR: <Response [400]>
    Traceback (most recent call last):
      File "/usr/src/app/controladores/AuthControlador.py", line 59, in getToken
        token = self.cache['token']
    TypeError: 'NoneType' object is not subscriptable
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "Collab.py", line 21, in <module>
        course_uuids.append(webService.getUUID(curso))
      File "/usr/src/app/webService.py", line 77, in getUUID
        token = self.getToken()
      File "/usr/src/app/webService.py", line 36, in getToken
        return(self.sesion.getToken())
      File "/usr/src/app/controladores/AuthControlador.py", line 69, in getToken
        return self.cache['token']
    TypeError: 'NoneType' object is not subscriptable
    

    Connection issue

    Hi there,

    I´m trying to download the recordings, but I was experiencing a lot of connection issues with us.bbcollab.com. I already tried to download from two diferente networks, but the issue persists. Could someone give me a good insight to try to resolve this issue?
    The script downloads one or two recordings and then the connection error happen.

    The error:

    Traceback (most recent call last):
    File "C:\Python37\lib\site-packages\urllib3\connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
    File "C:\Python37\lib\site-packages\urllib3\util\connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
    File "C:\Python37\lib\socket.py", line 752, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
    socket.gaierror: [Errno 11001] getaddrinfo failed

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File "C:\Python37\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
    chunked=chunked,
    File "C:\Python37\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
    File "C:\Python37\lib\site-packages\urllib3\connectionpool.py", line 976, in _validate_conn
    conn.connect()
    File "C:\Python37\lib\site-packages\urllib3\connection.py", line 308, in connect
    conn = self._new_conn()
    File "C:\Python37\lib\site-packages\urllib3\connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
    urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x000002CA0ED07A08>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File "C:\Python37\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
    File "C:\Python37\lib\site-packages\urllib3\connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
    File "C:\Python37\lib\site-packages\urllib3\util\retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
    urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='us.bbcollab.com', port=443): Max retries exceeded with url: /collab/api/csa/token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002CA0ED07A08>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File "Collab3.py", line 49, in
    ut.downloadOneRecordingPUCPR(recording_id,cursoid,linha)
    File "C:\PyCollab2\Utilidades.py", line 215, in downloadOneRecordingPUCPR
    recording_data = webService.get_recording_data(recording_id)
    File "C:\PyCollab2\webService.py", line 95, in get_recording_data
    sessions = SesionControlador.SesionControlador(self.DOMAIN_C,self.getJot(),self.CERT)
    File "C:\PyCollab2\webService.py", line 40, in getJot
    self.jsesion.setJot()
    File "C:\PyCollab2\controladores\JotControlador.py", line 64, in setJot
    r = requests.post(endpoint, data=self.payload, auth=(self.key, self.secret), verify=self.cert)
    File "C:\Python37\lib\site-packages\requests\api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
    File "C:\Python37\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
    File "C:\Python37\lib\site-packages\requests\sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
    File "C:\Python37\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
    File "C:\Python37\lib\site-packages\requests\adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
    requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.bbcollab.com', port=443): Max retries exceeded with url: /collab/api/csa/token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002CA0ED07A08>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

    Cesar Ferenc

    Error al descargar grabaciones

    Buen día

    Al intentar descargar las grabaciones de Collaborate con ayuda del reporte de grabaciones me arroja el siguiente error.

    PS D:\PyCollab> python CollabRecordingsDownload.py -f RecordingReport.csv

    Traceback (most recent call last):
    File "CollabRecordingsDownload.py", line 15, in
    recordings_ids = ut.readCollabReport(param[0])
    File "D:\PyCollab\Utilidades.py", line 97, in readCollabReport
    recording = recording_id(register['RecordingLink'])
    KeyError: 'RecordingLink'

    Me pueden ayudar por favor
    Gracias

    Recommend Projects

    • React photo React

      A declarative, efficient, and flexible JavaScript library for building user interfaces.

    • Vue.js photo Vue.js

      🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

    • Typescript photo Typescript

      TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

    • TensorFlow photo TensorFlow

      An Open Source Machine Learning Framework for Everyone

    • Django photo Django

      The Web framework for perfectionists with deadlines.

    • D3 photo D3

      Bring data to life with SVG, Canvas and HTML. 📊📈🎉

    Recommend Topics

    • javascript

      JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

    • web

      Some thing interesting about web. New door for the world.

    • server

      A server is a program made to process requests and deliver data to clients.

    • Machine learning

      Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

    • Game

      Some thing interesting about game, make everyone happy.

    Recommend Org

    • Facebook photo Facebook

      We are working to build community through open source technology. NB: members must have two-factor auth.

    • Microsoft photo Microsoft

      Open source projects and samples from Microsoft.

    • Google photo Google

      Google ❤️ Open Source for everyone.

    • D3 photo D3

      Data-Driven Documents codes.