minds-ai / zoom-drive-connector Goto Github PK

If there are multiple recordings per day it would be helpful to show some sort of human-readable timestamp in the Slack message. It would prevent confusion from something like this:

Starting time of the recording makes the most sense to me, but I'm open to suggestions @jbedorf.

After my discussion with Tijmen last week (9/5), I would like to nominate this project for potential open sourcing sometime in the future (pending legal approval and further discussion). Regardless of that potential project, I think it would be beneficial to fix a couple of things in this repository.

Initial Task List:

• Remove all reference to internal-only links, folder IDs, channel names, Zoom rooms, etc. These shouldn't be store in this repository regardless if it is public or not.
• Update inline code comments or docstrings that are unclear or are missing. For example this file is missing a docstring.
• Fix remaining linter and codestyle issues.
• Add Conda environment file for faster, plug-n-play development.
• Version out requirements file to prevent future API breakage.
• Alter Dockerfile such that configuration can be altered without having to rebuild the container.
• Make Docker built process easily versionable (i.e. don't tag every container with latest because that orphans older versions).
• Replace return tuples with dictionaries. This should make returning values from functions more obvious and less prone to errors.
• Replace print statements with proper logger.

Right now we only support a single Slack channel for reporting while it would be more flexible to do this reporting in a channel that is dedicated to a meeting.
This means we have to modify the configuration file to support this.

When trying to upload large files (~2GB) the script exits with the overflow error. More info can be found here:

https://programmer.group/overflow-error-string-longer-than-2147483647-bytes-problem.html

See 'Formatting dates' on https://api.slack.com/docs/message-formatting

There are a couple of type-hint errors that should be fixed.

Working in https://github.com/minds-ai/zoom-drive-connector/tree/type_fixing to fix those.

I always get a 401 when validating Zoom API calls

2019-03-11 22:19:07,547 main:INFO Application starting up.
2019-03-11 22:19:07,648 drive_api:INFO Drive connection established.
2019-03-11 22:19:08,098 zoom_api:ERROR HTTP_STATUS: 401-Unauthorized, Not authenticated.

It seems to be related to the application not using OAuth, Zoom only allows me to create an OAuth application.

Can you help?
Thanks

Hi everyone,

I have zero coding background, and I only try to learn from step-by-step tutorials such as this one.

I also will soon have 400 zoom cloud recordings to download.

I got through a lot of pain, to get to this step, but got stuck with this error, dont know what to do anymore

This is from Docker's logs, tried using CMD, now doing things in Ubuntu because I realized your code lines start with $, am super confused, please help.

`Traceback (most recent call last):

File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main

"main", mod_spec)

File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code

exec(code, run_globals)

File "/usr/local/lib/python3.7/site-packages/zoom_drive_connector/main.py", line 136, in

main()

File "/usr/local/lib/python3.7/site-packages/zoom_drive_connector/main.py", line 111, in main

app_config = config.ConfigInterface(os.getenv('CONFIG', '/conf/config.yaml'))

File "/usr/local/lib/python3.7/site-packages/zoom_drive_connector/configuration/configuration_interfaces.py", line 115, in init

self.__interface_factory()

File "/usr/local/lib/python3.7/site-packages/zoom_drive_connector/configuration/configuration_interfaces.py", line 151, in __interface_factory

raise RuntimeError(f'Configuration for section {value.class} failed validation step.')

RuntimeError: Configuration for section <class 'zoom_drive_connector.configuration.configuration_interfaces.DriveConfig'> failed validation step.`

Zoom does this weird thing where they use redirects to hide the direct download links to cloud recordings. Agent-less programs don't work because they hide their recordings behind a login screen or are detecting the user agent and preventing downloads from occurring.

• Look into falsifying user-agent string
• Figure out how to send proper login credentials to allow downloads.

When connections fail it creates exceptions that propagate until main and then kill the running program.
Should put main loop in a try-except structure to handle this more gracefully.

Also figure out why upload is called when the first exception came from the download/zoom part.

HTTPSConnectionPool(host='api.zoom.us', port=443): Max retries exceeded with url: /v2/meetings/<ID>/recordings?access_token=<token> (Caused by NewConnectionError('<urllib3.connectio
n.VerifiedHTTPSConnection object at 0x7f5b7d9083c8>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 1187, in _conn_request
    conn.connect()
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 999, in connect
    address_info = socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM)
  File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
  File "main.py", line 113, in <module>
    schedule.run_pending()
  File "/usr/local/lib/python3.5/dist-packages/schedule/__init__.py", line 493, in run_pending
    default_scheduler.run_pending()
  File "/usr/local/lib/python3.5/dist-packages/schedule/__init__.py", line 78, in run_pending
    self._run_job(job)
  File "/usr/local/lib/python3.5/dist-packages/schedule/__init__.py", line 131, in _run_job
    ret = job.run()
  File "/usr/local/lib/python3.5/dist-packages/schedule/__init__.py", line 411, in run
    ret = self.job_func()
  File "main.py", line 57, in all_steps
    upload(download_files, config.drive_conf)
  File "main.py", line 78, in upload
    drive = DriveAPI("", drive_conf.key, drive_conf.secret, drive_conf.folder_id)
  File "/app/drive/drive_api.py", line 33, in __init__
    self.setup()
  File "/app/drive/drive_api.py", line 46, in setup
    self._service = apiclient.discovery.build('drive', 'v3', http=creds.authorize(httplib2.Http()))
  File "/usr/local/lib/python3.5/dist-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/googleapiclient/discovery.py", line 222, in build
    requested_url, discovery_http, cache_discovery, cache)
  File "/usr/local/lib/python3.5/dist-packages/googleapiclient/discovery.py", line 269, in _retrieve_discovery_doc
    resp, content = http.request(actual_url)
  File "/usr/local/lib/python3.5/dist-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python3.5/dist-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 1514, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 1264, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 1194, in _conn_request
    raise ServerNotFoundError("Unable to find the server at %s" % conn.host)
httplib2.ServerNotFoundError: Unable to find the server at www.googleapis.com

@jbedorf and @MrFlynn. The overall code looks good. There are few minor suggested changes by me. @jvanheugten please add your reviews too. Please feel to modify or ignore. Though, I haven't tested the workflow by setting up the repo.

Typing

• https://github.com/minds-ai/zoom-drive-connector/blob/master/main.py#L32. The function returns a list, should be annotated as List[Dict[str, str]]. If you find difficult to annotate a dict, consider using namedtuple.
• Union[int, None] can also be written as Optional[int]. https://github.com/minds-ai/zoom-drive-connector/blob/master/zoom/zoom_api_exception.py#L24
• In https://github.com/minds-ai/zoom-drive-connector/blob/master/zoom/zoom_api.py#L131, return type should be Dict[] and not dict. date and filename in the dictionary can be None on failure, and helps to maintain the same structure.

Requests

• In https://github.com/minds-ai/zoom-drive-connector/blob/master/zoom/zoom_api.py#L68,

   if res.status_code == 401:
      # Handle unauthenticated requests.
      raise ZoomAPIException(401, 'Unauthorized', res.request, 'Not authenticated.')
    elif res.status_code == 409:
      # Handle error where content may have been removed already.
      raise ZoomAPIException(409, 'Resource Conflict', res.request, 'File deleted already.')

the above code can be written as

status_code = res.status_code 
if 400 <= status_code <= 499:
   message = {401: 'Not authenticated.', 409: 'File deleted already.')
   raise ZoomAPIException(status_code, res.reason, res.request, message.get(status_code, '')

Response API: http://docs.python-requests.org/en/master/api/#requests.Response

• https://github.com/minds-ai/zoom-drive-connector/blob/master/zoom/zoom_api.py#L98, the else block will run even if the status code is 500 which means their server failed. This block should be executed if the status_code is 2XX, you can check if res.ok or status code, so ideal flow could be something like

if 200 <= status_code <= 299:
# All well
elif 300 <= status_code <= 399:
# Server is redirecting, we can't do much
elif 400 <= status_code <= 499:
# Something we did wrong,
elif 500 <= status_code <= 599:
# Zoom Server Error
else:
# Never seen this but possible

General

• 'HTTP_STATUS: {c}-{n}, {m}'.format(c=self.status_code, n=self.name, m=self.message) rather than format string you can consider using f-string like f'HTTP_STATUS: {self.status_code}-{self.name}, {self.message}'.
• As a general practice __repr__ prints object creation method like Pipeline(steps=[(a, A())]). https://docs.python.org/3/library/functions.html#repr. https://github.com/minds-ai/zoom-drive-connector/blob/master/zoom/zoom_api_exception.py#L47
• Doc string is slightly different from our code base style. Our code base follows

def download_recording(self, url: str) -> str:
    """Downloads video file from Zoom to local folder.
    Args:
      url: Download URL for meeting recording.

    Returns: Path to the recording
    """

this repo follows

def download_recording(self, url: str) -> str:
    """Downloads video file from Zoom to local folder.
    :param url: Download URL for meeting recording.
    :return: Path to the recording
    """

• The URLS for various services are declared at multiple places. It can be put under one file with multiple classes like

class ZoomURLS(Enum):
   recordings: 'https://api.zoom.us/v2/meetings/{id}/recordings'
   delete_recordings: 'https://api.zoom.us/v2/meetings/{id}/recordings/{rid}'
   ...

The advantage of the method is, when zoom changes it's API, we can find all the URLS in one place and modify.

Typos

• entries is misspelled as entrees in README.

Testing

We can write unit tests for the code, we have written like ConfigParsers and other services specific code. One can mock the request, and response using Python Mock library, Responses. Once you have a list of scenarios like Zoom recording file is missing, record files are available then write tests for specific code parts.

This is a list of future improvements that can be made to this codebase. These are low priority changes and can be made sometime in the future.

• Modify existing boilerplate code to use configuration container classes. This includes migrating
  class init functions to pull configuration data from object and not from passed in data.
• Clean up unused code (especially code that has been commented out for debugging purposes).
• Remove deprecated MAINTAINER from Dockerfile. Replace with LABEL for future compatibility purposes.
• Add hashbang to shell scripts to match script best practices.
• Come up with logo for application instead of the generic on Slack provides when it posts updates to Slack.

We need to choose a license for this repository. The two most popular choices are Apache 2 and MIT. We have found that the dependencies we pull in have a mix of MIT, Apache 2, MPL, and LGPL which (AFAIK) are compatible with either MIT or Apache 2.

Hi,

I've used This Project for a month or so, doing my own upgrades on the project but I wanted to transfer the idea into a cloud based solution for a faster proccess. This allowed me to just run the script with JWT token and download all fies to local with proper categorization.

Does this only work with individually selected files or does it go through all the sub-accounts under the specified "username" in the config file?

PS: Tried Appscript getBlob but it only downloads files 50mb portion and i lack the knowledge to use resumeupload stuff.

Thanks,