raffaem / cs-dlp Goto Github PK

So even when download-notebooks option is specified, only an html of the previous page is saved. Is this going to be added in the future?

Describe the bug
Jupyter notebooks not downloading and other assets not downloading. There are unsupported typenames:
-Unsupported typename "ungradedWidget" in lecture "reading-getting-started-with-the-model-asset-exchange-and-the-data-asset"
-Unsupported typename "ungradedLti" in lecture "hands-on-lab-getting-started-with-jupyter-notebooks"
-Unsupported typename "staffGraded" in lecture "final-exam

To Reproduce
cs-dlp -ca YOURCAUTH open-source-tools-for-data-science (--download-notebooks included in conf file)

Expected behavior
Jupyter notebooks and other assets download.

ImportError: cannot import name 'HTMLParseError' from 'html.parser':

Last login: Mon Aug 14 19:43:49 on ttys000 User ~ % git clone https://github.com/raffaem/cs-dlp Cloning into 'cs-dlp'... remote: Enumerating objects: 4684, done. remote: Counting objects: 100% (233/233), done. remote: Compressing objects: 100% (115/115), done. remote: Total 4684 (delta 139), reused 197 (delta 117), pack-reused 4451 Receiving objects: 100% (4684/4684), 1.84 MiB | 1.08 MiB/s, done. Resolving deltas: 100% (2773/2773), done. user ~ % cd cs-dlp user cs-dlp % pip install --user . zsh: command not found: pip user cs-dlp % pip3 install --user . Processing /Users/user/cs-dlp Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: attrs>=18.1.0 in /Users/user/Library/Python/3.9/lib/python/site-packages (from cs-dlp==0.12.0b0) (18.1.0) Requirement already satisfied: beautifulsoup4>=4.1.3 in /Users/user/Library/Python/3.9/lib/python/site-packages (from cs-dlp==0.12.0b0) (4.1.3) Requirement already satisfied: browser-cookie3>=0.19.0 in /Users/user/Library/Python/3.9/lib/python/site-packages (from cs-dlp==0.12.0b0) (0.19.1) Requirement already satisfied: configargparse>=0.12.0 in /Users/user/Library/Python/3.9/lib/python/site-packages (from cs-dlp==0.12.0b0) (1.5.3) Requirement already satisfied: pyasn1>=0.1.7 in /Users/user/Library/Python/3.9/lib/python/site-packages (from cs-dlp==0.12.0b0) (0.5.0) Requirement already satisfied: requests>=2.20 in /Users/user/Library/Python/3.9/lib/python/site-packages (from cs-dlp==0.12.0b0) (2.31.0) Requirement already satisfied: urllib3>=1.23 in /Users/user/Library/Python/3.9/lib/python/site-packages (from cs-dlp==0.12.0b0) (2.0.3) Requirement already satisfied: lz4 in /Users/user/Library/Python/3.9/lib/python/site-packages (from browser-cookie3>=0.19.0->cs-dlp==0.12.0b0) (4.3.2) Requirement already satisfied: pycryptodomex in /Users/user/Library/Python/3.9/lib/python/site-packages (from browser-cookie3>=0.19.0->cs-dlp==0.12.0b0) (3.18.0) Requirement already satisfied: charset-normalizer<4,>=2 in /Users/user/Library/Python/3.9/lib/python/site-packages (from requests>=2.20->cs-dlp==0.12.0b0) (3.1.0) Requirement already satisfied: idna<4,>=2.5 in /Users/user/Library/Python/3.9/lib/python/site-packages (from requests>=2.20->cs-dlp==0.12.0b0) (3.4) Requirement already satisfied: certifi>=2017.4.17 in /Users/user/Library/Python/3.9/lib/python/site-packages (from requests>=2.20->cs-dlp==0.12.0b0) (2023.5.7) Building wheels for collected packages: cs-dlp Building wheel for cs-dlp (pyproject.toml) ... done Created wheel for cs-dlp: filename=cs_dlp-0.12.0b0-py2.py3-none-any.whl size=62981 sha256=0c107655acf909f6246af496ae547f858ae71a15d286e0ced48561e5edf94db4 Stored in directory: /Users/user/Library/Caches/pip/wheels/59/08/6d/49be50f7d78124cb8f845e3f02fe03d889293becee316d0ed2 Successfully built cs-dlp Installing collected packages: cs-dlp Attempting uninstall: cs-dlp Found existing installation: cs-dlp 0.12.0b0 Uninstalling cs-dlp-0.12.0b0: Successfully uninstalled cs-dlp-0.12.0b0 WARNING: The script cs-dlp is installed in '/Users/user/Library/Python/3.9/bin' which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. Successfully installed cs-dlp-0.12.0b0 user cs-dlp % /Users/user/Library/Python/3.9/bin/cs-dlp --cauth-auto safari modelthinking-004 Traceback (most recent call last): File "/Users/user/Library/Python/3.9/bin/cs-dlp", line 5, in <module> from cs_dlp.coursera_dl import main File "/Users/user/Library/Python/3.9/lib/python/site-packages/cs_dlp/coursera_dl.py", line 57, in <module> import bs4 File "/Users/user/Library/Python/3.9/lib/python/site-packages/bs4/__init__.py", line 29, in <module> from .builder import builder_registry File "/Users/user/Library/Python/3.9/lib/python/site-packages/bs4/builder/__init__.py", line 303, in <module> from . import _htmlparser File "/Users/user/Library/Python/3.9/lib/python/site-packages/bs4/builder/_htmlparser.py", line 7, in <module> from html.parser import ( ImportError: cannot import name 'HTMLParseError' from 'html.parser' (/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/html/parser.py) user cs-dlp %

Desktop:

• OS: macOS 13.4.1
• Browser safari
• Version 16.5.2
• Python Version - 3.9.6

Describe the bug
I followed installation instruction but the command cs-dlp is not found.

To Reproduce

1. activate conda environment
2. follow installation instruction and run:

git clone https://github.com/raffaem/cs-dlp
cd cs-dlp
pip install --user .

3. run cs-dlp:

zsh: command not found: cs-dlp

Expected behavior
It should execute. The package is indeed installed and it does run when I run python main.py. I don't know if this is an error in the code, documentation or my virtual environment setup.

I checked the bin folder of my conda environment (/opt/homebrew/Caskroom/miniconda/base/envs/my_env_name/bin) and indeed cs-dlp is not in the folder.

Desktop (please complete the following information):

• OS: macOS 14.0 (23A344)
• Python 3.9.5
• conda 4.9.2

The program is hanging at the last line "Not Found 404."
Is there any way to exit at this command, or do I need to restart PowerShell to continue running?
Thank you

I cannot seem to get this to work, with the master branch as well as this one. I keep getting this error, but I've triple checked the login info and it's all good. I have tried using -n for the .netrc file as well as doing the -u -p option. What is going on? ChatGPT was helpful up until this point, we cannot seem to find a work around and I've been at this for HOURS.

C:\Users\CatalystNovus>coursera-dl applying-project-management -n
coursera_dl version 0.11.5
Traceback (most recent call last):
File "c:\users\catalystnovus.pyenv\pyenv-win\versions\3.8.10\lib\site-packages\coursera\cookies.py", line 148, in login
r.raise_for_status()
File "c:\users\catalystnovus.pyenv\pyenv-win\versions\3.8.10\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.coursera.org/api/login/v3

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\users\catalystnovus.pyenv\pyenv-win\versions\3.8.10\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\catalystnovus.pyenv\pyenv-win\versions\3.8.10\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\CatalystNovus.pyenv\pyenv-win\versions\3.8.10\Scripts\coursera-dl.exe_main.py", line 7, in
File "c:\users\catalystnovus.pyenv\pyenv-win\versions\3.8.10\lib\site-packages\coursera\coursera_dl.py", line 239, in main
login(session, args.username, args.password)
File "c:\users\catalystnovus.pyenv\pyenv-win\versions\3.8.10\lib\site-packages\coursera\cookies.py", line 155, in login
raise AuthenticationFailed('Cannot login on coursera.org: %s' % e)
coursera.cookies.AuthenticationFailed: Cannot login on coursera.org: 400 Client Error: Bad Request for url: https://api.coursera.org/api/login/v3

Desktop (please complete the following information):

• OS: Windows (Dual Boot with QubesOS on a separate drive)
• Browser Firefox
• Dell OptiPlex 7020 + AMD RX340 + 240GB NVMe m.2 through PCIe adaptor

Hi,

I tried to run with coursera-dl.conf file that contained:

--username [email protected]
--password passw0rd
--subtitle-language en
--video-resolution 720p
--download-quizzes True
--download-notebooks True
--cauth COOKIEfromBrowser

But when I run:

cs-dlp --resume --path=./courses coursename

It dies claiming the --username and --password arguments are invalid:

cs-dlp: error: unrecognized arguments: [email protected] --password=passw0rd

Or did I do something wrong?

Download is no longer working, the course name is not being recognized:

xxxxxxx@MacBook-Air bin % /Users/xxxxxxx/Library/Python/3.9/bin/cs-dlp cs-dlp --cauth-auto safari --path=/Users/xxxxxxx/Downloads/Coursera/Classes/  --subtitle-language en gmba8033              
/Users/yuvalbeck/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py:34: NotOpenSSLWarning: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(
usage: cs-dlp [-h] [--jobs JOBS] [--download-delay DOWNLOAD_DELAY] [-b] [--path PATH] [-sl SUBTITLE_LANGUAGE] [--specialization] [--only-syllabus] [--download-quizzes] [--download-notebooks] [--about]
              [-f FILE_FORMATS] [--ignore-formats IGNORE_FORMATS] [-sf SECTION_FILTER] [-lf LECTURE_FILTER] [-rf RESOURCE_FILTER] [--video-resolution VIDEO_RESOLUTION] [--disable-url-skipping]
              [--wget [WGET]] [--curl [CURL]] [--aria2 [ARIA2]] [--axel [AXEL]] [--downloader-arguments DOWNLOADER_ARGUMENTS] [--list-courses] [--resume] [-o] [--verbose-dirs] [--quiet] [-r]
              [--combined-section-lectures-nums] [--unrestricted-filenames] [-ca COOKIES_CAUTH] [-caa BROWSER] [-c COOKIES_FILE] [--clear-cache] [--hook HOOKS] [-pl] [--mathjax-cdn MATHJAX_CDN_URL]
              [--skip-download] [--debug] [--cache-syllabus] [--version] [-l LOCAL_PAGE]
              [class_names ...]
cs-dlp: error: unrecognized arguments: gmba8033

Although found in the courses list:

xxxxxxx@MacBook-Air bin % /Users/xxxxxx/Library/Python/3.9/bin/cs-dlp cs-dlp --cauth-auto safari --list-courses
/Users/xxxxxxx/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py:34: NotOpenSSLWarning: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(
coursera_dl version 0.12.0b0
Listing enrolled courses
Found 2 courses
gmba8034
gmba8033

Python: 3.9.6
OS: macOS 14.2.1

Beleive related to #19
@raffaem can you please help in this matter?

Thank you!

When I run the command cs-dlp --cauth-auto firefox --subtitle-language en --download-quizzes algebra-i, the course videos and subtitles download just fine, but when the program tries downloading the exams/quizzes, it prints Error 403 Client Error. Am I missing something? My laptop is running Debian sid.

$ cs-dlp --cauth-auto firefox --subtitle-language en --download-quizzes algebra-i
coursera_dl version 0.12.0b0
Downloading class: algebra-i (1 / 1)
Parsing syllabus of on-demand course (id=TWNon2r-Eeumww4A4k-nvw). This may take some time, please be patient ...
Processing module  module-1-the-structure-of-numbers
Processing section     lesson-1-real-numbers
Processing lecture         real-numbers (lecture)
Processing lecture         notes-common-sets-of-numbers (supplement)
Processing lecture         sample-problems-real-numbers (supplement)
Processing lecture         structure-of-numbers (exam)
Error 403 Client Error: Forbidden for url: https://api.coursera.org/api/onDemandExamSessions.v1 getting page https://api.coursera.org/api/onDemandExamSessions.v1
The server replied: {"errorCode":"Not Authorized","message":null,"details":null}
Could not download exam eQuAo: 403 Client Error: Forbidden for url: https://api.coursera.org/api/onDemandExamSessions.v1
Processing section     lesson-2-properties-of-real-numbers
Processing lecture         properties-of-real-numbers (lecture)
Processing lecture         notes-operations (supplement)
Processing lecture         notes-working-with-fractions (supplement)
Processing lecture         sample-problems-properties-of-real-numbers (supplement)
Processing lecture         properties-of-real-numbers (exam)
Error 403 Client Error: Forbidden for url: https://api.coursera.org/api/onDemandExamSessions.v1 getting page https://api.coursera.org/api/onDemandExamSessions.v1
The server replied: {"errorCode":"Not Authorized","message":null,"details":null}
Could not download exam pQm5a: 403 Client Error: Forbidden for url: https://api.coursera.org/api/onDemandExamSessions.v1
Processing module  module-2-linear-equations
Processing section     lesson-1-linear-equations-in-one-variable
...
$

Trying to download foundations-of-digital-marketing-and-e-commerce but receiving the error:

cs-dlp: error: unrecognized arguments: foundations-of-digital-marketing-and-e-commerce

command used that produced error:

cs-dlp --cauth-auto chrome foundations-of-digital-marketing-and-e-commerce

after flowing the install instruction i get this msg

py 13.12 also tried with 11, 9

note working for me

C:\sat\cs-dlp>cs-dlp --cauth-auto chrome XXXX
'cs_dlp' is not recognized as an internal or external command,
operable program or batch file.

Please help me on this, I need a course to download which is rare

if it works for anyone kindly post Python version git version ...and every step please

According to the documentation, when executing the command "cs-dlp --cauth-auto chrome ........," the following error is reported:

Traceback (most recent call last):
File "E:\Soft\anaconda3\envs\Coursera\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "E:\Soft\anaconda3\envs\Coursera\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\HouseLiang\AppData\Roaming\Python\Python39\Scripts\cs-dlp.exe\__main__.py", line 4, in
ModuleNotFoundError: No module named 'main'

ModuleNotFoundError: No module named 'cs_dlp.coursera_dl'
The error you're encountering when running "cs-dlp --cauth-auto chrome ......" - how can I resolve it?

I am having trouble downloading a particular course but it works perfectly fine with other courses.
The course I had issue downloading is data-analysis-and-visualization-with-power-bi.
Can anyone please shine me some lights on how to fix it?

2023-12-21 15:52:35 Traceback (most recent call last):
2023-12-21 15:52:35 File "/usr/local/bin/cs-dlp", line 8, in
2023-12-21 15:52:35 sys.exit(main())
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/coursera_dl.py", line 283, in main
2023-12-21 15:52:35 error_occurred, completed = download_class(
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/coursera_dl.py", line 253, in download_class
2023-12-21 15:52:35 return download_on_demand_class(session, args, class_name)
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/coursera_dl.py", line 173, in download_on_demand_class
2023-12-21 15:52:35 error_occurred, modules = extractor.get_modules(
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/extractors.py", line 53, in get_modules
2023-12-21 15:52:35 error_occurred, modules = self._parse_on_demand_syllabus(
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/extractors.py", line 155, in _parse_on_demand_syllabus
2023-12-21 15:52:35 links = course.extract_links_from_lecture(
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/api.py", line 880, in extract_links_from_lecture
2023-12-21 15:52:35 links = self._extract_videos_and_subtitles_from_lecture(
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/api.py", line 1044, in _extract_videos_and_subtitles_from_lecture
2023-12-21 15:52:35 videos = VideosV1.from_json(dom)
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/api.py", line 523, in from_json
2023-12-21 15:52:35 videos = [VideoV1(resolution, links['mp4VideoUrl'])
2023-12-21 15:52:35 File "/usr/local/lib/python3.10/dist-packages/cs_dlp/api.py", line 523, in
2023-12-21 15:52:35 videos = [VideoV1(resolution, links['mp4VideoUrl'])
2023-12-21 15:52:35 KeyError: 'mp4VideoUrl'