shihabmridha / educative.io-downloader Goto Github PK

After downloading a course with "html, css and JS" code snapshot, I can't viewed

Hi,
I Can't open or view the mhtml saved files after downloading it.

I get a blank page with this message " We use cookies as well as similar tools and technologies to improve your experience and to understand how our site is being used. Review our PRIVACY POLICY to learn more about what cookies we use and how to amend your cookie settings."

Pages has been downloaded successfully using this nice utility.
mhtml pages are not blank, seems to have all contents.
But while opening using browser, pages are blank. Tried with chrome, edge, safari, but same behaviour. Exporting to pdf also result in blank pdf.
Would you please let us know how to view the saved pages?

I am getting this error after Processing batch of lessons.

I am getting below error when I execute "npm install" command to install dependencies,

npm install

[email protected] install C:\amit_fegade\softwares\educative-io\educative.io-downloader-master\node_modules\puppeteer
node install.js

ERROR: Failed to download Chromium r706915! Set "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" env variable to skip download.
Error: self signed certificate in certificate chain
at TLSSocket.onConnectSecure (_tls_wrap.js:1473:34)
at TLSSocket.emit (events.js:311:20)
at TLSSocket._finishInit (_tls_wrap.js:916:8)
at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:686:12)
-- ASYNC --
at BrowserFetcher. (C:\amit_fegade\softwares\educative-io\educative.io-downloader-master\node_modules\puppeteer\lib\helper.js:111:15)
at Object. (C:\amit_fegade\softwares\educative-io\educative.io-downloader-master\node_modules\puppeteer\install.js:62:16)
at Module._compile (internal/modules/cjs/loader.js:1158:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10)
at Module.load (internal/modules/cjs/loader.js:1002:32)
at Function.Module._load (internal/modules/cjs/loader.js:901:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
at internal/main/run_main_module.js:18:47 {
code: 'SELF_SIGNED_CERT_IN_CHAIN'
}
npm WARN rollback Rolling back [email protected] failed (this is probably harmless): EPERM: operation not permitted, lstat 'C:\amit_fegade\softwares\educative-io\educative.io-downloader-master\node_modules\https-proxy-agent\node_modules'
npm WARN [email protected] No repository field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node install.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\amitf\AppData\Roaming\npm-cache_logs\2020-05-23T17_24_29_780Z-debug.log.

2020-06-15T00_41_54_861Z-debug.log

https://www.educative.io/courses/web-development-unraveling-html-css-js/RM4EKvny1MO

I believe recently educative has enhanced the captcha

In PDF Code is not trimmed just in PNG.

Idea is to have firstPageWithCode where user can define where the code is and then navigate to that page. Look for code block determine languages available. Store those in array. For Loop on that array and first. navigate to above page, click on the first language tab by name of language as there is inconsistency in tab index and then continue with downloading pages, and so on. Also need to match code url with course url in case user forget to change it. Although this will only work if single course is present in config file. Do you have any idea?

I am getting an error that says "Reason: Protocol error (Page.printToPDF): PrintToPDF is not implemented" Which results in nothing being downloaded. Btw I have loginCheck and headless set as false.

https://www.educative.io/courses/web-development-unraveling-html-css-js/39DpLBvDW7p

I downloaded type script but after that what i have to do, i am not getting that much from the usage section. Please help.

internal/modules/cjs/loader.js:968
throw err;
^

Error: Cannot find module 'C:\Users\dEnOmInAtOr\Downloads\educative.io-downloader-master\build\src\app.js'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:965:15)
at Function.Module._load (internal/modules/cjs/loader.js:841:27)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: node build/src/app.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\dEnOmInAtOr\AppData\Roaming\npm-cache_logs\2020-07-28T01_43_35_456Z-debug.log

C:\Users\Aashish\Desktop\educative.io\educative.io-downloader>npm run install
npm ERR! missing script: install

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\Aashish\AppData\Roaming\npm-cache_logs\2020-05-07T04_24_18_631Z-debug.log

Below is my Scripts tag in package.json
"scripts": {
"start": "node build/src/app.js",
"dev": "nodemon build/src/app.js",
"compile": "tsc -p .",
"test": "echo "Error: no test specified" && exit 1",
},

Can you please guide me step by step. I am a newbie...!

Hi,
I downloaded pdf, because mhtml files are not opening in chrome or firefox. I set true also in browser.ts file to check login. I am successfully logged-in in the browser. Pdf files are not downloaded properly and their size is coming 1KB.
mhtml are not good to read, without CSS they are not looking good. Could you please change it like earlier version where file are downloaded in html format. Please check that pdf file issue also.

getting this error. Please help.

Sometimes the downloader crash after downloading 2 or 3 courses. Happens on 2 different PC (The weaker one is more subject to this).

Crash log:
2020-08-01T12_16_19_228Z-debug.log

downloadAllCourses:true used to work, but I think it stopped working. It only downloads What would be the best way to troubleshoot?

Found 9 courses to download on this page.
Found 30 courses to download on this page.
Found 30 courses to download on this page.
.
.
.
Found a total of 15 courses to download.

Thank, bro! It is very awsome!
These are some error message when i started the program.

Hello, Is it possible to accommodate the change to download all the courses under a given track, For example if I provide the track URL https://www.educative.io/track/ace-java-coding-interview it should download all the 9 courses under it

Thanks for this useful tool.

As the title says, while downloading a course, the mhtml fails to show an image (possibly svg+xml), but the pdf works fine.

I ran in headless=false mode, the image load successfully. But, after downloading, it was blank in mhtml file.

This is an example page, https://www.educative.io/courses/advanced-kubernetes-techniques/3YzvvGX3RxA

Thanks.

Here's my configuration
{ "email": "", "password": "", "loginCheck": true, "saveAs": "pdf", "multiLanguage": true, "courseUrl": "https://www.educative.io/courses/grokking-the-coding-interview", "downloadAllCourses": false }

https://www.educative.io/courses/advanced-react-patterns-with-hooks/R8JlpBZZkLK

Another Type with all questions on one page -> https://www.educative.io/courses/practical-guide-to-kubernetes/Y54vVYvvLN9

Attaching the downloaded folder screenshot

Sometimes the script fails to download a few lessons get of a course. One of the major reason is HTTP TIMEOUT. So, it makes sense to download only those missing lessons instead of the whole course again.

Initial approach:
Before downloading a lesson look in the respective download directory to check if the file is already downloaded. If yes, then skip the lesson.

The changes I added broke the image functionality. Throwing error Protocol error (Page.captureScreenshot): Cannot take screenshot with 0 width. So we need to rethink the approach or I will suggest to stick to just HTML as code will be trimmed in both image and pdf.

This downloads the separate PDF for each chapter, is it possible to add support for a single combined pdf?

Is it possible to provide comma separated URLs for courseUrl and download all those at a time. something like
"courseUrl": "https://www.educative.io/courses/docker-for-developers,https://www.educative.io/courses/web-application-software-architecture-101,https://www.educative.io/courses/introduction-microservice-principles-concepts"

Failed to download https://www.educative.io/courses/learn-java-from-scratch/RMwDgBWXQjL
Reason: WebSocket is not open: readyState 3 (CLOSED)

❯ npm run compile

[email protected] compile /Users/arjun/lab/educative/educative.io-downloader
tsc -p .

error TS6047: Argument for '--target' option must be 'ES3', 'ES5', or 'ES6'.
error TS5023: Unknown compiler option 'lib'.
error TS5023: Unknown compiler option 'moduleResolution'.
error TS5023: Unknown compiler option 'noUnusedLocals'.
error TS5023: Unknown compiler option 'noUnusedParameters'.
error TS5023: Unknown compiler option 'forceConsistentCasingInFileNames'.
error TS5023: Unknown compiler option 'skipLibCheck'.
error TS5023: Unknown compiler option 'pretty'.
error TS5023: Unknown compiler option 'allowUnreachableCode'.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] compile: tsc -p .
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] compile script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/arjun/.npm/_logs/2020-05-01T21_30_28_161Z-debug.log

https://www.educative.io/courses/fixing-random-techniques-in-c-sharp/Y5RvjXO6GAK

(node:72254) UnhandledPromiseRejectionWarning: Error: Page crashed!
at Page._onTargetCrashed (/home/user/work/other/educative.io-downloader/node_modules/puppeteer/lib/Page.js:216:24)
at CDPSession. (/home/user/work/other/educative.io-downloader/node_modules/puppeteer/lib/Page.js:124:56)
at CDPSession.emit (events.js:315:20)
at CDPSession._onMessage (/home/user/work/other/educative.io-downloader/node_modules/puppeteer/lib/Connection.js:200:12)
at Connection._onMessage (/home/user/work/other/educative.io-downloader/node_modules/puppeteer/lib/Connection.js:112:17)
at WebSocket. (/home/user/work/other/educative.io-downloader/node_modules/puppeteer/lib/WebSocketTransport.js:44:24)
at WebSocket.onMessage (/home/user/work/other/educative.io-downloader/node_modules/ws/lib/event-target.js:120:16)
at WebSocket.emit (events.js:315:20)
at Receiver.receiverOnMessage (/home/user/work/other/educative.io-downloader/node_modules/ws/lib/websocket.js:789:20)
at Receiver.emit (events.js:315:20)
(node:72254) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 7)

Why can I get this error?

I am getting access denied error.

https://www.screencast.com/t/inlA7b5opsm
I have checked the username and the password is working fine on the browser.

Some courses have videos so, can you add those videos to be downloaded.

https://www.educative.io/courses/web-development-unraveling-html-css-js/7ALp2B5vL1G

Sir,could you be more detailed?i am new to typescript as well as git

You need a subscription to use this

Does it mean I need to have a paid subscription to leverage this? I have headless mode set to true and still I get blank pdf pages.

I tried to debug by settings the headless mode set to false, then I see an error printToPDF not implemented.

https://www.educative.io/courses/coderust-hacking-the-coding-interview/k5llE

Every MHTML document is opening up the index file which when you click on opens the website. Also, the pdf option downloads empty 1Kb pdf files. Am I missing something?

Also, if headless is set to false, it does not login. It seems to log in only when headless is set to true.

{ "errorText": "Access denied. Please contact [email protected]", "request_id": "5e6e09ac00ff0600ffa5a4c6439e0001737e6564756361746976652d6865736976650001763230303330322d686f746669782d3230303330392d3132313600010101" }

Also it shouldn't proceed downloading course if login failed

This is an error that appeared in some courses while downloading it as PDF is the headless = false
and I made it true for better pdf generation but it's an empty file in some courses that i still can access to them ...