nitdgpos / medium_grabber Goto Github PK

Geckodriver can be used in headless mode to improve user experience.

We already have a basic skeleton for Medium_Grabber GUI. Many things can still be done.
I am opening this for potential future contributors.

I have an Idea. In addition with output.txt and output.pdf files , Can we provide these files directly to the Email Id of the registered user.

When I tried with an account that has Two Step Verification in their account settings, it did not login. As soon as the Two Step Verification was turned off, the program runs well and good. I have given the logs below. Can any functionality be added to bypass this problem?

Choose the browser you want to use:(Enter the appropriate number)

                1---> firefox

                2---> chrome

                0--->exit
2
1208/203550.580:ERROR:gpu_process_transport_factory.cc(967)] Lost UI shared context.

DevTools listening on ws://127.0.0.1:51752/devtools/browser/b13cb79c-627f-4ddf-86d2-927692096027
[1208/203600.810:INFO:CONSOLE(0)] "Refused to connect to 'https://srv-2018-12-08-15.pixel.parsely.com/event/?rand=1544281560799&plid=80778101&idsite=medium.com&url=https%3A%2F%2Fmedium.com%2F&urlref=&screen=800x600%7C800x600%7C24&data=%7B%7D&sid=1&surl=https%3A%2F%2Fmedium.com%2F&sref=&sts=1544281556599&slts=0&date=Sat+Dec+08+2018+20%3A36%3A00+GMT%2B0530+(India+Standard+Time)&action=heartbeat&inc=4&tt=4101&u=pid%3Dd1df50a78549de27c3bb7da80df99acf' because it violates the following Content Security Policy directive: "connect-src https://localhost https://*.instapaper.com https://*.stripe.com https://glyph.medium.com https://*.paypal.com https://getpocket.com https://medium.com:443 https://*.medium.com:443 https://*.medium.com https://medium.com https://*.medium.com https://*.algolia.net https://cdn-static-1.medium.com https://dnqgz544uhbo8.cloudfront.net https://cdn-videos-1.medium.com https://cdn-audio-1.medium.com https://*.lightstep.com https://*.branch.io https://app.zencoder.com 'self'".
", source: https://medium.com/ (0)
 Logging in to Medium by using Google
Traceback (most recent call last):
  File "Grab_that_article_on_Medium.py", line 66, in <module>
    user = driver.find_element_by_xpath('//*[@id="password"]/div[1]/div/div[1]/input')
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
    'value': value})['value']
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="password"]/div[1]/div/div[1]/input"}
  (Session info: headless chrome=71.0.3578.80)
  (Driver info: chromedriver=2.44.609538 (b655c5a60b0b544917107a59d4153d4bf78e1b90),platform=Windows NT 10.0.17134 x86_64)

There is an error when we try to run Medium_Grabber from the main script.
Open for fruitful contributors.

This would come as a separate option to download all (or preferably 10-20) articles in PDF form with the name of file as the title.
The first line would contain the URL of the article (just in case the user wishes to see in his browser). The article would begin from the next line.
This can be done as a separate script at first and then later integrated with the GUI.

The links in the text file need copy-pasting in the browser for viewing the articles. The possible outcome should be to add hyperlinks in the text file.

Please suggest some links to setup selenium and Firefox Webdriver. I installed selenium using pip install selenium but how do I setup Firefox Webdriver?

A pdf file with all the added hyperlinks instead of the text file.

Currently this script does't support searching medium for articles. We can add this search option in the option list, and search medium for the user input.

README still instructs "Put your password in pass.txt and your Email/Phone Number in the user.txt file" which is unnecessary.

We could try to assess what browsers the system has installed and accordingly run the script for chrome or Firefox. In case if both Firefox and chrome are installed then the default browser will be choose. If neither of them are the default browser then Firefox will be used.

A pop up like the one below generally makes user to skip the story and read some other article rather than taking up the paid membership plan.

This issue can be resolved by making different kinds of lists of articles with different kinds of membership plans. This means that we also need to know whether the user is a paid member or free member as well. How can this issue be resolved?

Using gif is more appreciated.

At the current stage, the login is very unsecure. Anyone and everyone can open the password.txt file and see what is the password. I think encrypting the password.txt would provide a sense of security to the user?
I can work on this if you like.

I have an Idea. In addition with output.txt and output.pdf file on machine , Can we provide these files directly to the Email Id of the registered user.