Incomprehendable Error about twitter-scraper-selenium HOT 10 CLOSED

I don't think this issue is related to the library but is probably due to some external factor. I am unable to recreate this issue. What is your environment? I found related issues here

Just side note: I would suggest using scrape_topic_with_api() for faster speed and accurate data in case you're scraping topics

from twitter-scraper-selenium.

I tried scrape_topic_with_api, I got this -

2022-10-29 09:23:32,697 - twitter_scraper_selenium.element_finder - ERROR - Failed to find key!
NoneType: None
2022-10-29 09:23:32,794 - twitter_scraper_selenium.element_finder - WARNING - Error at find_graphql_link : 'NoneType' object has no attribute 'split'
2022-10-29 09:27:50,535 - WARNING - Failed to make request!
Traceback (most recent call last):
  File "/root/Bots/DiscordBots/TwitterTopics/MTC/src/util/scrape.py", line 6, in <module>
    scrape_topic_with_api(URL='https://twitter.com/i/topics/1468157909318045697', output_filename='tweets', tweets_count=10)
  File "/usr/local/lib/python3.10/dist-packages/twitter_scraper_selenium/topic_api.py", line 156, in scrape_topic_with_api
    data.update(content)
TypeError: 'NoneType' object is not iterable

from twitter-scraper-selenium.

The issue is probably that firefox itself is not instantiating on your system. What is your environment?

from twitter-scraper-selenium.

The issue is probably that firefox itself is not instantiating on your system. What is your environment?

As in? I'm on Ubuntu 20.04, with Python 3.10

from twitter-scraper-selenium.

Okay, I don't have much idea about this issue. People all over the internet end up with this for different reasons which can be found here. I've kept a wait before accessing DOM as well. Please consider passing the argument headless=False to scrape_topic(), it'll show you what is happening with your browser

from twitter-scraper-selenium.

2022-10-29 12:47:24,770 - WARNING - Error at scrap : Message: Process unexpectedly closed with status 1

Traceback (most recent call last):
  File "/root/Bots/DiscordBots/TwitterTopics/MTC/src/util/scrape.py", line 6, in <module>
    scrape_topic_with_api(URL='https://twitter.com/i/topics/1468157909318045697', output_filename='tweets', tweets_count=10, headless=False)
  File "/usr/local/lib/python3.10/dist-packages/twitter_scraper_selenium/topic_api.py", line 156, in scrape_topic_with_api
    data.update(content)
AttributeError: 'NoneType' object has no attribute 'update'

from twitter-scraper-selenium.

No, I meant to say you can watch the browser opening by yourself which can let you know what is happening in your system while scraping. You might have seen browser opening automatically all that, that's what I was talking about

from twitter-scraper-selenium.

No, I meant to say you can watch the browser opening by yourself which can let you know what is happening in your system while scraping. You might have seen browser opening automatically all that, that's what I was talking about

I tested it, it works on my laptop for some reason, but not the vps

from twitter-scraper-selenium.

I can't say why it isn't running on VPS 🤔

from twitter-scraper-selenium.

nvm, turns out a reboot fixed it 😂 thank you for the help

from twitter-scraper-selenium.