Comments (9)
tzuhsial
commented on July 2, 2024
Can you give me more information & environment on your query?
(E.g. the command you used to crawl followers)
Thanks!
from instagramcrawler.
Thicool
commented on July 2, 2024
Hey Antonie,
thanks for your reply.
Here are the commands that i use and the response i get.
I have no problem scraping pictures. After executing the code, firefox
starts and i enter my login (in case of followers), which then opens the
instagram-account page and open the follower-list. However, script stops at
that. I hope you can help me!
Here is the command line in and output:
…________________________________________________________________________________________________________________
C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>python
instagramcrawler.py -q instagram -t followers -n 30
dir_prefix: ./data/, query: instagram, crawl_type: followers, number: 30,
caption: False
You will need to login to crawl followers
Scraping followers...
Traceback (most recent call last):
File "instagramcrawler.py", line 297, in <module>
main()
File "instagramcrawler.py", line 293, in main
caption=args.caption)
File "instagramcrawler.py", line 96, in crawl
self.scrape_followers_or_following(crawl_type, query, number)
File "instagramcrawler.py", line 214, in scrape_followers_or_following
title = self._driver.find_element_by_xpath(FOLLOW_PATH)
File
"C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 313, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File
"C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 791, in find_element
'value': value})['value']
File
"C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 256, in execute
self.error_handler.check_response(response)
File
"C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\errorhandler.py",
line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to
locate element: //div[contains(text(), 'Followers')]
_________________________________________________________________________________________________________________
When i add paranteces on followers, i get another error message:
_________________________________________________________________________________________________________________
C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>python
instagramcrawler.py -q instagram -t 'followers' -n 30
dir_prefix: ./data/, query: instagram, crawl_type: 'followers', number: 30,
caption: False
Saving...
Saving to directory..../data/instagram
Quitting driver...
C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>
_________________________________________________________________________________________________________________
Thanks for your help :)
2017-06-30 14:22 GMT+02:00 Antonie Lin <[email protected]>:
Can you give me more information & environment on your query?
(E.g. the command you used to crawl followers)
Thanks!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AVm_4jEV85BqAgbrUue8mIGbsaaeX9n1ks5sJOiUgaJpZM4OJSl2>
.
from instagramcrawler.
tzuhsial
commented on July 2, 2024
@Thicool
Did you login your instagram account?
(The webdriver will open up the browser for you to login)
from instagramcrawler.
Thicool
commented on July 2, 2024
Yes it opens firefox and i login, then the driver continues to visit the
follower Page. Everything works fine until here. Maybe i try another
computer later.
Regards,
Jan
Am 03.07.2017 6:11 vorm. schrieb "Antonie Lin" <[email protected]>:
… Did you login your instagram account?
(The webdriver will open up the browser for you to login)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AVm_4ufCdBxcOeHSb2AihsgLXtdfjRBJks5sKGnUgaJpZM4OJSl2>
.
from instagramcrawler.
tzuhsial
commented on July 2, 2024
@Thicool
I think its because after clicking on the element,
my code only waits for 1 second before scrolling the followers list
I'll fix and push it later
Antonie
from instagramcrawler.
tzuhsial
commented on July 2, 2024
@Thicool
Pushed, please check if you have time...
And also, thanks for notifying!
Regards,
Antonie Lin
from instagramcrawler.
Thicool
commented on July 2, 2024
Thanks for your work, but the problem still exists in my case. I Attached Error Message and Gecklog.
Hope that helps!
from instagramcrawler.
tzuhsial
commented on July 2, 2024
@Thicool
It works for me when crawling followers/ following at the moment
The only solution I know have is to set the timeout value a bit larger(make it wait a little longer)...
from instagramcrawler.
mrpatricque
commented on July 2, 2024
Hi there! I am encountering the same issue here! How can I set the timeout value a bit larger? I can crawl for photos, but get the same error for followers/following. When crawling photos, there is no login done, and when I include crawling reactions, I get a different, more complicated error. Thanks!
from instagramcrawler.
Related Issues (20)
- Erratic but very common timeout exception HOT 1
- why can't I get the photo links?
- Timeout exception HOT 2
- css are dynamically generated, not hard coded HOT 1
- 'geckodriver' executable needs to be in PATH HOT 3
- how to crawl one user's all photos
- UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 637: character maps to <undefined>
- #168 loadmore.click() seems to be unstable
- How does this work?
- Only crawl 1 following or followers HOT 2
- Timeout error HOT 2
- "photo_links" only return 51(max) elements
- Crawler for "followers" stops after opening follower list HOT 9
- AttributeError: 'NoneType' object has no attribute 'group' HOT 6
- Firefox path not solve and Need gecko driver executable in path HOT 1
- Followers crawling HOT 2
- raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: Unable to find a matching set of capabilities
- Error when scraping captions HOT 4
- is there any way to run it as headless? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from instagramcrawler.