Crawler for "followers" stops after opening follower list about instagramcrawler HOT 9 OPEN

I had a similar issue.
How about check getting updated 'page_source' ?
I fixed little after check that.

from instagramcrawler.

What exactly has to be done to solve this issue? I am not able to fix it. Will there be an update, otherwise this app has to be flagged as not working.

from instagramcrawler.

I fixed it little but its not perfect
I think Instagram changed something.

from instagramcrawler.

And could you tell us WHAT you changed?

from instagramcrawler.

I changed it to parse the html (maybe '_driver.page_source' in code) everytime after scroll down.
You find where the code stucks?
my answer won't be the answer because i used it to scrape the image

from instagramcrawler.

The problem is inside scrape_followers_or_following. I added some debug text and it does find the list of followers:

    List = title_ele.find_element_by_xpath(
        '..').find_element_by_tag_name('ul')
    print('found list', List.text)

==> generates:

('found list', u'peko11jun123\njun\nFollow\n_tural_255\nSherif\nFollow\njlidoctor\nJoseph Li\nFollow\ncamalmemmedov690\ncamal memmedov\nFollow\nsevgi.unvani\nSeVgi.Unvani \U0001f1e6\U0001f1ffAZE\nFollow\ntada0724\n\u30bf\u30c3\u30c9\nFollow\ntfvwbsevha\nBjSz2ID35HK2Y\nFollow\niwaishida\n\u5d0e\u5ca9\nFollow\nspchie137\nMorita.Chieko\nFollow\nismayil.62.840\nIsmayil Hesenov\nFollow')

However, the list is of length 1. It's being interpreted ad a single item. I've never used this framework so not familiar with how it's supposed to interpret lists.

    print('list len', len(List.find_elements_by_xpath('*')))

==>

('list len', 1)

from instagramcrawler.

Are there any solutions for it right now? Doesn't find a solution for this...

from instagramcrawler.

    # Loop through list till target number is reached
    num_of_shown_follow = len(List.find_elements_by_xpath('*'))
    print("Follower", num_of_shown_follow)

It's really the problem, that "num_of_shown_follow" is 1, even if there is a whole full follower list. What is the problem there?

from instagramcrawler.

When I print out the exception in this snippet

try:
                element.send_keys(Keys.PAGE_DOWN)
            except Exception as e:
	            print(e)

I get this exception Message:

Message: Element <li class="_6e4x5"> is not reachable by keyboard

from instagramcrawler.