TypeError: the JSON object must be str, not 'bytes' about facebook-page-post-scraper HOT 5 OPEN

If anyone is having the same issue, I've found how to fix that! Just change the following code from the comments scraper:

def request_until_succeed(url):
    req = Request(url)
    success = False
    while success is False:
        try:
            response = urlopen(req)
            if response.getcode() == 200:
                success = True
        except Exception as e:
            print(e)
            time.sleep(5)

            print("Error for URL {}: {}".format(url, datetime.datetime.now()))
            print("Retrying.")

    return response.read()

To this one (i've added .decode('utf-8') before returning the value):

    req = Request(url)
    success = False
    while success is False:
        try:
            response = urlopen(req)
            if response.getcode() == 200:
                success = True
        except Exception as e:
            print(e)
            time.sleep(5)

            print("Error for URL {}: {}".format(url, datetime.datetime.now()))
            print("Retrying.")

    return response.read().decode('utf-8')

Now it's working fine here, but don't know if it's reliable for everyone, so I'm not going to submit a pull request with this fix.

from facebook-page-post-scraper.

The script does encoding/decoding shenanigans in order to be compatible with both Python 2 and 3. I will have to check if that solution will work for Python 2.

from facebook-page-post-scraper.

Thanks for the fast reply, @minimaxir !

from facebook-page-post-scraper.

Guys, again I have an issue with paging. Cannot figure out why it is happening. Can you help me? Thanks!
`---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
in ()
176
177 if name == 'main':
--> 178 scrapeFacebookPageFeedStatus(group_id, access_token)

in scrapeFacebookPageFeedStatus(group_id, access_token)
160 if 'paging' in statuses:
161 next_url = statuses['paging']['next']
--> 162 until = re.search('until=([0-9]*?)(&|$)', next_url).group(1)
163 if until is None:
164 return None

AttributeError: 'NoneType' object has no attribute 'group'`

from facebook-page-post-scraper.

@paladini thanks worked for me

from facebook-page-post-scraper.