anuvrat / scrape-google-play Goto Github PK

When I run crwalPlay.py(python version 3.2.3) ,I get error below

A fresh start ...
Type = free Cateory = BOOKS_AND_REFERENCE
https://play.google.com/store/apps/details?id=com.nhn.android.search
Traceback (most recent call last):
File "crawlPlay.py", line 191, in
getApps( url )
File "crawlPlay.py", line 168, in getApps
apps, skipped_apps = getTopAppsData( url, start_idx, size, app_type )
File "crawlPlay.py", line 157, in getTopAppsData
if app_details: apps.append( app_details )
UnboundLocalError: local variable 'app_details' referenced before assignment

Hi, We are also looking at scraping Google Play, and I was wondering about rate limiting. Did you ever experienced IP banning or rate limiting because you make so many requests? Do you use any sleep() function, or do you just download as much webpages as you can?