example for twitter topic about twitter-scraper-selenium HOT 3 CLOSED

Thanks for the review @rachmadaniHaryono.

this is little bit confusing because all import error is catch with general exception see also example 1 below
if possible just let error happened and end the program

Yeah, moving import outside of try/catch will help catch the bug.

save json will replace old data, so be careful

The behaviour to write data in write mode is definitely misleading, this issue created misunderstanding for the user of my other library. Here's the issue. That's the feature I was even thinking of implementing to check if the file already exists and switch writing mode.

selenium can use custom profile folder, currently i have to edit on either set_properties or set_driver_for_browser on driver_initialization.Initializer

Yeah, Selenium can use a custom profile but I don't think It is going to help much in scraping as long as you're scraping unauthenticated way. Is it going to help in any way?

any reason why Keyword.scrap have to return json string? why not just return it as dict? when saving the data as csv, it have to be decoded back to dict

The output will be invalid JSON.

from twitter-scraper-selenium.

Yeah, Selenium can use a custom profile but I don't think It is going to help much in scraping as long as you're scraping unauthenticated way. Is it going to help in any way?

i'm thinking of scraping my twitter frontpage with this and use some firefox extension while scraping the data

maybe i will create pr to scrap twitter topic later

from twitter-scraper-selenium.

Oh, Okay. Thanks

from twitter-scraper-selenium.