mkearney / tweetbotornot Goto Github PK
View Code? Open in Web Editor NEW🤖 R package for detecting Twitter bots via machine learning
Home Page: https://tweetbotornot.mikewk.com
License: Other
🤖 R package for detecting Twitter bots via machine learning
Home Page: https://tweetbotornot.mikewk.com
License: Other
Hi there.
I am very keen on testing out your package on R, but I have been unable to install this package from the steps you have provided. I have tried this on multiple computers, to no avail. This is error message that I have been getting:
`Downloading GitHub repo mkearney/botornot@master
from URL https://api.github.com/repos/mkearney/botornot/zipball/master
Installing botornot
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save
--no-restore --quiet CMD INSTALL
'/private/var/folders/8r/4b1mddcd5_d2_89fyb8x2_4r0000gn/T/RtmpPZMT3N/devtools8be10fdd7ef/mkearney-botornot-780fb5e'
--library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library' --install-tests
During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C"
2: Setting LC_TIME failed, using "C"
3: Setting LC_MESSAGES failed, using "C"
4: Setting LC_MONETARY failed, using "C"
I hope you can help me out! Your project really excites me, and I can't wait to get started with this package. Thank you!
Hi, I am newish using R and very interested in your package, but I can't download it. I have asked a colleague to install it and they couldn't either. Please advice.
I am using RStudio Version 1.1.419 and when I coy your code to install the package I get the following msg:
ERROR: dependencies 'dplyr', 'tibble' are not available for package 'tweetbotornot'
Installation failed: command failed (1)
It's probably because I'm ~50% robot…or it could be the Twitter API— I'm not sure. 🤔
Do I need token to play 😭?
## load package
library(botornot)
## select users
users <- c("realdonaldtrump", "netflix_bot",
"kearneymw", "dataandme", "hadleywickham",
"ma_salmon", "juliasilge", "tidyversetweets")
## get botornot estimates
data <- botornot(users)
#> Error code: 32
#> Could not authenticate you.
#> Error in if (tub == "tweets") {: argument is of length zero
Created on 2018-02-16 by the reprex package (v0.2.0).
I was testing the package on my follower list. I am having this error. I have about 900 followers in twitter.
> data <- botornot(df.haseebmahmud$screenName)
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Warning: Rate limit exceeded - 88
Error in `$<-.data.frame`(`*tmp*`, "app", value = "twitter") :
replacement has 1 row, data has 0
Could anyone can give me any hint what is wrong? I have the same error with an account having 80000 followers. Also, I tried to run the example codes and had the same error in data.frame.
> users <- c("realdonaldtrump", "netflix_bot",
+ "kearneymw", "dataandme", "hadleywickham",
+ "ma_salmon", "juliasilge", "tidyversetweets",
+ "American__Voter", "mothgenerator", "hrbrmstr")
> dataq <- botornot(users)
Warning: Rate limit exceeded - 88
Error in `$<-.data.frame`(`*tmp*`, "app", value = "twitter") :
replacement has 1 row, data has 0
My sessioninfo() is as follows,
> sessionInfo()
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 18.2
Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=de_DE.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=de_DE.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] botrnot_0.0.2 gridExtra_2.3 forcats_0.3.0 stringr_1.3.0 dplyr_0.7.4 purrr_0.2.4 readr_1.1.1
[8] tidyr_0.8.0 tibble_1.4.2 ggplot2_2.2.1 tidyverse_1.2.1 rtweet_0.6.20 twitteR_1.1.9
loaded via a namespace (and not attached):
[1] reshape2_1.4.3 haven_1.1.1 lattice_0.20-35 colorspace_1.3-2 yaml_2.1.18 rlang_0.2.0 pillar_1.2.1
[8] foreign_0.8-69 glue_1.2.0 DBI_0.8 bit64_0.9-7 modelr_0.1.1 readxl_1.0.0 bindrcpp_0.2.2
[15] bindr_0.1.1 plyr_1.8.4 munsell_0.4.3 gtable_0.2.0 cellranger_1.1.0 rvest_0.3.2 psych_1.8.3.3
[22] parallel_3.4.4 curl_3.1 broom_0.4.4 Rcpp_0.12.16 openssl_1.0.1 scales_0.5.0 jsonlite_1.5
[29] bit_1.1-12 mnormt_1.5-5 rjson_0.2.15 hms_0.4.2 stringi_1.1.7 grid_3.4.4 cli_1.0.0
[36] tools_3.4.4 magrittr_1.5 lazyeval_0.2.1 crayon_1.3.4 pkgconfig_2.0.1 xml2_1.2.0 lubridate_1.7.3
[43] assertthat_0.2.0 httr_1.3.1 rstudioapi_0.7 R6_2.2.2 nlme_3.1-131.1 compiler_3.4.4
Running the code shown below gives me the error
"Error: 'join_rtweet' is not an exported object from 'namespace:rtweet' "
tl = get_timeline("realDonaldTrump", n = 1000)
botornot(tl)
botrnot(tl)
botornot("realDonaldTrump")
botrnot("realDonaldTrump")
Any ideas?
Hi! I'm getting this error from the Twitter API:
Warning: /1.1/statuses/user_timeline.json - Not authorized.
Im calling the script with the Fast parameter set to true, so it shouldn't get the user's timeline, right?
data <- tweetbotornot(users, fast=TRUE)
Hello Mike,
Thanks for this great package! I have been using it quite extensively lately (to characterize >12 000 user timelines) and cannot wait for your detailed description of it ;-)
I'm not quite sure that my remark qualifies as "issue" but:
I have noticed that accounts with fewer than 100 tweets, no user description, or protected accounts, lead to a failure of the algorithm, which is perfectly understandable... but can't we also take these "missing data" as clues as to whether or not a Twitter user is a bot?
I mean, protected accounts probably mean human-driven ones (I guess?). I would also say that a lack of tweets for a rather old account tends to point to a human being, too. As for lack of description, well... I don't know. I would have said "human" too but I'm not quite sure about this one.
So:
What's your take on this?
Do you think that the fact that these data are missing could be somehow turned into data at some point?
Thanks and keep up the great work :-)
Lise
Hi,
I tried to use the package, but it gives me the following error. What should I do?
## load package
library(botornot)
## select users
users <- c("realdonaldtrump", "netflix_bot",
"kearneymw", "dataandme", "hadleywickham",
"ma_salmon", "juliasilge", "tidyversetweets")
## get botornot estimates
data <- botornot(users)
#> Error code: 89
#> Invalid or expired token.
#> Error in if (tub == "tweets") {: argument is of length zero
## arrange by prob ests
data[order(data$prob_bot), ]
#> Error in data$prob_bot: object of type 'closure' is not subsettable
devtools::session_info()
#> Session info -------------------------------------------------------------
#> setting value
#> version R version 3.4.3 (2017-11-30)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_United States.1252
#> tz America/New_York
#> date 2018-02-16
#> Packages -----------------------------------------------------------------
#> package * version date source
#> backports 1.1.2 2017-12-13 CRAN (R 3.4.1)
#> base * 3.4.3 2017-12-06 local
#> botornot * 0.0.1 2018-02-16 Github (mkearney/botornot@c8920a5)
#> compiler 3.4.3 2017-12-06 local
#> curl 3.1 2018-02-16 Github (gaborcsardi/curl@cf186e8)
#> datasets * 3.4.3 2017-12-06 local
#> devtools 1.13.4 2017-11-09 CRAN (R 3.4.3)
#> digest 0.6.15 2018-01-28 CRAN (R 3.4.3)
#> evaluate 0.10.1 2017-06-24 CRAN (R 3.4.3)
#> graphics * 3.4.3 2017-12-06 local
#> grDevices * 3.4.3 2017-12-06 local
#> hms 0.4.1 2018-01-24 CRAN (R 3.4.1)
#> htmltools 0.3.6 2017-04-28 CRAN (R 3.4.1)
#> httr 1.3.1 2017-08-20 CRAN (R 3.4.3)
#> jsonlite 1.5 2017-06-01 CRAN (R 3.4.1)
#> knitr 1.19 2018-01-29 CRAN (R 3.4.1)
#> magrittr 1.5 2014-11-22 CRAN (R 3.4.2)
#> memoise 1.1.0 2017-04-21 CRAN (R 3.4.1)
#> methods * 3.4.3 2017-12-06 local
#> openssl 1.0 2018-02-02 CRAN (R 3.4.1)
#> pillar 1.1.0 2018-01-14 CRAN (R 3.4.1)
#> pkgconfig 2.0.1 2017-03-21 CRAN (R 3.4.1)
#> R6 2.2.2 2017-06-17 CRAN (R 3.4.1)
#> Rcpp 0.12.15 2018-01-20 CRAN (R 3.4.1)
#> readr 1.1.1 2017-05-16 CRAN (R 3.4.1)
#> rlang 0.1.6.9003 2018-02-16 Github (tidyverse/rlang@f538768)
#> rmarkdown 1.8 2017-11-17 CRAN (R 3.4.1)
#> rprojroot 1.3-2 2018-01-03 CRAN (R 3.4.1)
#> rtweet 0.6.17 2018-02-16 Github (mkearney/rtweet@7e106d6)
#> stats * 3.4.3 2017-12-06 local
#> stringi 1.1.6 2017-11-17 CRAN (R 3.4.2)
#> stringr 1.2.0 2017-02-18 CRAN (R 3.4.1)
#> tibble 1.4.2 2018-01-22 CRAN (R 3.4.1)
#> tools 3.4.3 2017-12-06 local
#> utils * 3.4.3 2017-12-06 local
#> withr 2.1.1.9000 2018-01-15 Github (jimhester/withr@df18523)
#> yaml 2.1.16 2017-12-12 CRAN (R 3.4.3)
Created on 2018-02-16 by the reprex package (v0.2.0).
Getting the following error when running the sample code from the ReadMe.
Error in predict.xgb.Booster(predmodel, as.matrix(x[, -(1:3)])) : [15:54:07] amalgamation/../src/learner.cc:506: Check failed: mparam_.num_feature != 0 (0 vs. 0) : 0 feature is supplied. Are you using raw Booster interface? Stack trace: [bt] (0) 1 xgboost.so 0x0000000119225fb0 dmlc::LogMessageFatal::~LogMessageFatal() + 112 [bt] (1) 2 xgboost.so 0x00000001192b3b09 xgboost::LearnerConfiguration::ConfigureNumFeatures() + 265 [bt] (2) 3 xgboost.so 0x00000001192a899b xgboost::LearnerConfiguration::Configure() + 971 [bt] (3) 4 xgboost.so 0x00000001192a9fda xgboost::LearnerImpl::Predict(std::__1::shared_ptr<xgboost::DMatrix>, bool, xgboost::HostDeviceVector<float>*, unsigned int, bool, bool, bool, bool, bool) + 90 [bt] (4) 5 xgboost.so 0x0000000119286d2a XGBoosterPredict + 266 [bt] (5) 6 xgboost.so 0x0000000119223ea9 XGBoosterPredict_R + 121 [bt] (6) 7 libR.dylib
Is this an xgboost bug or something on my end?
I updated rtweet and RStudio after my Twitter API was being picky (meaning, taking forever to load). Tweetbotornot worked for one round, but then I started receiving 404 Errors.
I checked a sample of the usernames and I got the 404 error — which is weird because when I manually looked up those usernames, they all existed. At first, I thought it was an API error. But when I tried it with a different API, it still didn't work. And that API token also worked with typical rtweet functions.
So now I can't help but wonder if it's some sort of package error. I am a bit new to scraping Twitter so perhaps this makes zero sense, but I'm slightly desperate for answers. Given the rtweet update, I thought there was a chance this could make sense.
Hi all, I've followed the install instructions found here: https://www.rdocumentation.org/packages/botrnot/versions/0.0.2
After running this code:
if (!requireNamespace("devtools", quietly = TRUE)) {
install.packages("devtools")
}
devtools::install_github("mkearney/botrnot")
I receive the following error:
√ checking for file 'C:\Users\Julie\AppData\Local\Temp\Rtmp6ZLiEY\remotes31c866dd117c\mkearney-tweetbotornot-7109d40/DESCRIPTION' (850ms)
Installing package into �C:/Users/Julie/Documents/R/win-library/4.0�
(as �lib� is unspecified)
Thanks for your help!
Hey,
I've installed every dependency but still I'm facing issue, please check below:
users <- c("realDonaldTrump")
data <- botornot(users)
↪ Counting features in text...
↪ Sentiment analysis...
↪ Parts of speech...
✔ Job's done!
↪ Counting features in text...
↪ Sentiment analysis...
↪ Parts of speech...
✔ Job's done!
Error: 'by' can't contain join column 'user_id' which is missing from LHS
Callrlang::last_error()
to see a backtrace
> rlang::last_error()
message: 'by' can't contain join column 'user_id' which is missing from LHS
class: rlang_error
backtrace:
tweetbotornot::botornot(users)
tweetbotornot:::botornot.character(users)
tweetbotornot:::botornot.data.frame(x, fast = fast)
tweetbotornot:::extract_features_ytweets(x)
dplyr:::left_join.data.frame(dd1, dd2, by = "user_id")
dplyr:::left_join.tbl_df(...)
dplyr:::common_by.character(by, x, y)
dplyr:::common_by.list(by, x, y)
dplyr:::bad_args(...)
dplyr:::glubort(fmt_args(args), ..., .envir = .envir)
Call rlang::last_trace() to see the full backtrace
Thanks in advance
Getting this error when executing data <- tweetbotornot(users)
. Tried with R 3.5.1 and 3.4.4.
I will try to fix this by myself. In the meantime I leave this issue reported here.
Thanks!
Package looks great! When I attempt to install from github, it ultimately fails saying the dependency textfeatures
is not available. This package isn't on CRAN and the only github repo I found is seriously incomplete (like no NAMESPACE
). Is this dependency meant to be there and, if so, where can it be found?
Hi.
@mkearney, you mentioned the accuracy of the model, but you didn't mention the threshold used to get to that accuracy. Is it 0.5, 0.95?
Is there language-specific features in any of the models, or are all independent of the user's language?
Thank you in advance.
Hello. Just beginning working with rtweet and tweetbotornot.
Attempting to follow the synopsis code:
> library(tweetbotornot)
> users <- c("realdonaldtrump", "netflix_bot",
+ "kearneymw", "dataandme", "hadleywickham",
+ "ma_salmon", "juliasilge", "tidyversetweets",
+ "American__Voter", "mothgenerator", "hrbrmstr")
> users
[1] "realdonaldtrump" "netflix_bot" "kearneymw"
[4] "dataandme" "hadleywickham" "ma_salmon"
[7] "juliasilge" "tidyversetweets" "American__Voter"
[10] "mothgenerator" "hrbrmstr"
> data <- botornot(users)
Error in get(name, envir = asNamespace(pkg), inherits = FALSE) :
object 'char_to_b64' not found
>
Gives me an error.
Any suggestions?
TIA!
Windows: 8.1
R: x64 3.5.1 (2018-07-02) -- "Feather Spray"
R Studio: Version 1.1.456
hello,
I just wonder: is this a real package? Like juliasilge with a 50% bot probability seems a pretty bad prediction right? Am I missing something?
Thanks!
I tried install from repo but it said not available for R 3.4.3 then I try to clone from github and I cannot compile the package:
The downloaded binary packages are in
G:\AppData\Local\Temp\RtmpwflZAo\downloaded_packages
checking for file 'G:\AppData\Local\Temp\RtmpwflZAo\remotes494062b02c5b\mkearney-tweetbotornot-4631a82/DESCRIPTION' ...
checking for file 'G:\AppData\Local\Temp\RtmpwflZAo\remotes494062b02c5b\mkearney-tweetbotornot-4631a82/DESCRIPTION' ...
√ checking for file 'G:\AppData\Local\Temp\RtmpwflZAo\remotes494062b02c5b\mkearney-tweetbotornot-4631a82/DESCRIPTION' (537ms)
preparing 'tweetbotornot':
checking DESCRIPTION meta-information ...
checking DESCRIPTION meta-information ...
√ checking DESCRIPTION meta-information
checking for LF line-endings in source and make files and shell scripts
checking for empty or unneeded directories
looking to see if a 'data/datalist' file should be added
building 'tweetbotornot_0.1.0.tar.gz'
When following the example code on the repo, I get an error with the line data <- botornot(users)
Error in init_oauth1.0(self$endpoint, self$app, permission = self$params$permission, :
Unauthorized (HTTP 401).
Do I need to be providing info to rtweet? If so, would be good to modify the example.
Just noting that for people not used to GitHub installs, as of 5 March the dependency on textfeatures is another mkearney GitHub repo that needs installing first. Maybe mentioning this in the ReadMe might be good guidance? I think it is likely to generate queries from people.
Thanks for writing this. I tried your sample code and got the following error:
Error in init_oauth1.0(self$endpoint, self$app, permission = self$params$permission, :
Unauthorized (HTTP 401).
Any insight into this?
Thanks!
On installing botrnot package I get the following error
ERROR: dependencies 'rlang', 'textfeatures' are not available for package 'botrnot'
On attempting to install rlang manually I get this error...
install.packages("rlang")
Installing package into ‘C:/Users/xxxx/Documents/R/win-library/3.3’
(as ‘lib’ is unspecified)
Warning message:
package ‘rlang’ is not available (for R version 3.3.2)
interesting, I tried this with @nbohorad, my twitter account, and it ranked highly as Bot. but I'm not:) Is it based on RT'ing out more articles vs. more human conversations?
Do I need to create a data frame with specific column names?
I am trying to install the package to my Windows 10 PC. None of the following works:
install.packages('tweetbotornot',dependencies = T)
devtools::install_github("mkearney/tweetbotornot",INSTALL_opts = c("--no-multiarch", "--no-test-load"))
devtools::install_github("mkearney/tweetbotornot")
If I try the first one, I get the following error:
Warning in install.packages :
package ‘tweetbotornot’ is not available (for R version 3.5.3)
And the other devtools options give:
Error in i.p(...) :
(converted from warning) installation of package ‘C:..../tweetbotornot_0.1.0.tar.gz’ had non-zero exit status
I've downloaded the package and attempted to run it on a vector of usernames, but I get an error result:
Error in readRDS(x) : error reading from connection
I am getting an error that suggests the package is successful in getting tweets in but fails at processing them.
tweetbotornot('oganm')
Error in tolower(char_v) :
invalid input ' poor harry í ½í¸¢' in 'utf8towcs'
The "poor harry" is indeed one of my tweets but it seems to be having troubles with it. This is a windows machine so there might be some unexpected encoding issues
Hello,
I’m having issues with the accuracy of tweetbotornot’s accuracy. For instance, my research partner who I speak with almost daily shows a 98% probability of being a bot, a friend who’s home I’ve eaten in a 96%, 2/3rds of my followers (2395 out of 3319) are showing 90% or higher, and only 200 have less than a 50% probability, and less than 40 are in single digits. I’m wondering what can be done to improve that accuracy rate, as right now it doesn’t seem to be in that 91-93% accuracy mentioned in the readme. If I’m misreading the results, sorry in advance.
Hi!
I'm running this snippet on a csv file containing 200 user names:
library("tweetbotornot")
rts <- read.csv(file="rts.csv", header=TRUE, sep=",")
data <- tweetbotornot(rts$Twitter.Name, fast = TRUE)
I'm setting the fast argument on, however I'm getting this error:
Warning message:
“Rate limit exceeded - 88”
Error in if (n%/%200 < n.times) {: argument is of length zero
Traceback:
Am i doing something wrong?
Thanks!
I've been using botornot package (haven't done any analysis yet), but just realized there's tweetbotornot.. Is this different from botornot R package? Also, I can't download tweetbotornot like I did for botornot
if (!requireNamespace("devtools", quietly = TRUE)) {
install.packages("devtools")
}
devtools::install("C:\Users\glori\Documents\R\win-library\3.4\botrnot-master\botrnot-master")
library(botrnot)
if (!requireNamespace("devtools", quietly = TRUE)) {
install.packages("devtools")
}
devtools::install("\durfile01\redirected$\KimG1\Documents\R\win-library\3.5\tweetbotornot-master")
library(tweetbotrnot)
Do I need to specify more afre tweetbotornot-master on the path?
I see that rtweet supports a parameter called retryonratelimit
, but I cannot find this parameter in the botrnot package. I have a list of about 1 million user IDs to classify, so I need rate limiting handled properly.
Any suggestions?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.