jubins / machinelearning-detecting-twitter-bots Goto Github PK

Custom classification algorithm to sense the bots vs human on social media space like twitter

License: MIT License

Python 1.24% Jupyter Notebook 98.76%

bot-detection machine-learning twitter-bot-detection

machinelearning-detecting-twitter-bots's Introduction

Detecting Bots on Twitter Using Machine Learning

Twitter bot is a program used to produce automated posts, follow Twitter users or serve as spam to entice clicks on the Twitter microblogging service. In this project, we will use Machine Learning techniques to predict weather an account on Twitter is a Bot or a real user. We have performed significant amount of feature engineering, along with feature extraction - selected features out of 20 helped us to identify whether an account is bot or non bot. We implemented various algorithm but finally implemented our own which gave us AUC>95%.

1. Depenedencies

Python 3, Pandas, Numpy, Seaborn, MatplotLib, Sklearn

2. Code

The code and datasets can be found under 'Final Project and Code' folder.

3. Video Link

https://www.youtube.com/embed/bRjxeovhL50?ecver=2

Thank you

Thank you for visiting this repository and looking at this project. Please feel free to contribute and take our analysis further.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

machinelearning-detecting-twitter-bots's People

Contributors

Stargazers

Watchers

Forkers

bbr234 holmes0078 calchulus ne18ha shivani156 utkutugrul pierluigicdc silpa12345 annurfuad dimmope datacontrol akshaytoshniwal vihanga-bandara tendai-zw saman45 rubenrodriguezcasan kaka2nd rsquared2016 martha92 raju-srkrec fernandoruizanton nileshbakshi concretesteelbond raulpena bochuxt turbonemesis asjqmasjqm ashutoshbarve filipeamorais rajmookerjee theonlyjoelj orchestor shashankdhyan rioyuniaedu japc78 andimo11 kant jheisecke raffienficiaud ajay0103 param15119 sitanshuk08 pamidimarrirushitha issa-kabir mishav78 rajatverma7 omarjnb mysticaltech srinidhi12 chakreshvarshney trivedisorabh jacobfv soumya-dey amigox napo178 antonyboza tarrun99 jd174 dix6621 dreew23 hams12khan sravanikasula12 ltgrp abeeralsharu road1228 er2826 patrickveillon

machinelearning-detecting-twitter-bots's Issues

No Test Data

Hey
Thanks for posting this

When I run python3 on the program it throws a unicode error
Also I believe the test data set is empy

Your main algorithm may not be that accurate than it is showing

I have seen your algorithm for studying purposes but found that it is not accurate.But coincidentally it showin above 95% accuracy.In the fucntion of the bot_prediction_algorithm() prediction_df is not sorted according to the indexes of the rows,while the train_df is by sequence indexing of the rows.So while finding the accuracy it is coincidentally showing above 95%.When I sorted the prdeiction_df it is showing aroung 69 % of the accuracy.Correct me if I am wrong

Test Data

Hey
So i have a file of tweets but its not exactly in the same order or contain the header fields like your test data, any idea how I can test on this different tweet file ?

I can rearrange the columns to match your test data file but is there an alternative you may have tried?

TypeError: only integer scalar arrays can be converted to a scalar index

I executed the code and I get the following error

Training the classifier. Please wait 30 seconds.
Traceback (most recent call last):
File "BotDetection.py", line 169, in <module>
print("Train Accuracy: ", twitter_bot.get_accuracy_score(train_df))
File "BotDetection.py", line 120, in get_accuracy_score
(X_train, y_train, X_test, y_test) = twitter_bot.perform_train_test_split(df)
File "BotDetection.py", line 25, in perform_train_test_split
train, test = df[msk], df[~msk]
TypeError: only integer scalar arrays can be converted to a scalar index

Kindly help me rectify this error

DataFrame' object has no attribute 'ix'. Did you mean: 'id'?

Hey! I tried to run your project but I got an error like this. How can I fix this?

Pandas parsing error

when I run the code with python2 or python3 it throws this error, even though it worked before.

Training the classifier. Please wait 30 seconds.
Traceback (most recent call last):
File "BotDetection.py", line 152, in
test_df = pd.read_csv(filepath + 'test_data_4_students.csv' , sep='\t', encoding='ISO-8859-1')
File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 655, in parser_f
return _read(filepath_or_buffer, kwds)
File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 411, in _read
data = parser.read(nrows)
File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 1005, in read
ret = self._engine.read(nrows)
File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 1748, in read
data = self._reader.read(nrows)
File "pandas/_libs/parsers.pyx", line 890, in pandas._libs.parsers.TextReader.read (pandas/_libs/parsers.c:10862)
File "pandas/_libs/parsers.pyx", line 912, in pandas._libs.parsers.TextReader._read_low_memory (pandas/_libs/parsers.c:11138)
File "pandas/_libs/parsers.pyx", line 966, in pandas._libs.parsers.TextReader._read_rows (pandas/_libs/parsers.c:11884)
File "pandas/_libs/parsers.pyx", line 953, in pandas._libs.parsers.TextReader._tokenize_rows (pandas/_libs/parsers.c:11755)
File "pandas/_libs/parsers.pyx", line 2184, in pandas._libs.parsers.raise_parser_error (pandas/_libs/parsers.c:28765)
pandas.errors.ParserError: Error tokenizing data. C error: Expected 10 fields in line 3, saw 105