ffrankies / reddit_seer Goto Github PK

Considerations:

• Should they be the same bag of words, or separate?
• The bag of words has to be created across all titles and all post bodies, not just a single one.

Considerations:

• Should, at the very least, use the bag of words from the title and the post text to try to predict popularity
• Do not confuse linear regression with logistic regression. We can try to use logistic regression separately, but it will also predict "buckets" or categories, like the naive bayes algorithm.
• Ideally, would be extensible and allow additional features to be easily added

Considerations:

• This will not give us an actual number, but rather use "buckets" or categories (i.e.: unpopular, popular, neutral, average, etc)
• Defining and setting up these buckets will be a major consideration and will affect our output
• Will probably need to do a refresher on how this works (hopefully, though, there's a scikit-learn module for this to do the bulk of the work for us)
• Try to make this extensible in case we choose to add more features (e.g.: sentiment, variability from the average content, etc)

Need visualizations to make comparisons like the following:

• Does sentiment predict popularity?
• Does time of posting predict popularity?
• Does difference from average predict popularity? (semi-stretch goal?)

Considerations

• A stretch goal for us
• Multiple ways to use image info for popularity prediction:
  • Directly infer popularity from image (simple CNN or ANN, ANN would more easily allow us to add more features, but is less suitable for images than CNN)
  • Use some kind of ANN to automatically 'label' images and perform analysis on the labels
  • Use some kind of ANN to extract text from images, and perform analysis on the text

Considerations:

• Stretch goal
• Suggested by Dr. Moore - can lead to interesting discussion about whether or not reddit promotes discussion or not

Graphs we need:

• Loss over time for NNs
• Accuracy over time for NNs
• Accuracy for other methods

Considerations:

• Most likely, will be using the actual text, not bag-of-words
• Need to be careful with the libraries we use, since most of them just do a simple num_positive_words - num_negative_words calculation
• May be possible to do more complex sentiment analysis, such as extracting emotions

Currently, a basic idea is to use find an 'average' bag-of-words for a subreddit, and use the root-mean-square-difference algorithm on each post's bag-of-words to find how much it differs from the 'average'.

This might be even more useful if we do analysis on comments.