thisandagain / sentiment Goto Github PK
View Code? Open in Web Editor NEWAFINN-based sentiment analysis for Node.js.
License: MIT License
AFINN-based sentiment analysis for Node.js.
License: MIT License
Currently we have no way of processing smileys since they are stripped away before analysis (in tokenize.js
).
Such a feature serves the same purpose as supporting emojis.
For example, here is a non-exhaustive list of smileys:
The first expression in tokenize
takes out hyphens, wheres some words with hyphens exist in the AFINN database.
For example, "self-deluded" is mapped to -2, whereas "self" and "deluded" don’t exist in the AFINN database (thus result in a score of 0).
This can be fixed by allowing the hyphen in tokens.
Note: I haven’t tested if other punctuation exists in AFINN words.
I would like sentiment to be able to load external word lists of some kind. One of the following ways would be appreciated:
I think this would make sentiment much more versatile for other languages (german in my case).
Hi,
Could you tell me what would be the differences with this other module you created? https://github.com/thisandagain/troll
Thanks,
Philmod
In Node:
> s = require('sentiment')
> s('constructor')
{ score: '0function Object() { [native code] }',
comparative: NaN,
tokens: [ 'constructor' ],
words: [ 'constructor' ],
positive: [],
negative: [] }
This can be fixed by using a hasOwnProperty
check, instead of just getting it from the AFINN object.
You have good code coverage. What do you think of adding coveralls to Travis and a badge to the readme?
Can I install this on top of create-react-app
and run it in the browser?
All the messages are getting logged, how do I turn that off
The code that does the scoring of the tokens isn't actually doing anything asynchronously.
Adding an async call for each iteration adds a bit more overhead.
I suggest leaving the callback at the top level, but just doing a simple forEach on the tokens.
All we really need at this point is extend
(which can be brought in directly) and the forEach
iterator is best rewritten as a for loop (performance) anyway.
I want to have a compiled score of array of phrases ( for example compiled or single result for post sentiment against array of comments)
At the moment the sentence "John is not a good person" is evaluated as with a positive score. It would be great if you could implement a "negative expression" detection and proper score.
Thanks
It is not totally clear regarding the resulting values what the max value could be.
Please add to the documentation.
Nice work on this library! I found it helpful for one of my pet-projects, but when I tried to use it in the browser I saw you don't have a bundled version.
Have you considered using something like rollup to bundle you packages into a single deliverable?
The AFINN-111 wordlist was published in 2011. While it works well for quite a large range of tasks, a few lexicons have been published since that may provide improved accuracy. Using this issue as a place to track the evaluation of each. A little background on alternate approaches:
http://sentiment.christopherpotts.net/lexicons.html
http://www.umiacs.umd.edu/~saif/WebPages/Abstracts/NRC-SentimentAnalysis.htm
http://dmir.inesc-id.pt/project/SentiLex-PT_02_in_English
I have only reviewed sentiment analysis in respect to your comparative rate slightly but it seems a negative value like your Cats are stupid.
example isn't right after referencing this article Opinion Mining, Sentiment Analysis, and Opinion Spam Detection.
Can you point me in the direction or explain how this metric applies.
Love this package, but is there a way to be able to overwrite the value of emoji in a similar way to overwriting word values? Thanks!
The current benchmark script is pretty naive. Look at benchmark.js as a potential base.
Hi,
It is possible to allow for a option which first finds string distances to words in the positive/negative list, and then, if it is above some threshold, categorize it as that word so spelling mistakes and/casual writing style are not lost.
e.g.
> sentiment('Cats are dumb');
{ score: -3,
comparative: -1,
tokens: [ 'cats', 'are', 'dumb' ],
words: [ 'dumb' ],
positive: [],
negative: [ 'dumb' ] }
> sentiment('Cats are dumbbb');
{ score: 0,
comparative: 0,
tokens: [ 'cats', 'are', 'dumbbb' ],
words: [],
positive: [],
negative: [] }
This example dumbbb
is so close to dumb
that it should be classified as such. Using a library like natural makes this easy.
require('natural').JaroWinklerDistance('dumb', 'dumbbb')
0.9333333333333333
If adding natural is out of scope, maybe a way that someone could inject it in some processing step could work too.
What do you think? Would this work?
This is a very sweet project!
I would like to know how can I convert the comparative value to a whole number (percentage 100), where 100 is the most positive, 50 is neutral and 0 is most negative?
This can also be out of 10.
$ cat AFINN-111.txt | grep " "
can't stand -3
cashing in -2
cool stuff 3
does not work -3
dont like -2
fed up -3
green wash -3
green washing -3
messing up -2
no fun -3
not good -2
not working -3
right direction 3
screwed up -3
some kind 0
$ node -pe "require('sentiment')('it does not work').score"
0
``
Doing something like Seth is not not amazing
gives a negative score.
"Not" can totally flip a string's meaning, but it and other similar adverbs are getting overlooked.
i.e.,
"I am not happy"
scores a 3, when, because of the negative adverb in there preceding the adjective, it should get a score of -3.
Is this project still being maintained? The negation and emoji support would be helpful if it could be implemented.
tokenize.js
ignores !
, .
, and other punctuation symbols already, yet it does not escape ?
.
As sentiment is actually a sync method under the hood – the async interface is confusing to some users. This is particularly an issue as it doesn't use process.nextTick
to ensure that this behavior is normative. The default interface should become sync, while allowing an optional callback which is wrapped using process.nextTick
to ensure that the async interface works as expected.
When I first built this thing I was using Makefile
s for everything. While they work well for me, they cause a lot of problems for folks on Windows and other platforms where GNU Make is not available.
Could you explain what is the comparative value, and how it is computed
I have version 2.1.0 on both my computer and on a server. On my computer it works, on my server, I get the following
{ score: 3,
comparative: 3,
tokens: [ 'happy' ],
words: [ 'happy' ],
positive: [ 'happy' ],
negative: [] }
{ score: -2,
comparative: -2,
tokens: [ 'sad' ],
words: [ 'sad' ],
positive: [],
negative: [ 'sad' ] }
{ score: 0,
comparative: 0,
tokens: [ 'ok' ],
words: [],
positive: [],
negative: [] }
Any thoughts?
Feature Request: DefinitelyTyped typescript definitions
when I use NPM install sentiment - the installation fails - here is the output of the install
This install under windows
:\Program Files\NodeJS>npm install sentiment
npm WARN package.json [email protected] No README.md file found!
npm WARN package.json [email protected] No README.md file found!
npm WARN package.json [email protected] No README.md file found!
npm WARN package.json [email protected] No README.md file found!
npm http GET https://registry.npmjs.org/sentiment
npm http 200 https://registry.npmjs.org/sentiment
npm http GET https://registry.npmjs.org/sentiment/-/sentiment-0.2.0.tgz
npm http 200 https://registry.npmjs.org/sentiment/-/sentiment-0.2.0.tgz
npm http GET https://registry.npmjs.org/async
npm http GET https://registry.npmjs.org/lodash
npm http 200 https://registry.npmjs.org/async
npm http GET https://registry.npmjs.org/async/-/async-0.1.22.tgz
npm http 200 https://registry.npmjs.org/lodash
npm http GET https://registry.npmjs.org/lodash/-/lodash-1.0.0-rc.3.tgz
npm http 200 https://registry.npmjs.org/async/-/async-0.1.22.tgz
npm http 200 https://registry.npmjs.org/lodash/-/lodash-1.0.0-rc.3.tgz
[email protected] install c:\Program Files\NodeJS\node_modules\sentiment\node_
modules\lodash
node build/post-install
Oops! There was a problem detecting the install mode. If you're installing the
Lo-Dash command-line executable (via npm install -g lodash
), you'll need to
manually install UglifyJS and the Closure Compiler by running:
curl -H 'Accept: application/vnd.github.v3.raw' api.github.comhttps://api.github
.com/repos/bestiejs/lodash/git/blobs/a2787b470c577cee2404d186c562dd9835f779f5 |
tar xvz -C 'c:\Program Files\NodeJS\node_modules\sentiment\node_modules\lodash\v
endor'
curl -H 'Accept: application/vnd.github.v3.raw' api.github.comhttps://api.github
.com/repos/bestiejs/lodash/git/blobs/7ecae09d413eb48dd5785fe877f24e60ac3bbcef |
tar xvz -C 'c:\Program Files\NodeJS\node_modules\sentiment\node_modules\lodash\v
endor'
Please submit an issue on the GitHub issue tracker: https://github.com/bestiejs/
lodash/issues
{ [Error: Command failed: CreateProcessW: The system cannot find the file specif
ied.
] killed: false, code: 127, signal: null }
[email protected] node_modules\sentiment
├── [email protected]
└── [email protected]
New feature request: display the score calculation. Something like the following:
const sentiment = new Sentiment();
sentiment.analyze("Great job! 😃 👏", {debug: true} );
// returns { score: 7, calculation: [{'👏': 2 }, {'😃': 2 }, {'great': 3 }] , ... }
I was testing the the the library with the string "I'll be the one who killing the buy guys",
So, the replacement text .replace(/[^a-z0-9á-úñäâàéèëêïîöôùüûœç\-\' ]+/g, '')
in tokenize.js is chancing "I'll" to "ill" and killing and ill addup and give a more negative score.
The change I made works for me, I did this .replace(/[^a-z0-9á-úñäâàéèëêïîöôùüûœç\-\' ]+/g, '')
but unfortunately it's failing 3 tests. I am not every good at regular expressions.
Let me know what you think.
Hello,
I'm trying to override the AFINN scores for 2-grams, but it doesn't seem to work:
sentiment.analyze( 'This stuff is made up', { extras: { 'made up': -1 } } )
{ score: 0,
comparative: 0,
tokens: [ 'this', 'is', 'made', 'up' ],
words: [],
positive: [],
negative: [] }
The effect is even more accentuated when a 2-gram would flip the overall score of a phrase; here "fucking good" reinforces a positive word, but the overall score is -1:
sentiment.analyze( 'This stuff is fucking good', { extras: { 'fucking good': 3 } } )
{ score: -1,
comparative: -0.2,
tokens: [ 'this', 'stuff', 'is', 'fucking', 'good' ],
words: [ 'good', 'fucking' ],
positive: [ 'good' ],
negative: [ 'fucking' ] }
>
Would it be possible and a good idea to add support for overridden 2-grams?
From PR #93:
We'll need to establish a set of labeled sentences (see #103 as well as the original AFINN paper) in order be able to determine the accuracy of each translation. While Google Translate is great, sentiment can be pretty subtle and so I wonder if something like Pootle or Transifex might be able to help us source translations of the AFINN word list with greater confidence.
I've been thinking about ways to fund a CrowdFlower campaign to handle establishing labeled corpora for different languages. I don't think it would take much (maybe $60 - $120 USD) per language to get things started.
/cc @dkocich
PR #74 initially incorporated a change which used spliddit
's hasPair
method to split emoji without a "space" or other delimiter. While useful, this change had fairly serious performance consequences. I think it's worthwhile to include this functionality, but it should be optional.
This code to see if a token has a corresponding AFINN label:
if (!labels.hasOwnProperty(obj)) continue;
in many non English language does not work for inflected words.
In English you say: "the cat is stupid" and "the cats are stupid" (btw cats are smart! :D )
In French you say: "le chat est stupide" and "les chats sont stupides".
The stemmed word "stupid" is flected into stupide and stupides.
if we use "stupid" in the labels, the scoring is never done. Using all the possible inflected words is tedious and long.
may be you could change the line of code to take into account stemmed (or truncated) words?
Is maintaining compatibility with Node 4.x a priority, or could we bump it up to at least Node 6.9 (LTS)?
This would allow us to use the newer ES6 syntax, such as the class
construct, const
, let
, arrow functions, etc. This would also be timely with the upcoming major release.
Furthermore, could we update the linter to allow 2-spaces indents and 100 max line length?
Thanks for creating such a nice lib. but I regret to inform you that it is giving wrong results when a negative word is used in a positive manner.
var sentiment = require('sentiment');
var result = sentiment('I am dying to eat a kitkat!');
console.dir(result);
///or
result = sentiment('your internet is not bad', knowladgeBase);
console.dir(result);
Please help
I don't have much experience with js and I'm probably doing something wrong, but is there a proper way to use this in the browser? I'm trying to pass a variable from php to js in the browser so I can display the sentiment scores on a webpage. I have tried browserify but I when I attempt to output the variable it says undefined. Sorry if this makes no sense, I appreciate any help!
The AFINN wordlist is missing a fairly critical piece of contemporary discourse: the emoji. I'd like to add emoji support to sentiment. Major questions:
I just stumbled upon this article on VADER.
Perhaps we can draw inspiration from it and incorporate some elements in our scoring strategy as explained in the article to improve our precision.
See:
e.g:
sentiment('this is good'); // score: 3
sentiment('this is crazy good'); // score: 1
sentiment('this is fucking good'); // score: -1
I don't think this is a duplicate of "Support for double negatives" (#109) as it's not really a "double negative", it just contains a word that could in some contexts be negative but in other contexts be used to emphasize the meaning of another word.
Ideally sentiment would understand that in the string this is fucking good
the word fucking
is emphasizing the word good
.
So fucking good
would have a score of good
multiplied by the intensity of fucking
.
So if good
has a score of 3, crazy
has an intensity of 1.2 and fucking
has an intensity of 2. You'd get:
sentiment('this is good'); // score: 3
sentiment('this is crazy good'); // score: 3.6
sentiment('this is fucking good'); // score: 6
I don't really have a clue what I'm talking about so maybe this is way outside of the scope of this module. It would be great if this worked though.
Currently sentiment analyzer is not able to findout sentiments properly for multi-word like
"does not work": -3,
Below code is checking only one word at a time not multiple
var len = tokens.length;
while (len--) {
var obj = tokens[len];
var item = afinn[obj];
if (!afinn.hasOwnProperty(obj)) continue;
words.push(obj);
if (item > 0) positive.push(obj);
if (item < 0) negative.push(obj);
score += item;
}
Fixed the same in my code.
sentiment
is a straightforward implementation of the AFINN wordlist which has some drawbacks including lack of support for negation (e.g. "that is not awesome"). This has become a fairly common question / feature request and something I'd like to support. The major questions in my mind are:
Hello, actually for support your sentiment algorithm my api translate spanish to english and response a phrase in english, but I need to support spanish lenguage, The process is add the spanish words in AFINN.txt or I need to add in other places?.
Thanks
I keep getting the following error when I try to use your package:
ERROR in ./~/sentiment/build/AFINN.json
Module parse failed: /Users/joneslloyd/my_project/node_modules/sentiment/build/AFINN.json Unexpected token (1:10)
The JSON file in question looks to be correct though..
We currently validate against a dataset from UCI that includes Amazon, Yelp, and IMDB. This is great but it would be nice to have less formal texts (particularly those that include emoji) included in validation. Various NLP areas are well explored using Twitter as a corpus so I don't think this should be too difficult to track down, but will require some research.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.