spaCy pipeline object for negating concepts in text. Based on the NegEx algorithm.
NegEx - A Simple Algorithm for Identifying Negated Findings and Diseasesin Discharge Summaries Chapman, Bridewell, Hanbury, Cooper, Buchanan
Install the library.
pip install negspacy
Import library and spaCy.
import spacy
from negspacy.negation import Negex
Load spacy language model. Add negspacy pipeline object. Filtering on entity types is optional.
nlp = spacy.load("en_core_web_sm")
negex = Negex(nlp, ent_types=["PERSON","ORG"])
nlp.add_pipe(negex, last=True)
View negations.
doc = nlp("She does not like Steve Jobs but likes Apple products.")
for e in doc.ents:
print(e.text, e._.negex)
Steve Jobs True
Apple False
Consider pairing with scispacy to find UMLS concepts in text and process negations.
- psuedo_negations - phrases that are false triggers, ambiguous negations, or double negatives
- preceding_negations - negation phrases that preceed an entity
- following_negations - negation phrases that follow an entity
- termination - phrases that cut a sentence in parts, for purposes of negation detection (.e.g., "but")
Use own patterns
nlp = spacy.load("en_core_web_sm")
negex = Negex(nlp, termination=["but", "however", "nevertheless", "except"])
View patterns in use
patterns_dict = negex.get_patterns
Depending on the Named Entity Recognition model you are using, you may have negations "chunked together" with nouns. For example when using scispacy:
nlp = spacy.load("en_core_sci_sm")
doc = nlp("There is no headache.")
for e in doc.ents:
print(e.text)
# no headache
This would cause the Negex algorithm to miss the preceding negation. To account for this, you can add a chunk_prefix
:
nlp = spacy.load("en_core_sci_sm")
negex = Negex(nlp, chunk_prefix = ["no"])
nlp.add_pipe(negex)
doc = nlp("There is no headache.")
for e in doc.ents:
print(e.text, e._.negex)
# no headache True
- Jeno Pizarro