Comments (2)
I pushed a commit for v0.7.0 that does normalization by default, where normalization is downcasing and NFKD-based diacritic removal. This cannot be configured by the convenience methods wn.words()
, wn.synsets()
, etc., but it can be configured when creating a wn.Wordnet
object via the normalize
parameter. I think morphological analysis, such as morphy, would be a separate process, perhaps configurable via a lemmatize
parameter, since the purpose is to find a lemmatic form.
from wn.
My thinking now is that this hypothetical lemmatize
parameter is a function with the following type signature:
Lemmatizer = Callable[[str, Optional[str]], Iterator[str]]
...where the two arguments are the word form and optional part of speech. For example:
def lemmatize(form: str, pos: str = None) -> Iterator[str]:
...
If the pos
is None, it yields values for all parts of speech. Also this function does not need to yield valid lemmatic forms. The results will be used for queries against the database, and these queries act as the filter, so there's no point in doing it twice. Something that requires a lexicon, such as Morphy's exception lists, can be instantiated to make use of a wordnet and still follow the signature:
class Morphy:
def __init__(self, wordnet: wn.Wordnet):
...
def __call__(self, form: str, pos: str = None) -> Iterator[str]:
...
I'm not sure if lemmatize
should take the output of any normalize
function, or vice versa. I can see arguments both ways. Maybe it needs some preprocessing pipeline instead: preprocessor=[normalize, lemmatize]
.
from wn.
Related Issues (20)
- Generalize glob-search for lexicon/project specifiers
- Syntactic behaviours not being properly added
- If you try to loop through senses first, hypernym_paths() hangs HOT 5
- Document "default mode" queries
- Missing Spanish definitions HOT 3
- antonyms in languages other than German and English HOT 5
- Support for PTB and Universal POS tags HOT 6
- If you create an entry with an ILIDefinition, but ill.id='' you lose the definition HOT 2
- Tracing back 'inferred' synsets to their reference lexicons HOT 3
- Stumped by multilingual relation traversal HOT 7
- Synset.relations() for some lexicons uses synset id as relation name
- Update Python versions HOT 2
- Is there any mapping between different English wordnet? HOT 4
- synset.relations fails with a KeyError HOT 3
- Merged synsets are lost in translation HOT 8
- SQLite objects created in a thread can only be used in that same thread HOT 3
- Add OEWN 2022 to index HOT 2
- Addition to NLTK migration guide w.r.t. offsets HOT 4
- Add a `conda` install option for `wn` on conda-forge channel HOT 5
- Allow contributors to self-assign issues with GitHub workflow HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wn.