Code Monkey home page Code Monkey logo

embedeval's Introduction

NLP Embeddings Evaluation Tool

PyPI License
Actions Status Code style: black
PyPI version PyPI PyPI


The NLP Embeddings Evaluation Tool is a command line tool to evaluate Natural Language Processing Embeddings using custom intrinsic and extrinsic tasks.

Installation

embedeval is available as pip package:

python -m pip install embedeval

NOTE: it might not be installable as of today using pip with PyPI. However, installing from source will work. Use . instead of embedeval in the pip command.

Getting started

Run the word-analogy Task on your Word Embedding:

embedeval embedding.vec -t word-analogy

Run the word-analogy and word-similarity Tasks on your Word Embedding:

embedeval embedding.vec -t word-analogy -t word-similarity

Documentation

The whole documentation of embedeval is available on Read The Docs.

Supported platforms

embedeval is supported on Windows, Mac and Linux

Contribution

Yes, we are looking for some contributors and people who spread out a word about embedeval. Help us to improve these piece of software. You don't know what to do? Just have a look at the Issues or create a new one. Please have a look at the Contributing Guidelines, too.

Project Information

embedeval is released under the MIT license, its documentation lives at Read The Docs, the code on GitHub, and the latest release on PyPI. It’s rigorously tested on Python 3.5+.

If you'd like to contribute to embedeval you're most welcome and we've written a little guide to get you started!


This project is published under MIT.
A Timo Furrer project.
- 🎉 -

embedeval's People

Contributors

daviddust avatar timofurrer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

embedeval's Issues

Research possible Report formats

  • Is Command Line Output enough?
  • Clarify the need of PDF reports
  • What possible ways would we have to generate rich text reports?

Task definition

Extrinsischer Task

GermEval Task 2018 — Shared Task on the Identification of Offensive Language.

Subtask 1 (Binary classification)

Intrinisiche Tasks

Odd-one-out:

from gensim.models.fasttext import load_facebook_vectors


wv = load_facebook_vectors(external / 'cc.de.300.bin')

wv.doesnt_match(['Sonnencreme', 'Sonnenbrand', 'Sonnenmilch', 'Sonnenschutzcreme'])
wv.doesnt_match(['kaffee', 'tee', 'espresso', 'cappucino'])
wv.doesnt_match(['Metzgerei', 'Bäckerei', 'Bäckerladen', 'Brotgeschäft'])

Word analogy:

  • man to king as woman to ?
  • paris to france as bern to ?

und als multiple choice:

  • Q: mason:stone

    A:

  1. teacher:chalk
  2. carpenter:wood
  3. soldier:gun
  4. photographer:camera
  5. book:word

Schlusspräsentation erstellen

Für die Abschlusspräsentation wird auf Initiative der Betreuungsperson hin ein Prüfungstermin wäh-
rend der regulären Modulendprüfungs-Zeit (3 Wochen gemäss Eckdaten) gemeinsam mit der Exper-
tin/dem Experten festgelegt.

Aufbau (beispielhafte Zeitangaben):

  • Schlusspräsentation (20 Minuten)
  • Befragung/Verteidigung (20-30 Minuten)
  • Betreuungsperson & Experte/Expertin füllen gemeinsam Bewertungsraster aus (20 Minuten)
  • Mündliches Feedback an den/die Studierenden, mündliche Notenbekanntgabe optional
    Der Bewertungsbogen wird danach der Transferstelle eingereicht bis spätestens zum Montag vor dem
    offiziellen Termin der Bewertungserfassung gemäss Eckdaten.

Auftraggeber oder/und andere Projektpartner können an die Präsentation eingeladen werden, ohne je-
doch Einfluss auf die Bewertung zu haben.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.