rtapiaoregui / tokenizer_api Goto Github PK
View Code? Open in Web Editor NEWNLP task I was asked to carry out by a company
NLP task I was asked to carry out by a company
1) Files included in this package: - tokenizer_api (root folder): - README: this file - setup.py: script to build a python package with setuptools, including a description of the package's contents, metadata and dependencies - setup.cfg: configuration options for setup.py - MANIFEST.in: more details on the configuration of setup.py; it is meant to allow non-'.py' files to be installed as well. - pytest.ini: setting some of the parameters for pytest's configuration - usage_example.py: code to ilustrate how to lauch the RESTful API - .gitignore: lists the files that should not be uploaded to git - core (the folder with the package's code): - __init__.py: empty file to mark where the package's code is to be found - tokenizer.py: code to build and launch the RESTful API, as well as to tokenize the strings passed as input - collocations_dict.pickle: python dictionary containing the Oxford Online Collocations Dictionary - test (folder with code to test the package): - api_test.py: code to test the accessibility and error handling of the RESTful API's POST method - conftest.py: code to generate a pytest.fixture for api_test.py - process_text_test.py: code to test the function designed to tokenize strings, including its ability to raise the necessary exceptions - documentation (folder with pdf files): - Problem Statement.pdf: the PDF the company looking to hire me supplied me with, which includes the problem statement - to_git_or_not.pdf: the PDF where I explain how I have decided to use git and why 2) Package Installation: To install the package, please run >> python setup.py build from the root directory */tokenizer_api Once that is done, you should run >> python -m pip install . which will install the package in your active python environment You can now import custom_tokenizer (for an example, see usage_example.py) 3) Test Suite: The tests has been developed using pytest. To run the tests, simply execute >> python setup.py test from the root directory */tokenizer_api Don't be surprised to find out there are two tests the system is failing. This is part of its expected behaviour.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.