Code Monkey home page Code Monkey logo

udon2's Introduction

UDon2 logo

Prepare your UD trees to be served!

UDon2 is a library for manipulating dependency trees in the Universal Dependencies format. Written in C++ with Python bindings through Boost.Python. Pre-compiled for Python 3.7-3.9 on Linux (both i386 and x86_64), Windows (win32 and win_amd64) and macOS (x86_64 only).

Note: Python 3.6 is supported for versions prior to v0.1.0 on Linux only (both i386 and x86_64).

pre-commit Build Status Downloads

📝 The development is on hiatus until February 2024

Documentation

Our online documentation is available at https://udon2.github.io/

Dealing with unsupported Python versions

If your version of Python is currently not supported by UDon2, you could take one of the 2 approaches:

  • Building the package from the sources, as described here. The advantage of this method is that you will most probably get a speed boost, compared to the pre-compiled versions on PyPI (see benchmarks to get a feeling for how much boost to expect). This approach is recommended for those people who feel comfortable with compiling stuff from sources.
  • Building a Docker container with one of the supported versions of Python and installing UDon2 package there. This is recommended for those who have never compiled anything in their lives. In fact to support this approach, we have provided a number of scripts out of the box (see below).

Using Docker

The first basic requirement is installing Docker. Docker has 2 concepts: images and containers. You can think of images as sort of templates for the kind of development environment you want (in our case the one that has Python and UDon2 installed). Containers are the actual environments that are built from these templates (Docker images). Once you get comfortable with this, the process is pretty simple. You can either go ahead and create your own Docker images and spin your own containers or use the provided helper scripts (in which case, please continue reading).

If you decide to use the helper scripts, please clone this repository first. Then follow these steps:

  1. Create a Docker image with Python and UDon2 installed. If you run sh tools/create_udon2_python_image.sh, this will create a Docker image called udon2:latest with Python 3.9 and UDon2 0.1.0 pre-installed. You can run docker images and see if you have such image listed (its size should be 922MB).
  2. Now we need to spin the actual container that uses this image. Create a folder for the project that will be using UDon2 and put all the files you need there. Additionally copy tools/run_udon2_docker.sh to that folder. Now navigate to that folder and run sh tools/run_udon2_docker.sh, which should start the docker container and give you access to the bash prompt. The folder from which you ran the shell script, will be available under $HOME/workspace inside the Docker container. Happy scripting!

Contributing

Please check our contribution guidelines.

Citing

The associated paper is accepted to the Universal Dependencies Workshop 2020 @ COLING 2020. If UDon2 is of any use in your research, please cite the paper.

@inproceedings{kalpakchi2020udon2,
  title={UDon2: a library for manipulating Universal Dependencies trees},
  author={Kalpakchi, Dmytro and Boye, Johan},
  booktitle={Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020)},
  pages={120--125},
  year={2020}
}

udon2's People

Contributors

dkalpakchi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

udon2's Issues

BUG

Describe the bug
I followed the documentation (https://udon2.github.io/basic/) and tried to parse a simple conllu file (see example below from https://github.com/EmilStenstrom/conllu/blob/master/README.md) using the IDLE Shell 3.8.10:

import udon2
# assumes that you have a file named 'test.conllu' with a valid CoNLL-U format
roots = udon2.ConllReader.read_file('test.conllu')

It didn't work. Instead, it caused a crash of the shell:


================================ RESTART: Shell ================================
>>> 

The follwing error was reported:

Failed to split a line of the CONLL-U format: Resource temporarily unavailable

To Reproduce

1   The     the    DET    DT   Definite=Def|PronType=Art   4   det     _   _
2   quick   quick  ADJ    JJ   Degree=Pos                  4   amod    _   _
3   brown   brown  ADJ    JJ   Degree=Pos                  4   amod    _   _
4   fox     fox    NOUN   NN   Number=Sing                 5   nsubj   _   _
5   jumps   jump   VERB   VBZ  Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin   0   root    _   _
6   over    over   ADP    IN   _                           9   case    _   _
7   the     the    DET    DT   Definite=Def|PronType=Art   9   det     _   _
8   lazy    lazy   ADJ    JJ   Degree=Pos                  9   amod    _   _
9   dog     dog    NOUN   NN   Number=Sing                 5   nmod    _   SpaceAfter=No
10  .       .      PUNCT  .    _                           5   punct   _   _

Expected behavior
The code above should work according to the documentation: https://udon2.github.io/basic/

Screenshots
If applicable, add screenshots to help explain your problem.
bug_udon2

Environment specs (please complete the following information):

  • OS: Ubuntu 20.04.4 LTS
  • Python version: Python 3.8.10
  • UDon2 version: '0.1.0'
  • Precompiled?: Yes

Additional context

No support via PyPI for Python 3.10

Describe the bug
I am trying (but failing) to install udon2 via pip.

To Reproduce
I tried running

pip install udon2

This fails with a metadata-generation-failed

Defaulting to user installation because normal site-packages is not writeable
Collecting udon2
  Using cached udon2-0.1.0.tar.gz (45 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-z_iz0t73/udon2_ef5a35f34543433fb466908e16db2def/setup.py", line 11, in <module>
          BOOST_DIR = os.environ['BOOST_DIR']
        File "/usr/lib/python3.10/os.py", line 680, in __getitem__
          raise KeyError(key) from None
      KeyError: 'BOOST_DIR'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

After taking a look at the docs, I tried to install Boost from my package manager and set LIB_BOOST accordingly, only to get a legacy-install-failure:

      In file included from udon2_cpp_src/Importer.cpp:5:
      udon2_cpp_src/Importer.h:9:10: fatal error: boost/python/list.hpp: No such file or directory
          9 | #include <boost/python/list.hpp>
            |          ^~~~~~~~~~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for udon2
  Running setup.py clean for udon2
Failed to build udon2
Installing collected packages: udon2
  Running setup.py install for udon2 ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for udon2 did not run successfully.
  │ exit code: 1
  ╰─> [47 lines of output]
      running install
      /home/harisont/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/udon2
      copying src/udon2/__init__.py -> build/lib.linux-x86_64-cpython-310/udon2
      creating build/lib.linux-x86_64-cpython-310/udon2/visual
      copying src/udon2/visual/utils.py -> build/lib.linux-x86_64-cpython-310/udon2/visual
      copying src/udon2/visual/templates.py -> build/lib.linux-x86_64-cpython-310/udon2/visual
      copying src/udon2/visual/render.py -> build/lib.linux-x86_64-cpython-310/udon2/visual
      copying src/udon2/visual/__init__.py -> build/lib.linux-x86_64-cpython-310/udon2/visual
      running build_ext
      building 'udon2.core' extension
      creating build/temp.linux-x86_64-cpython-310
      creating build/temp.linux-x86_64-cpython-310/udon2_cpp_src
      gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -I/usr/include/python3.10 -I/var/lib/pacman/local/boost-python2-1.78.0-1/include -I/usr/include/python3.10 -c udon2_cpp_src/ConllReader.cpp -o build/temp.linux-x86_64-cpython-310/udon2_cpp_src/ConllReader.o
      In file included from udon2_cpp_src/ConllReader.h:12,
                       from udon2_cpp_src/ConllReader.cpp:5:
      udon2_cpp_src/Node.h:211:44: warning: ‘template<class _Arg, class _Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
        211 | struct compare_to_node_by_id : public std::unary_function<Node *, bool> {
            |                                            ^~~~~~~~~~~~~~
      In file included from /usr/include/c++/12.2.0/string:48,
                       from udon2_cpp_src/ConllReader.h:8:
      /usr/include/c++/12.2.0/bits/stl_function.h:117:12: note: declared here
        117 |     struct unary_function
            |            ^~~~~~~~~~~~~~
      gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -I/usr/include/python3.10 -I/var/lib/pacman/local/boost-python2-1.78.0-1/include -I/usr/include/python3.10 -c udon2_cpp_src/ConllWriter.cpp -o build/temp.linux-x86_64-cpython-310/udon2_cpp_src/ConllWriter.o
      In file included from udon2_cpp_src/ConllWriter.h:10,
                       from udon2_cpp_src/ConllWriter.cpp:5:
      udon2_cpp_src/Node.h:211:44: warning: ‘template<class _Arg, class _Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
        211 | struct compare_to_node_by_id : public std::unary_function<Node *, bool> {
            |                                            ^~~~~~~~~~~~~~
      In file included from /usr/include/c++/12.2.0/string:48,
                       from udon2_cpp_src/ConllWriter.h:8:
      /usr/include/c++/12.2.0/bits/stl_function.h:117:12: note: declared here
        117 |     struct unary_function
            |            ^~~~~~~~~~~~~~
      gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -I/usr/include/python3.10 -I/var/lib/pacman/local/boost-python2-1.78.0-1/include -I/usr/include/python3.10 -c udon2_cpp_src/Importer.cpp -o build/temp.linux-x86_64-cpython-310/udon2_cpp_src/Importer.o
      In file included from udon2_cpp_src/Importer.cpp:5:
      udon2_cpp_src/Importer.h:9:10: fatal error: boost/python/list.hpp: No such file or directory
          9 | #include <boost/python/list.hpp>
            |          ^~~~~~~~~~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> udon2

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Expected behavior
Successful installation of udon2.

Environment specs (please complete the following information):

  • OS: Manjaro 22.0.0 Sikaris
  • Python version: 3.10.8
  • Udon verision: latest on pip

BUG

Describe the bug
Error: TypeError: write() argument must be str, not None
while trying to visualize the Tree

Screenshot from 2022-06-18 14-53-54

To Reproduce
import udon2 as ud2
from udon2.visual import render_dep_tree
roots = ud2.ConllReader.read_file('/home/greg/res1.conllu')
render_dep_tree(roots[0], '/home/greg/pict.svg')

I get the picture of the tree on the screen and the mentioned error.
Environment specs (please complete the following information):

  • OS: ubuntu 20.04
  • Python version: 3.9.12
  • UDon2 version:
  • Precompiled?: Yes if installed via pip

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.