thalesbertaglia / enelvo Goto Github PK

A flexible normalizer for user-generated content

Home Page: http://thalesbertaglia.com/enelvo

License: MIT License

Python 99.79% Makefile 0.21%

normalization normalization-data brazilian-portuguese user-generated-content normalisation brazilianportuguese natural-language-processing machine-learning nlp

enelvo's People

Contributors

Stargazers

Watchers

Forkers

franksantos1 fabiobif staticdev janes gazzola ferroeduardo

enelvo's Issues

Erro normalização na nova versão

Olá Thales,

Acabei de baixar o código atual do enelvo e começou a dar erro (estava funcionando normalmente). Veja abaixo:

Error setup.py

Foi tudo instalado corretamente, porem quando roda o setup.py acontece esse erro:
$ python3 setup.py install
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

[Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/test-easy-install-10765.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

/usr/local/lib/python3.6/dist-packages/

Tentei com sudo e deu o seguinte erro:

error: urllib3 1.25.2 is installed but urllib3<1.25,>=1.20; python_version >= "3.4" is required by {'botocore'}

Erro ao instalar a biblioteca via `pip install`

Utilizando Python 3.10.4, basta rodar o comando:
pip install enelvo

Obtive o seguinte traceback em mais de uma máquina:

pip install enelvo
Collecting enelvo
  Using cached enelvo-0.12-py3-none-any.whl (27.5 MB)
Collecting tabulate==0.8.9
  Downloading tabulate-0.8.9-py3-none-any.whl (25 kB)
Collecting numpy==1.19.5
  Using cached numpy-1.19.5.zip (7.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [227 lines of output]
      setup.py:67: RuntimeWarning: NumPy 1.19.5 may not yet support Python 3.10.
        warnings.warn(
      Running from numpy source directory.
      setup.py:480: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
        run_build = parse_setuppy_commands()
      Processing numpy/random\_bounded_integers.pxd.in
      Processing numpy/random\bit_generator.pyx
      Processing numpy/random\mtrand.pyx
      Processing numpy/random\_bounded_integers.pyx.in
      Processing numpy/random\_common.pyx
      Processing numpy/random\_generator.pyx
      Processing numpy/random\_mt19937.pyx
      Processing numpy/random\_pcg64.pyx
      Processing numpy/random\_philox.pyx
      Processing numpy/random\_sfc64.pyx
      Cythonizing sources
      blas_opt_info:
      blas_mkl_info:
      No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
      customize MSVCCompiler
        libraries mkl_rt not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      blis_info:
        libraries blis not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      openblas_info:
        libraries openblas not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
      get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
      customize GnuFCompiler
      Could not locate executable g77
      Could not locate executable f77
      customize IntelVisualFCompiler
      Could not locate executable ifort
      Could not locate executable ifl
      customize AbsoftFCompiler
      Could not locate executable f90
      customize CompaqVisualFCompiler
      Could not locate executable DF
      customize IntelItaniumVisualFCompiler
      Could not locate executable efl
      customize Gnu95FCompiler
      Found executable C:\Strawberry\c\bin\gfortran.exe
      Using built-in specs.
      COLLECT_GCC=C:\Strawberry\c\bin\gfortran.exe
      COLLECT_LTO_WRAPPER=C:/Strawberry/c/bin/../libexec/gcc/x86_64-w64-mingw32/8.3.0/lto-wrapper.exe
      Target: x86_64-w64-mingw32
      Configured with: ../../../src/gcc-8.3.0/configure --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --prefix=/mingw64 --enable-shared --enable-static --disable-multilib --enable-languages=c,c++,fortran,lto --enable-libstdcxx-time=yes --enable-threads=posix --enable-libgomp --enable-libatomic --enable-lto --enable-graphite --enable-checking=release --enable-fully-dynamic-string --enable-version-specific-runtime-libs --enable-libstdcxx-filesystem-ts=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-bootstrap --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-gnu-as --with-gnu-ld --with-arch=nocona --with-tune=core2 --with-libiconv --with-system-zlib --with-gmp=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-mpfr=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-mpc=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-isl=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-pkgversion='x86_64-posix-seh, Built by strawberryperl.com project' --with-bugurl=https://sourceforge.net/projects/mingw-w64 CFLAGS='-O2 -pipe -fno-ident -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' CXXFLAGS='-O2 -pipe -fno-ident -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' CPPFLAGS=' -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' LDFLAGS='-pipe -fno-ident -L/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/lib -L/opt/build/prerequisites/x86_64-zlib-static/lib -L/opt/build/prerequisites/x86_64-w64-mingw32-static/lib ' LD_FOR_TARGET=/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/bin/ld.exe
      Thread model: posix
      gcc version 8.3.0 (x86_64-posix-seh, Built by strawberryperl.com project)
        NOT AVAILABLE

      atlas_3_10_blas_threads_info:
      Setting PTATLAS=ATLAS
        libraries tatlas not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      atlas_3_10_blas_info:
        libraries satlas not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      atlas_blas_threads_info:
      Setting PTATLAS=ATLAS
        libraries ptf77blas,ptcblas,atlas not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      atlas_blas_info:
        libraries f77blas,cblas,atlas not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      accelerate_info:
        NOT AVAILABLE

      C:\Users\rodri\AppData\Local\Temp\pip-install-mda3xeph\numpy_772f7a47b7514d1d9082934a9c70bfe4\numpy\distutils\system_info.py:1914: UserWarning:
          Optimized (vendor) Blas libraries are not found.
          Falls back to netlib Blas library which has worse performance.
          A better performance should be easily gained by switching
          Blas library.
        if self._calc_info(blas):
      blas_info:
        libraries blas not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      C:\Users\rodri\AppData\Local\Temp\pip-install-mda3xeph\numpy_772f7a47b7514d1d9082934a9c70bfe4\numpy\distutils\system_info.py:1914: UserWarning:
          Blas (http://www.netlib.org/blas/) libraries not found.
          Directories to search for the libraries can be specified in the
          numpy/distutils/site.cfg file (section [blas]) or by setting
          the BLAS environment variable.
        if self._calc_info(blas):
      blas_src_info:
        NOT AVAILABLE

      C:\Users\rodri\AppData\Local\Temp\pip-install-mda3xeph\numpy_772f7a47b7514d1d9082934a9c70bfe4\numpy\distutils\system_info.py:1914: UserWarning:
          Blas (http://www.netlib.org/blas/) sources not found.
          Directories to search for the sources can be specified in the
          numpy/distutils/site.cfg file (section [blas_src]) or by setting
          the BLAS_SRC environment variable.
        if self._calc_info(blas):
        NOT AVAILABLE

      non-existing path in 'numpy\\distutils': 'site.cfg'
      lapack_opt_info:
      lapack_mkl_info:
        libraries mkl_rt not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      openblas_lapack_info:
        libraries openblas not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
      get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
      customize GnuFCompiler
      customize IntelVisualFCompiler
      customize AbsoftFCompiler
      customize CompaqVisualFCompiler
      customize IntelItaniumVisualFCompiler
      customize Gnu95FCompiler
      Using built-in specs.
      COLLECT_GCC=C:\Strawberry\c\bin\gfortran.exe
      COLLECT_LTO_WRAPPER=C:/Strawberry/c/bin/../libexec/gcc/x86_64-w64-mingw32/8.3.0/lto-wrapper.exe
      Target: x86_64-w64-mingw32
      Configured with: ../../../src/gcc-8.3.0/configure --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --prefix=/mingw64 --enable-shared --enable-static --disable-multilib --enable-languages=c,c++,fortran,lto --enable-libstdcxx-time=yes --enable-threads=posix --enable-libgomp --enable-libatomic --enable-lto --enable-graphite --enable-checking=release --enable-fully-dynamic-string --enable-version-specific-runtime-libs --enable-libstdcxx-filesystem-ts=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-bootstrap --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-gnu-as --with-gnu-ld --with-arch=nocona --with-tune=core2 --with-libiconv --with-system-zlib --with-gmp=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-mpfr=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-mpc=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-isl=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-pkgversion='x86_64-posix-seh, Built by strawberryperl.com project' --with-bugurl=https://sourceforge.net/projects/mingw-w64 CFLAGS='-O2 -pipe -fno-ident -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' CXXFLAGS='-O2 -pipe -fno-ident -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' CPPFLAGS=' -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' LDFLAGS='-pipe -fno-ident -L/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/lib -L/opt/build/prerequisites/x86_64-zlib-static/lib -L/opt/build/prerequisites/x86_64-w64-mingw32-static/lib ' LD_FOR_TARGET=/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/bin/ld.exe
      Thread model: posix
      gcc version 8.3.0 (x86_64-posix-seh, Built by strawberryperl.com project)
        NOT AVAILABLE

      openblas_clapack_info:
        libraries openblas,lapack not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
      get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
      customize GnuFCompiler
      customize IntelVisualFCompiler
      customize AbsoftFCompiler
      customize CompaqVisualFCompiler
      customize IntelItaniumVisualFCompiler
      customize Gnu95FCompiler
      Using built-in specs.
      COLLECT_GCC=C:\Strawberry\c\bin\gfortran.exe
      COLLECT_LTO_WRAPPER=C:/Strawberry/c/bin/../libexec/gcc/x86_64-w64-mingw32/8.3.0/lto-wrapper.exe
      Target: x86_64-w64-mingw32
      Configured with: ../../../src/gcc-8.3.0/configure --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --prefix=/mingw64 --enable-shared --enable-static --disable-multilib --enable-languages=c,c++,fortran,lto --enable-libstdcxx-time=yes --enable-threads=posix --enable-libgomp --enable-libatomic --enable-lto --enable-graphite --enable-checking=release --enable-fully-dynamic-string --enable-version-specific-runtime-libs --enable-libstdcxx-filesystem-ts=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-bootstrap --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-gnu-as --with-gnu-ld --with-arch=nocona --with-tune=core2 --with-libiconv --with-system-zlib --with-gmp=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-mpfr=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-mpc=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-isl=/opt/build/prerequisites/x86_64-w64-mingw32-static --with-pkgversion='x86_64-posix-seh, Built by strawberryperl.com project' --with-bugurl=https://sourceforge.net/projects/mingw-w64 CFLAGS='-O2 -pipe -fno-ident -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' CXXFLAGS='-O2 -pipe -fno-ident -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' CPPFLAGS=' -I/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/include -I/opt/build/prerequisites/x86_64-zlib-static/include -I/opt/build/prerequisites/x86_64-w64-mingw32-static/include' LDFLAGS='-pipe -fno-ident -L/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/opt/lib -L/opt/build/prerequisites/x86_64-zlib-static/lib -L/opt/build/prerequisites/x86_64-w64-mingw32-static/lib ' LD_FOR_TARGET=/opt/build/x86_64-830-posix-seh-rt_v6/mingw64/bin/ld.exe
      Thread model: posix
      gcc version 8.3.0 (x86_64-posix-seh, Built by strawberryperl.com project)
        NOT AVAILABLE

      flame_info:
        libraries flame not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      atlas_3_10_threads_info:
      Setting PTATLAS=ATLAS
        libraries lapack_atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\lib
        libraries tatlas,tatlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\lib
        libraries lapack_atlas not found in C:\
        libraries tatlas,tatlas not found in C:\
        libraries lapack_atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\libs
        libraries tatlas,tatlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\libs
      <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
        NOT AVAILABLE

      atlas_3_10_info:
        libraries lapack_atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\lib
        libraries satlas,satlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\lib
        libraries lapack_atlas not found in C:\
        libraries satlas,satlas not found in C:\
        libraries lapack_atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\libs
        libraries satlas,satlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\libs
      <class 'numpy.distutils.system_info.atlas_3_10_info'>
        NOT AVAILABLE

      atlas_threads_info:
      Setting PTATLAS=ATLAS
        libraries lapack_atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\lib
        libraries ptf77blas,ptcblas,atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\lib
        libraries lapack_atlas not found in C:\
        libraries ptf77blas,ptcblas,atlas not found in C:\
        libraries lapack_atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\libs
        libraries ptf77blas,ptcblas,atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\libs
      <class 'numpy.distutils.system_info.atlas_threads_info'>
        NOT AVAILABLE

      atlas_info:
        libraries lapack_atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\lib
        libraries f77blas,cblas,atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\lib
        libraries lapack_atlas not found in C:\
        libraries f77blas,cblas,atlas not found in C:\
        libraries lapack_atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\libs
        libraries f77blas,cblas,atlas not found in C:\Users\rodri\AppData\Local\Programs\Python\Python310\libs
      <class 'numpy.distutils.system_info.atlas_info'>
        NOT AVAILABLE

      lapack_info:
        libraries lapack not found in ['C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\rodri\\AppData\\Local\\Programs\\Python\\Python310\\libs']
        NOT AVAILABLE

      C:\Users\rodri\AppData\Local\Temp\pip-install-mda3xeph\numpy_772f7a47b7514d1d9082934a9c70bfe4\numpy\distutils\system_info.py:1748: UserWarning:
          Lapack (http://www.netlib.org/lapack/) libraries not found.
          Directories to search for the libraries can be specified in the
          numpy/distutils/site.cfg file (section [lapack]) or by setting
          the LAPACK environment variable.
        return getattr(self, '_calc_info_{}'.format(name))()
      lapack_src_info:
        NOT AVAILABLE

      C:\Users\rodri\AppData\Local\Temp\pip-install-mda3xeph\numpy_772f7a47b7514d1d9082934a9c70bfe4\numpy\distutils\system_info.py:1748: UserWarning:
          Lapack (http://www.netlib.org/lapack/) sources not found.
          Directories to search for the sources can be specified in the
          numpy/distutils/site.cfg file (section [lapack_src]) or by setting
          the LAPACK_SRC environment variable.
        return getattr(self, '_calc_info_{}'.format(name))()
        NOT AVAILABLE

      numpy_linalg_lapack_lite:
        FOUND:
          language = c
          define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]

      C:\Users\rodri\AppData\Local\Temp\pip-build-env-gsp1d34l\overlay\Lib\site-packages\setuptools\_distutils\dist.py:275: UserWarning: Unknown distribution option: 'define_macros'
        warnings.warn(msg)
      running dist_info
      running build_src
      build_src
      building py_modules sources
      creating build
      creating build\src.win-amd64-3.10
      creating build\src.win-amd64-3.10\numpy
      creating build\src.win-amd64-3.10\numpy\distutils
      building library "npymath" sources
      error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Erro no setup.py

Em algumas máquinas o setup.py está dando o seguinte erro:

error in Enelvo setup command: 'python_requires' must be a string containing valid version specifiers; Invalid specifier: '~=3'

Tenho quase certeza que é incompatibilidade com o distutils. Vou tentar corrigir aqui.

Normalização de aniversário(s) e história(s)

Olá!

Estou com um problema porque quando uma pessoa escreve aniversario ou aniversários sem acento, ele não está colocando o acento. O mesmo acontece com história. Teria como acrescentar essas palavras?

pp_line variable error on force_list corrections

Prezado

ao tentar rodar o programa utilizando uma lista de correção forçada, o programa levantou um erro na linha 124 do arquivo enelvo.normiliser.py

o método normalise da classe Normaliser (na linha 124) insere o objeto pp_line como sendo self.pp_line, porém, nenhum attributo pp_line foi setado na classe Normaliser.

testando aqui, apenas a remoção do "self." funcionou bem

Agradeço a atenção

Erro normalização de palavras estranhas

Boa noite Thales.. com o uso mais intenso do seu normalizador, detectei um caso atípico em que você levanta um normalizador e ele gera uma exceção não tratada, para reproduzir utilize o código:

from enelvo.normaliser import Normaliser

norm = Normaliser(tokenizer='readable')
message = 'aosdfiosdmfaoi'
reply = norm.normalise(message)

Traceback:

09/01/2019 22:04:54 mensagem: aosdfiosdmfaoi
09/01/2019 22:04:54 192.168.208.13 - - [09/Jan/2019 22:04:54] "POST /reply HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/app/__init__.py", line 61, in reply
    reply = norm.normalise(message)
  File "/app/enelvo/normaliser.py", line 107, in normalise
    self.logger.error(
  AttributeError

Abraços

Thiago

Problema com o módulo emoji

Ao fazer o import from enelvo.normaliser import Normaliser, esse erro é apresentado:

ImportError: cannot import name 'UNICODE_EMOJI' from 'emoji' (/home/eduar/.local/lib/python3.10/site-packages/emoji/__init__.py)

Imagino que seja devido ao módulo emoji que está na versão 2.2.0. Instalar a versão 1.6.3 resolveu o problema.

Estou usando o Python 3.10.6+WSL e enelvo 0.13

"The urllib3 library before 1.24.2 for Python mishandles certain cases where the desired set of CA certificates is different from the OS store of CA certificates, which results in SSL connections succeeding in situations where a verification failure is the correct outcome. This is related to use of the ssl_context, ca_certs, or ca_certs_dir argument."

Lembrar de arrumar

Dependência desnecessária importlib_metadata

Essa dependência pode ser removida. Ref: https://hynek.me/articles/packaging-metadata/

Criar pacote PyPI

Sugiro usar o Poetry e criar um pyproject.toml (ele veio com as PEP 517/518) para substituir o setup.py, MANIFEST, setup.cfg e requirements.txt).
Alterar travis para usar poetry.

Teste Cython

Olá Thales,

Já que vai fazer commits, poderia aproveitar e trocar o setup.py e o requirements para usar o Cython-0.29.7? Vou tentar gerar uma imagem pra microservicos menor usando o alpine com um comando que descobri aqui:

docker-library/python#312

Aí aproveito e faço uma versão nova tb.

Abraços

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.