pip install JapaneseTokenizer
make install
make install_neologd

install_tokenizers.sh: line 89: ldconfig: command not found

[install-mecab-ipadic-NEologd] :     unxz is not found.
make: *** [install_neologd] Error 1

[Y/12/06 15:03:43]ERROR - mecab_wrapper.py#__CallMecab:137: ('',)
[Y/12/06 15:03:43]ERROR - mecab_wrapper.py#__CallMecab:138: Possibly Path to userdict is invalid. Check the path
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    mecab_wrapper = JapaneseTokenizer.MecabWrapper(dictType='ipadic')
  File "/Users/rpryzant/kana/venv/lib/python2.7/site-packages/JapaneseTokenizer/mecab_wrapper/mecab_wrapper.py", line 45, in __init__
    self.mecabObj = self.__CallMecab()
  File "/Users/rpryzant/kana/venv/lib/python2.7/site-packages/JapaneseTokenizer/mecab_wrapper/mecab_wrapper.py", line 139, in __CallMecab
    raise subprocess.CalledProcessError(returncode=-1, cmd="Failed to initialize Mecab object")
subprocess.CalledProcessError: Command 'Failed to initialize Mecab object' returned non-zero exit status -1

    result = self.juman.analysis(input_str)
  File "/usr/local/lib/python2.7/dist-packages/pyknp/juman/juman.py", line 128, in analysis
    return self.juman(input_str)
  File "/usr/local/lib/python2.7/dist-packages/pyknp/juman/juman.py", line 121, in juman
    result = MList(self.juman_lines(input_str))
  File "/usr/local/lib/python2.7/dist-packages/pyknp/juman/juman.py", line 116, in juman_lines
    return self.socket.query(input_str, pattern=self.pattern)
  File "/usr/local/lib/python2.7/dist-packages/pyknp/juman/juman.py", line 41, in query
    return recv.strip().decode('utf-8')
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 0-1: invalid continuation byte

Exception: You could not call neologd dictionary bacause you do NOT install the package neologdn.

Exception: You could not call neologd dictionary bacause you do NOT install the package neologdn. run pip install neologdn

Traceback (most recent call last):
  File "generate_theme1.py", line 3, in <module>
    from JapaneseTokenizer import MecabWrapper
  File "/share/data/home/kensuke_mitsuzawa/outsource-ds-py-company-review/conda-env/lib/python3.5/site-packages/JapaneseTokenizer/__init__.py", line 2, in <module>
    from JapaneseTokenizer.juman_wrapper import JumanWrapper
  File "/share/data/home/kensuke_mitsuzawa/outsource-ds-py-company-review/conda-env/lib/python3.5/site-packages/JapaneseTokenizer/juman_wrapper/__init__.py", line 2, in <module>
    from .juman_wrapper import JumanWrapper
  File "/share/data/home/kensuke_mitsuzawa/outsource-ds-py-company-review/conda-env/lib/python3.5/site-packages/JapaneseTokenizer/juman_wrapper/juman_wrapper.py", line 11, in <module>
    from pyknp import MList
ImportError: No module named 'pyknp'

Installed /opt/conda/lib/python3.5/site-packages/kytea-0.1.3-py3.5-linux-x86_64.egg
Searching for pyknp
Reading https://pypi.python.org/simple/pyknp/
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Couldn't find index page for 'pyknp' (maybe misspelled?)
No local packages or working download links found for pyknp
error: Could not find suitable distribution for Requirement.parse('pyknp')

In [4]: input_sentence = '10日放送の「中居正広のミになる図書館」（テレビ朝日系）で、SMAPの中居正広が、篠原信一の過去の勘違いを明かす一幕があった。'
In [17]: mecab_wrapper.tokenize(input_sentence).filter(stopwords=['SMAP']).convert_list_object()
Out[17]: 
['1',
 '0',
 '日',
 '放送',
 'の',
 '「',
 '中居',
 '正広',
 'の',
 'ミ',
 'に',
 'なる',
 '図書館',
 '」',
 '(',
 'テレビ朝日',
 '系',
 ')',
 'で',
 '、',
 'SMAP',
 'の',
 '中居',
 '正広',
 'が',
 '、',
 '篠原',
 '信一',
 'の',
 '過去',
 'の',
 '勘違い',
 'を',
 '明かす',
 '一幕',
 'が',
 'ある',
 'た',
 '。']

Best match: neologdn 0.2.1
Processing neologdn-0.2.1.tar.gz
Writing /tmp/easy_install-mdYouk/neologdn-0.2.1/setup.cfg
Running neologdn-0.2.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-mdYouk/neologdn-0.2.1/egg-dist-tmp-9BvPzi
cc1plus: 警告: コマンドラインオプション ‘-Wstrict-prototypes’ は Ada/C/ObjC 用としては有効ですが、C++ 用としては有効ではありません [デフォルトで有効]
cc1plus: エラー: 認識できないコマンドラインオプション ‘-std=c++11’ です
error: Setup script exited with error: command 'gcc' failed with exit status 1

    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/kensuke-mi/.pyenv/versions/anaconda3-5.3.1/include -arch x86_64 -I/Users/kensuke-mi/.pyenv/versions/anaconda3-5.3.1/include -arch x86_64 -I/usr/local/Cellar/mecab/0.996/include -I/Users/kensuke-mi/.pyenv/versions/anaconda3-5.3.1/include/python3.7m -c MeCab_wrap.cpp -o build/temp.macosx-10.7-x86_64-3.7/MeCab_wrap.o
    warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
    MeCab_wrap.cpp:3051:10: fatal error: 'stdexcept' file not found
    #include <stdexcept>
             ^~~~~~~~~~~
    1 warning and 1 error generated.
    error: command 'gcc' failed with exit status 1

MeCab_wrap.cxx:8434:80: error: ‘MECAB_ONE_BEST’ was not declared in this scope
MeCab_wrap.cxx:8435:77: error: ‘MECAB_NBEST’ was not declared in this scope
MeCab_wrap.cxx:8436:79: error: ‘MECAB_PARTIAL’ was not declared in this scope
MeCab_wrap.cxx:8437:85: error: ‘MECAB_MARGINAL_PROB’ was not declared in this scope
MeCab_wrap.cxx:8438:83: error: ‘MECAB_ALTERNATIVE’ was not declared in this scope
MeCab_wrap.cxx:8439:82: error: ‘MECAB_ALL_MORPHS’ was not declared in this scope
MeCab_wrap.cxx:8440:89: error: ‘MECAB_ALLOCATE_SENTENCE’ was not declared in this scope
MeCab_wrap.cxx:8441:84: error: ‘MECAB_ANY_BOUNDARY’ was not declared in this scope
MeCab_wrap.cxx:8442:86: error: ‘MECAB_TOKEN_BOUNDARY’ was not declared in this scope
MeCab_wrap.cxx:8443:84: error: ‘MECAB_INSIDE_TOKEN’ was not declared in this scope
error: Setup script exited with error: command 'gcc' failed with exit status 1

[Y/09/29 16:54:36]WARNING - jumanpp_wrapper.py#call_juman_interface:197: Re-starting unix process because it tak
es longer time than 30 seconds...
[Y/09/29 16:55:06]WARNING - jumanpp_wrapper.py#call_juman_interface:197: Re-starting unix process because it tak
es longer time than 30 seconds...

Traceback (most recent call last):
  File "/share/data/home/kensuke_mitsuzawa/fuman-ds-py-academic-service/conda-env/lib/python3.5/site-packages/p$
xpect-4.2.1-py3.5.egg/pexpect/spawnbase.py", line 150, in read_nonblocking
    s = os.read(self.child_fd, size)
OSError: [Errno 5] Input/output error

Traceback (most recent call last):
  File "/Users/kensuke-mi/Desktop/analysis_work/fuman-ds-py-fuman2vector/job_scripts/train_word2vec_jumanpp.py", line 55, in <module>
    port=config_obj.get('Tokenizer', 'jumanpp_port'))
  File "/Users/kensuke-mi/.pyenv/versions/anaconda3-4.0.0/lib/python3.5/site-packages/JapaneseTokenizer/jumanpp_wrapper/jumanpp_wrapper_python3.py", line 73, in __init__
    self.jumanpp_obj = JumanppClient(hostname=server, port=port, timeout=timeout)
  File "/Users/kensuke-mi/.pyenv/versions/anaconda3-4.0.0/lib/python3.5/site-packages/JapaneseTokenizer/jumanpp_wrapper/jumanpp_wrapper_python3.py", line 30, in __init__
    self.sock.connect((hostname, port))
TypeError: an integer is required (got type str)

    Complete output from command /Users/kensuke-mi/.pyenv/versions/anaconda3-4.0.0/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/sp/z0_0lktj7nn2s31db2dt5md40000gq/T/pip-build-_kqnspts/neologdn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/sp/z0_0lktj7nn2s31db2dt5md40000gq/T/pip-hylasjm2-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'neologdn' extension
    creating build
    creating build/temp.macosx-10.6-x86_64-3.5
    /usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/kensuke-mi/.pyenv/versions/anaconda3-4.0.0/include -arch x86_64 -I/Users/kensuke-mi/.pyenv/versions/anaconda3-4.0.0/include/python3.5m -c neologdn.cpp -o build/temp.macosx-10.6-x86_64-3.5/neologdn.o -std=c++11
    neologdn.cpp:255:10: fatal error: 'unordered_map' file not found
    #include <unordered_map>
             ^
    1 error generated.
    error: command '/usr/bin/clang' failed with exit status 1

MacBook-Pro% pip install JapaneseTokenizer
Requirement already satisfied (use --upgrade to upgrade): JapaneseTokenizer in /Users/kensuke-mi/Desktop/analysis_work/python_morphology_splitters
Requirement already satisfied (use --upgrade to upgrade): future in /Users/kensuke-mi/.pyenv/versions/3.5.1/lib/python3.5/site-packages/future-0.15.2-py3.5.egg (from JapaneseTokenizer)
Requirement already satisfied (use --upgrade to upgrade): six in /Users/kensuke-mi/.pyenv/versions/3.5.1/lib/python3.5/site-packages (from JapaneseTokenizer)
Collecting mecab-python (from JapaneseTokenizer)
  Downloading mecab-python-0.996.tar.gz (40kB)
    100% |████████████████████████████████| 40kB 6.4MB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/private/var/folders/nq/13lcpk354h51bgkmx4q4ttr00000gp/T/pip-build-kd5ixe8l/mecab-python/setup.py", line 18, in <module>
        include_dirs=cmd2("mecab-config --inc-dir"),
      File "/private/var/folders/nq/13lcpk354h51bgkmx4q4ttr00000gp/T/pip-build-kd5ixe8l/mecab-python/setup.py", line 10, in cmd2
        return string.split (cmd1(str))
    AttributeError: module 'string' has no attribute 'split'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/nq/13lcpk354h51bgkmx4q4ttr00000gp/T/pip-build-kd5ixe8l/mecab-python
You are using pip version 7.1.2, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

checking for boostlib >= 1.57... configure: We could not detect the boost libraries (version 1.57 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.