Test Failing on Windows 10 (64 bit) and Python 64 Bit (3.6 & 3.7) about py-tree-sitter HOT 22 CLOSED

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.96. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

from py-tree-sitter.

Yeah, it sounds like something is not working on 64-bit Windows. I will try to look into this soon, but if anyone else wants to help debug it, it would be much appreciated!

from py-tree-sitter.

To verify, it really did work in 32-bit python for me either.

from py-tree-sitter.

I think this is fixed in #45.

from py-tree-sitter.

Maybe the extension is getting compiled in 32-bit mode even when using the 64-bit Python binary. In that case, the wrong type would be getting set here.

The compilation flags for the extension are set here, using the normal setuptools.Extension class. Maybe there's some flag missing there? I would have thought that it would pick the matching architecture by default.

from py-tree-sitter.

Thanks.
I did see that the 64 bit compiler was used while compilation and linking. I did not pay attention to flags though. Will check and try to put the lines on stdout (where compiler & linker are getting invoked)..
I had printed some values in python (init.py) & binding.c. That gave me the impression of having some address/value change (I am not very well conversant with bindings).. My suspect was something is expecting 32 bit address/value and 64 bit was provided resulting into overflow..

from py-tree-sitter.

Here is the stdout when I ran the tests.. It pasted the output where the language is getting compiled. There were some extra outputs I had added

You can see that it is picking up the compiler from x64 folder. Not sure if we have to add any flags..

####################################
running test
running egg_info
writing tree_sitter.egg-info\PKG-INFO
writing dependency_links to tree_sitter.egg-info\dependency_links.txt
writing top-level names to tree_sitter.egg-info\top_level.txt
reading manifest file 'tree_sitter.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'tree_sitter.egg-info\SOURCES.txt'
running build_ext
copying build\lib.win-amd64-3.7\tree_sitter\binding.cp37-win_amd64.pyd -> tree_sitter
PLATFORM----- win-amd64
List of available compilers:
--compiler=bcpp Borland C++ Compiler
--compiler=cygwin Cygwin port of GNU C Compiler for Win32
--compiler=mingw32 Mingw32 port of GNU C Compiler for Win32
--compiler=msvc Microsoft Visual C++
--compiler=unix standard UNIX-style compiler
creating C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests
creating C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures
creating C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-python
creating C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-python\src
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -Itests\fixtures\tree-sitter-python\src "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tctests\fixtures\tree-sitter-python\src\parser.c /FoC:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-python\src\parser.obj
parser.c
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -Itests\fixtures\tree-sitter-python\src "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /EHsc /Tptests\fixtures\tree-sitter-python\src\scanner.cc /FoC:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-python\src\scanner.obj
scanner.cc
tests\fixtures\tree-sitter-python\src\scanner.cc(104): warning C4267: '=': conversion from 'size_t' to 'char', possible loss of data
tests\fixtures\tree-sitter-python\src\scanner.cc(114): warning C4244: '=': conversion from 'unsigned short' to 'char', possible loss of data
tests\fixtures\tree-sitter-python\src\scanner.cc(117): warning C4267: 'return': conversion from 'size_t' to 'unsigned int', possible loss of data
creating C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-javascript
creating C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-javascript\src
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -Itests\fixtures\tree-sitter-javascript\src "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tctests\fixtures\tree-sitter-javascript\src\parser.c /FoC:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-javascript\src\parser.obj
parser.c
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -Itests\fixtures\tree-sitter-javascript\src "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tctests\fixtures\tree-sitter-javascript\src\scanner.c /FoC:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-javascript\src\scanner.obj
scanner.c
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /nodefaultlib:libucrt.lib ucrt.lib /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.23.28105\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64" C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-python\src\parser.obj C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-python\src\scanner.obj C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-javascript\src\parser.obj C:\Users\Anand\AppData\Local\Temp\tmpwvl8rfadtree_sitter_language\tests\fixtures\tree-sitter-javascript\src\scanner.obj /OUT:build\languages.so
Creating library build\languages.lib and object build\languages.exp
Generating code
Finished generating code

from py-tree-sitter.

Anybody has any inputs?

from py-tree-sitter.

Nope, still working in 32 bit. This is kind of a blocker for me as well though, as I want to release my package soon. Unfortunately I really don't have much proficiency on C, so I can't just come up with a fix :(

from py-tree-sitter.

Just a small update: I tried many variations of compile args for parser.c file and __init__ return types (as in here), played with tree_sitter_{language} function in parser.c and checked out the returned pointers, yet found no solution or a reasoning for the bug.

One thing I wasn't able to do is to compare the pointers returned by the tree_sitter_{language} function and the value registered to language_id, which should obviously be the same, and if that is the case then the only possible culprit is something wrong in the bindings.c file. Maybe a flag for the extension compilation can solve the issue, but I doubt it.

from py-tree-sitter.

I ran into the same issue. It was using MSVC, but I usually use GCC (I have [build] compiler=mingw32 in distutils.cfg, but I found while debugging that it's only used when building packages.)

Anyway, in my project I ended up "setting the compiler" with this monkey patch:

import tree_sitter
nc = tree_sitter.new_compiler
def new_compiler(plat=None, compiler=None, *args, **kwargs):
	if compiler is None:
		compiler = 'mingw32'
	return nc(plat, compiler, *args, **kwargs)
tree_sitter.new_compiler = new_compiler

It would be nice if treesitter got the preferred compiler from distutils.cfg and used it when calling new_compiler. Here's one way to do it:

def get_build_compiler_name() -> Optional[str]:
	from distutils import dist, ccompiler
	d = dist.Distribution()
	d.parse_config_files(d.find_config_files())
	b = d.get_command_obj('build_ext')
	b.ensure_finalized()
	return b.compiler

I can make a PR if you want.

from py-tree-sitter.

I ran into the same issue. It was using MSVC, but I usually use GCC (I have [build] compiler=mingw32 in distutils.cfg, but I found while debugging that it's only used when building packages.)

Anyway, in my project I ended up "setting the compiler" with this monkey patch:

import tree_sitter
nc = tree_sitter.new_compiler
def new_compiler(plat=None, compiler=None, *args, **kwargs):
	if compiler is None:
		compiler = 'mingw32'
	return nc(plat, compiler, *args, **kwargs)
tree_sitter.new_compiler = new_compiler

It would be nice if treesitter got the preferred compiler from distutils.cfg and used it when calling new_compiler. Here's one way to do it:

def get_build_compiler_name() -> Optional[str]:
	from distutils import dist, ccompiler
	d = dist.Distribution()
	d.parse_config_files(d.find_config_files())
	b = d.get_command_obj('build_ext')
	b.ensure_finalized()
	return b.compiler

I can make a PR if you want.

I have the same problem. It crashed in windows but works well in vmware ubuntu. I have mingw as well. But when I change the code to use mingw32, the ld failed, it cannot load msvcr140.

Is my env-variables missing something? Can you give me an advice? @valtron

D:/Program Files (x86)/Dev-Cpp/MinGW64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmsvcr140

from py-tree-sitter.

@zhangtianxiang Sorry, I don't have that problem so I can't say what causes it. But if you're willing to re-do your environment, I maintain this guide for the best way to set up windows to be more linux-like.

from py-tree-sitter.

@valtron Thanks for reply. I failed finally. It still crashing on my win10. It's not that easy that just using mingw32 and changing msvcr140 to vcruntime140. So I have to develop my program on ubuntu totally. It's sad that my package cannot run on windows simply. I didn't try MSYS2 yet, because not all users will have it. Maybe I will have a try on MSYS2 by follow your guide.

from py-tree-sitter.

I'm now doing this: Using Anaconda3. Set environment variable CONDA_FORCE_32BIT to 1 and create a new env. Then the new env will be 32bit.
This is not a long-term solution but can be a temporary convenience.

from py-tree-sitter.

I am getting this error on windows 10. Python 3.6 - 32 bit.

OSError                                   Traceback (most recent call last)
<ipython-input-2-48453f75ad4d> in <module>
     10 
     11 ##Load languages as language app
---> 12 GO_LANGUAGE = Language('build/my-languages.so', 'go')
     13 JS_LANGUAGE = Language('build/my-languages.so', 'javascript')
     14 PY_LANGUAGE = Language('build/my-languages.so', 'python')

C:\ProgramData\Anaconda3\envs\py36_32\lib\site-packages\tree_sitter\__init__.py in __init__(self, library_path, name)
     79         """
     80         self.name = name
---> 81         self.lib = cdll.LoadLibrary(library_path)
     82         language_function = getattr(self.lib, "tree_sitter_%s" % name)
     83         language_function.restype = c_void_p

C:\ProgramData\Anaconda3\envs\py36_32\lib\ctypes\__init__.py in LoadLibrary(self, name)
    424 
    425     def LoadLibrary(self, name):
--> 426         return self._dlltype(name)
    427 
    428 cdll = LibraryLoader(CDLL)

C:\ProgramData\Anaconda3\envs\py36_32\lib\ctypes\__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
    346 
    347         if handle is None:
--> 348             self._handle = _dlopen(self._name, mode)
    349         else:
    350             self._handle = handle

OSError: [WinError 193] %1 is not a valid Win32 application

What can I do to correct this?

from py-tree-sitter.

I was also able to verify that when the parser.c is compiled manually with MinGW-w64 (https://sourceforge.net/projects/mingw-w64) instead of MSVC, it works. The command I used to compile was:

gcc -fPIC -o parser.so -shared -I . -O3 -Wall parser.c -m64

from py-tree-sitter.

Did you succeed in compiling the different parsers (java, javascript, python) with minGW-w64 ? I do not. I got errors for the python and javascript parsers. And I still get the same problem when trying to set the parser to Java : parser.set_language(JAVA_LANGUAGE) in my python script.

Thks in advance

from py-tree-sitter.

I experienced the same issues with Python 3.8.5 on Windows 10.

ATM, using the default installer download of Python for Windows gets you 32-bit. That seems to work with MSVC and friends in yielding a usable py-tree-sitter.

No luck so far with the 64-bit version though for MSVC -- like others who have commented before.

For Python 3.8.5 (64-bit) I did see output similar to that mentioned in: #18 (comment) though the command that yielded some similar output for me was:

python setup.py build_ext --inplace

I'll put the full output at the end as it's a bit long.

I noticed strings like "possible loss of data" -- that doesn't sound good. @maxbrunsfeld is there something I can do to help investigate this?

Also, subsequent to this build, like others, trying to use set_language ends in the process dying. For reference, the code I tested with was:

from tree_sitter import Language, Parser

print("start")

Language.build_library(
  # Store the library in the `build` directory
  'build/my-languages.dll',

  # Include one or more languages
  [
    'vendor/tree-sitter-clojure'
  ]
)

print("before Language")

CLJ_LANGUAGE = Language("build/my-languages.dll", 'clojure')

print("before Parser")

parser = Parser()

print("before set_language")

parser.set_language(CLJ_LANGUAGE)

# never seems to get here

print("before parse")

tree = parser.parse(bytes("(def a 1)", "utf8"))

print("after parse")

root_node = tree.root_node

print(root_node.type)

print("end")

Output of python setup.py build_ext --inplace:

running build_ext
building 'tree_sitter.binding' extension
creating build\temp.win-amd64-3.8
creating build\temp.win-amd64-3.8\Release
creating build\temp.win-amd64-3.8\Release\tree_sitter
creating build\temp.win-amd64-3.8\Release\tree_sitter\core
creating build\temp.win-amd64-3.8\Release\tree_sitter\core\lib
creating build\temp.win-amd64-3.8\Release\tree_sitter\core\lib\src
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Itree_sitter/core/lib/include -Itree_sitter/core/lib/src -IC:\Users\user\AppData\Local\Programs\Python\Python38\include -IC:\Users\user\AppData\Local\Programs\Python\Python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" /Tctree_sitter/core/lib/src/lib.c /Fobuild\temp.win-amd64-3.8\Release\tree_sitter/core/lib/src/lib.obj
lib.c
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./alloc.h(47): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./alloc.h(47): note: consider using '%zu' in the format string
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./alloc.h(56): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./alloc.h(56): note: consider using '%zu' in the format string
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./alloc.h(65): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./alloc.h(65): note: consider using '%zu' in the format string
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./array.h(114): warning C4267: 'function': conversion from 'size_t' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./lexer.c(376): warning C4267: '=': conversion from 'size_t' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./parser.c(567): warning C4267: '=': conversion from 'size_t' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./parser.c(1738): warning C4996: 'fdopen': The POSIX name for this item is deprecated. Instead, use the ISO C and C++ conformant name: _fdopen. See online help for details.
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\stdio.h(2457): note: see declaration of 'fdopen'
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./parser.c(1812): warning C4267: 'function': conversion from 'size_t' to 'unsigned int', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./parser.c(1885): warning C4267: 'function': conversion from 'size_t' to 'unsigned int', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(157): warning C4244: 'function': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(450): warning C4244: 'initializing': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(482): warning C4244: 'initializing': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(504): warning C4244: 'initializing': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(583): warning C4244: 'initializing': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(632): warning C4244: 'initializing': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(662): warning C4244: 'initializing': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(719): warning C4244: 'initializing': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(791): warning C4244: '=': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./query.c(802): warning C4244: '=': conversion from '__int64' to 'uint32_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./subtree.c(175): warning C4244: 'initializing': conversion from 'TSSymbol' to 'uint8_t', possible loss of data
c:\users\user\desktop\py-tree-sitter\tree_sitter\core\lib\src\./subtree.c(225): warning C4244: '=': conversion from 'TSSymbol' to 'uint8_t', possible loss of data
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Itree_sitter/core/lib/include -Itree_sitter/core/lib/src -IC:\Users\user\AppData\Local\Programs\Python\Python38\include -IC:\Users\user\AppData\Local\Programs\Python\Python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" /Tctree_sitter/binding.c /Fobuild\temp.win-amd64-3.8\Release\tree_sitter/binding.obj
binding.c
tree_sitter/binding.c(126): warning C4244: 'function': conversion from 'Py_ssize_t' to 'uint32_t', possible loss of data
tree_sitter/binding.c(524): warning C4267: 'function': conversion from 'size_t' to 'uint32_t', possible loss of data
tree_sitter/binding.c(546): warning C4312: 'type cast': conversion from 'long' to 'TSLanguage *' of greater size
tree_sitter/binding.c(746): warning C4244: 'function': conversion from 'Py_ssize_t' to 'uint32_t', possible loss of data
tree_sitter/binding.c(762): warning C4244: 'function': conversion from 'Py_ssize_t' to 'int', possible loss of data
creating C:\Users\user\Desktop\py-tree-sitter\build\lib.win-amd64-3.8
creating C:\Users\user\Desktop\py-tree-sitter\build\lib.win-amd64-3.8\tree_sitter
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\user\AppData\Local\Programs\Python\Python38\libs /LIBPATH:C:\Users\user\AppData\Local\Programs\Python\Python38\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64" /EXPORT:PyInit_binding build\temp.win-amd64-3.8\Release\tree_sitter/core/lib/src/lib.obj build\temp.win-amd64-3.8\Release\tree_sitter/binding.obj /OUT:build\lib.win-amd64-3.8\tree_sitter\binding.cp38-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.8\Release\tree_sitter/core/lib/src\binding.cp38-win_amd64.lib
   Creating library build\temp.win-amd64-3.8\Release\tree_sitter/core/lib/src\binding.cp38-win_amd64.lib and object build\temp.win-amd64-3.8\Release\tree_sitter/core/lib/src\binding.cp38-win_amd64.exp
Generating code
Finished generating code
copying build\lib.win-amd64-3.8\tree_sitter\binding.cp38-win_amd64.pyd -> tree_sitter

from py-tree-sitter.

Has anybody got any lead on how to parse on Windows? I too experienced the same issue as @sogaiu i.e. python abruptly exits after parser.set_language(py_lang) command. Thanks!

from py-tree-sitter.

May not be what you want to hear, but I think if you use the 32-bit Python it might work.

from py-tree-sitter.

There may be similar problems on other operating system, as #41 mentioned.

I think the core problem is that the compile environment used here to build python bindings and tree-sitter core

I'm not good at playing with compliers, but I think there is a easier way to avoid that: just collect every repo from github for tree-sitter language (as submodule, like https://github.com/mozilla/rust-code-analysis
does), then build everything in the setup.py stage (maybe some extra boilerplate here?). We could even get rid of the build_library and set_language(), thus simplify the usage of py-tree-sitter.

I think it is worthwhile to have a try. The only thing is that the package size may grow to an unacceptable size (maybe not).

And, the bonus part, pre-built libraries are also feasible to provide if we build everything at the very begining. That would be super convenient for Windows users.

from py-tree-sitter.