Comments (11)
I don't have Anaconda installed and I encountered the same issue so I don't think it's an anaconda-specific problem. Not a maintainer, but I think creating a PR would help resolve the problem for a fair few others as well (including me!)
Edit: Managed to fix it on my side. For anyone else experiencing similar issues,
navigate it to:
https://github.com/tree-sitter/py-tree-sitter/blob/master/tree_sitter/__init__.py#L46
Change:
if cpp:
if find_library("c++"):
compiler.add_library("c++")
elif find_library("stdc++"):
compiler.add_library("stdc++")
to
compiler.add_library("stdc++")
and then install it again. (Alternatively - just edit it inside your environment for installed packages)
from py-tree-sitter.
FWIW, I think there might also be some relevant info in this issue: #41
from py-tree-sitter.
Has anyone come up with a general solution to this that could be incorporated into the library? It would be great if we could somehow detect what version of the C++ standard library the current Python interpreter is linked against (if any).
from py-tree-sitter.
I believe the problem is caused by the fact that distutils
uses cc
as the driver to link the final shared object, so the C++ standard library (libstdc++
or libc++
, depending on the platform) is not linked automatically. If c++
were used, the compiler should automatically select the correct (default) STL for the current platform.
I’m not sure if this is the proper API to change the compiler executable in distutils
, but this (somewhat hacky?) patch fixes the problem for me:
diff --git a/tree_sitter/__init__.py b/tree_sitter/__init__.py
index cc94ac9..b2e23d4 100644
--- a/tree_sitter/__init__.py
+++ b/tree_sitter/__init__.py
@@ -3,6 +3,7 @@
from ctypes import cdll, c_void_p
from ctypes.util import find_library
from distutils.ccompiler import new_compiler
+from distutils.unixccompiler import UnixCCompiler
from os import path
from platform import system
from tempfile import TemporaryDirectory
@@ -43,11 +44,8 @@ class Language:
]
compiler = new_compiler()
- if cpp:
- if find_library("c++"):
- compiler.add_library("c++")
- elif find_library("stdc++"):
- compiler.add_library("stdc++")
+ if isinstance(compiler, UnixCCompiler):
+ compiler.compiler_cxx[0] = "c++"
if max(source_mtimes) <= output_mtime:
return False
@@ -69,7 +67,11 @@ class Language:
extra_preargs=flags,
)[0]
)
- compiler.link_shared_object(object_paths, output_path)
+ compiler.link_shared_object(
+ object_paths,
+ output_path,
+ target_lang="c++" if cpp else "c",
+ )
return True
def __init__(self, library_path, name):
This should let the C++ compiler link the same STL as is expected by scanner.cc
. I don’t have access to a Mac so I only tested this on Linux.
from py-tree-sitter.
If anyone has the time and ability to thoroughly test this (or any other) approach, I'd definitely be open to merging it. I think it's fine as long as it can be shown to work on macOS, some common linux distro, and windows.
from py-tree-sitter.
@maxbrunsfeld What would you consider a thorough test for this? Would passing the test suite be enough? As one of the tests uses tree-sitter-python
(which has a scanner.cc
file), compiling and loading the resulting tree sitter library should be sufficient to find out if the correct STL was linked.
Because the py-tree-sitter
CI currently does not test on Mac, I set up a GitHub action to test this on Ubuntu (with and without libc++
installed), Windows and Mac here. CI results here. (Note that the build failure for PyPy on Mac is not related to the patch, but also present on current master.)
If this is OK for you, I will make a PR. Should I include the GitHub action?
from py-tree-sitter.
What would you consider a thorough test for this? Would passing the test suite be enough?
👍 Yes.
Note that the build failure for PyPy on Mac is not related to the patch, but also present on current master
That's strange; I personally develop on a mac, and the tests run fine for me. The error message is interesting:
clang: warning: include path for libstdc++ headers not found; pass '-stdlib=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
tests/fixtures/tree-sitter-python/src/scanner.cc:2:10: fatal error: 'vector' file not found
#include <vector>
^~~~~~~~
It looks like, for some reason, the flag -stdlib=libstdc++
is being passed to clang, indicating that it should use the non-default C++ standard library.
from py-tree-sitter.
I reran the tests without --quiet
. This is the compiler invocation on PyPy/Mac that errors:
cc -arch x86_64 -fPIC -Itests/fixtures/tree-sitter-python/src -c tests/fixtures/tree-sitter-python/src/scanner.cc -o /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpg8_knoprtree_sitter_language/tests/fixtures/tree-sitter-python/src/scanner.o
and on Python3.8/Mac:
cc -fPIC -Itests/fixtures/tree-sitter-python/src -c tests/fixtures/tree-sitter-python/src/scanner.cc -o /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpqvfc5o5vtree_sitter_language/tests/fixtures/tree-sitter-python/src/scanner.o
So the only difference is -arch
? I don’t see why this should influence clang
’s include path, though I don’t know Apple Clang at all.
from py-tree-sitter.
Oh, I think it's because we should be using c++
instead of cc
as the executable?
from py-tree-sitter.
I also think it's fine to just not include PyPy on mac in our CI matrix. I'm sorry, I misunderstood you and didn't realize that there was a successful macOS build using Python3.8.
from py-tree-sitter.
Oh, I think it's because we should be using
c++
instead ofcc
as the executable?
It shouldn’t matter. cc
as the compiler driver usually selects the right compiler based on the extension.
I also think it's fine to just not include PyPy on mac in our CI matrix.
from py-tree-sitter.
Related Issues (20)
- Broken (?) README instructions to pip install pre-compiled binary wheels HOT 1
- Python 3.12 support for version 0.21.0 HOT 2
- both node ==None and node != None return False HOT 1
- Remove/replace the usage of distutils HOT 1
- Missing aarch64 packages for 0.21.x HOT 1
- Make the Node class hashable. HOT 1
- [tree-sitter-python] No matching distribution found for tree-sitter-python HOT 3
- Python nil pointer crash while iterating over query captures
- Results returned by `Query.matches()` and `Query.captures()` are different HOT 2
- FutureWarning: Language(path, name) is deprecated. HOT 1
- Deprecation warning `Language.build_library is deprecated. Use the new bindings instead` HOT 4
- Tree sitter gets stuck HOT 3
- Can't pip install tree-sitter-python on mac os with apple silicon HOT 1
- way to get parser for typescript/tsx HOT 2
- How to get the location of a variable name in the whole function?
- Does that mean it's a conversion error? HOT 2
- How to build 0.22.x from source HOT 1
- TypeError on initializing Language with tree_sitter_python after following example code in readme. HOT 2
- Replacement of `build_library`
- Fails to load language on 32 bit HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from py-tree-sitter.