js1010 / cuhnsw Goto Github PK

CUDA implementation of Hierarchical Navigable Small World Graph algorithm

License: Apache License 2.0

Cuda 62.36% C++ 14.66% Python 22.98%

hnsw gpu ann cuda

cuhnsw's Issues

Some APIs are avaiable?

Hi, I created a image matching system using the hnswlib.
I am going to replace the hnswlib with the cuhnsw to use GPUs.

Do some APIs of the hnswlib exist in the cuhnsw too?
For example,

hnswlib.add_items(data, data_labels) for adding custom id scheme and APPENDING(not setting) new data
hnswlib.get_items(ids)

Thanks

ERROR: Could not build wheels for cuhnsw, which is required to install pyproject.toml-based projects

When "pip install cuhnsw" in Ubuntu 16.04, meet a problem:
ERROR: Could not build wheels for cuhnsw, which is required to install pyproject.toml-based projects
Is there a built wheel file for linux?
Thanks!

ModuleNotFoundError: No module named'cuhnsw.cuhnsw_bind' is prompted after installation ubuntu18.04

I want to try the effect and continue to develop on the existing basis, but found that “ModuleNotFoundError: No module named'cuhnsw.cuhnsw_bind' is prompted after installation”，What's the reason for this，thanks！
Below is the relevant log（Username desensitization）

(base) xxxxxxx@ubuntu7:~/project/cuhnsw$ python setup.py install
running install
running bdist_egg
running egg_info
writing cuhnsw.egg-info/PKG-INFO
writing dependency_links to cuhnsw.egg-info/dependency_links.txt
writing entry points to cuhnsw.egg-info/entry_points.txt
writing requirements to cuhnsw.egg-info/requires.txt
writing top-level names to cuhnsw.egg-info/top_level.txt
reading manifest file 'cuhnsw.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'cuhnsw.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying cuhnsw/version.py -> build/lib.linux-x86_64-3.8/cuhnsw
running build_ext
cuhnsw.cuhnsw_bind
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/cuhnsw
copying build/lib.linux-x86_64-3.8/cuhnsw/pyhnsw.py -> build/bdist.linux-x86_64/egg/cuhnsw
copying build/lib.linux-x86_64-3.8/cuhnsw/config_pb2.py -> build/bdist.linux-x86_64/egg/cuhnsw
copying build/lib.linux-x86_64-3.8/cuhnsw/cuhnsw_bind.cpython-38-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg/cuhnsw
copying build/lib.linux-x86_64-3.8/cuhnsw/aux.py -> build/bdist.linux-x86_64/egg/cuhnsw
copying build/lib.linux-x86_64-3.8/cuhnsw/version.py -> build/bdist.linux-x86_64/egg/cuhnsw
copying build/lib.linux-x86_64-3.8/cuhnsw/__init__.py -> build/bdist.linux-x86_64/egg/cuhnsw
byte-compiling build/bdist.linux-x86_64/egg/cuhnsw/pyhnsw.py to pyhnsw.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/cuhnsw/config_pb2.py to config_pb2.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/cuhnsw/aux.py to aux.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/cuhnsw/version.py to version.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/cuhnsw/__init__.py to __init__.cpython-38.pyc
creating stub loader for cuhnsw/cuhnsw_bind.cpython-38-x86_64-linux-gnu.so
byte-compiling build/bdist.linux-x86_64/egg/cuhnsw/cuhnsw_bind.py to cuhnsw_bind.cpython-38.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying cuhnsw.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cuhnsw.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cuhnsw.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cuhnsw.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cuhnsw.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cuhnsw.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
cuhnsw.__pycache__.aux.cpython-38: module references __file__
cuhnsw.__pycache__.cuhnsw_bind.cpython-38: module references __file__
creating 'dist/cuhnsw-0.0.5-py3.8-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing cuhnsw-0.0.5-py3.8-linux-x86_64.egg
removing '/home/xxxxxxx/anaconda3/lib/python3.8/site-packages/cuhnsw-0.0.5-py3.8-linux-x86_64.egg' (and everything under it)
creating /home/xxxxxxx/anaconda3/lib/python3.8/site-packages/cuhnsw-0.0.5-py3.8-linux-x86_64.egg
Extracting cuhnsw-0.0.5-py3.8-linux-x86_64.egg to /home/xxxxxxx/anaconda3/lib/python3.8/site-packages
cuhnsw 0.0.5 is already the active version in easy-install.pth

Installed /home/xxxxxxx/anaconda3/lib/python3.8/site-packages/cuhnsw-0.0.5-py3.8-linux-x86_64.egg
Processing dependencies for cuhnsw==0.0.5
Searching for grpcio-tools==1.27.1
Best match: grpcio-tools 1.27.1
Adding grpcio-tools 1.27.1 to easy-install.pth file

Using /home/xxxxxxx/anaconda3/lib/python3.8/site-packages
Searching for protobuf==3.10.0
Best match: protobuf 3.10.0
Adding protobuf 3.10.0 to easy-install.pth file

Using /home/xxxxxxx/anaconda3/lib/python3.8/site-packages
Searching for numpy==1.19.2
Best match: numpy 1.19.2
Adding numpy 1.19.2 to easy-install.pth file
Installing f2py script to /home/xxxxxxx/anaconda3/bin
Installing f2py3 script to /home/xxxxxxx/anaconda3/bin
Installing f2py3.8 script to /home/xxxxxxx/anaconda3/bin

Using /home/xxxxxxx/anaconda3/lib/python3.8/site-packages
Searching for jsmin==2.2.2
Best match: jsmin 2.2.2
Adding jsmin 2.2.2 to easy-install.pth file

Using /home/xxxxxxx/anaconda3/lib/python3.8/site-packages
Searching for setuptools==50.3.1.post20201107
Best match: setuptools 50.3.1.post20201107
Adding setuptools 50.3.1.post20201107 to easy-install.pth file
Installing easy_install script to /home/xxxxxxx/anaconda3/bin

Using /home/xxxxxxx/anaconda3/lib/python3.8/site-packages
Searching for grpcio==1.35.0
Best match: grpcio 1.35.0
Adding grpcio 1.35.0 to easy-install.pth file

Using /home/xxxxxxx/anaconda3/lib/python3.8/site-packages
Searching for six==1.15.0
Best match: six 1.15.0
Adding six 1.15.0 to easy-install.pth file

Using /home/xxxxxxx/anaconda3/lib/python3.8/site-packages
Finished processing dependencies for cuhnsw==0.0.5
(base) xxxxxxx@poisson-ubuntu7:~/project/cuhnsw$ python
Python 3.8.5 (default, Sep  4 2020, 07:30:14) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cuhnsw
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/xxxxxxx/project/cuhnsw/cuhnsw/__init__.py", line 6, in <module>
    from cuhnsw.pyhnsw import CuHNSW
  File "/home/xxxxxxx/project/cuhnsw/cuhnsw/pyhnsw.py", line 15, in <module>
    from cuhnsw.cuhnsw_bind import CuHNSWBind
ModuleNotFoundError: No module named 'cuhnsw.cuhnsw_bind'

gpu：rtx3090 ， show error MemoryError: std::bad_alloc: cudaErrorMemoryAllocation: out of memory

File "/media/data/senith_e/anaconda3/lib/python3.9/site-packages/cuhnsw-0.0.8-py3.9-linux-x86_64.egg/cuhnsw/pyhnsw.py", line 63, in build
self.obj.build_graph()

when in gpu ：titan xp，can right run
so why in rtx3090 can’t

Wrong distance values?

I extracted 500 * 128 sized SIFT features of an image.
I built a cuhnsw index of that vector and queried with the same vector.
Desired distance values(topk = 1, l2 distance) are all 0, but I got very big values(greater than 10^6).
I tried to change some hyperparameters, but got the same result.

Hope help me with this issue.

Thanks

GPU: H100 , CUDA: 12.2 - issues getting arch recognized in build scripts.

This project hasn't been updated in a while, but in case the maintainer is still around:

Getting a failure with:

 UnboundLocalError: local variable 'arch' referenced before assignment

likely because the cuda_setup.py is quite a bit stale at this point with selection. Any guidance on extending it or even if it is going to work?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.