Comments (2)
pquentin
commented on May 27, 2024
Thanks for the report, Max. However, I can't reproduce.
Going to https://colab.research.google.com/github/elastic/elasticsearch-labs/blob/main/supporting-blog-content/multilingual-e5/multilingual-e5.ipynb and running the !pip install elasticsearch eland[pytorch] code block does work:
Collecting elasticsearch
Downloading elasticsearch-8.12.0-py3-none-any.whl (431 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 431.9/431.9 kB 4.2 MB/s eta 0:00:00
Collecting eland[pytorch]
Downloading eland-8.12.0-py3-none-any.whl (163 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.5/163.5 kB 9.4 MB/s eta 0:00:00
Collecting elastic-transport<9,>=8 (from elasticsearch)
Downloading elastic_transport-8.12.0-py3-none-any.whl (59 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.9/59.9 kB 6.0 MB/s eta 0:00:00
Requirement already satisfied: pandas<2,>=1.5 in /usr/local/lib/python3.10/dist-packages (from eland[pytorch]) (1.5.3)
Requirement already satisfied: matplotlib>=3.6 in /usr/local/lib/python3.10/dist-packages (from eland[pytorch]) (3.7.1)
Requirement already satisfied: numpy<2,>=1.2.0 in /usr/local/lib/python3.10/dist-packages (from eland[pytorch]) (1.23.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from eland[pytorch]) (23.2)
Collecting torch<2.0,>=1.13.1 (from eland[pytorch])
Downloading torch-1.13.1-cp310-cp310-manylinux1_x86_64.whl (887.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 887.5/887.5 MB 1.7 MB/s eta 0:00:00
Collecting sentence-transformers<=2.2.2,>=2.1.0 (from eland[pytorch])
Downloading sentence-transformers-2.2.2.tar.gz (85 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.0/86.0 kB 11.3 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting transformers[torch]<=4.33.2,>=4.31.0 (from eland[pytorch])
Downloading transformers-4.33.2-py3-none-any.whl (7.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.6/7.6 MB 81.7 MB/s eta 0:00:00
Requirement already satisfied: urllib3<3,>=1.26.2 in /usr/local/lib/python3.10/dist-packages (from elastic-transport<9,>=8->elasticsearch) (2.0.7)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from elastic-transport<9,>=8->elasticsearch) (2023.11.17)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.6->eland[pytorch]) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.6->eland[pytorch]) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.6->eland[pytorch]) (4.47.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.6->eland[pytorch]) (1.4.5)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.6->eland[pytorch]) (9.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.6->eland[pytorch]) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.6->eland[pytorch]) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<2,>=1.5->eland[pytorch]) (2023.3.post1)
Requirement already satisfied: transformers<5.0.0,>=4.6.0 in /usr/local/lib/python3.10/dist-packages (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (4.35.2)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (4.66.1)
Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (0.16.0+cu121)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (1.2.2)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (1.11.4)
Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (3.8.1)
Collecting sentencepiece (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch])
Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 64.0 MB/s eta 0:00:00
Requirement already satisfied: huggingface-hub>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (0.20.3)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch<2.0,>=1.13.1->eland[pytorch]) (4.5.0)
Collecting nvidia-cuda-runtime-cu11==11.7.99 (from torch<2.0,>=1.13.1->eland[pytorch])
Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 849.3/849.3 kB 59.9 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu11==8.5.0.96 (from torch<2.0,>=1.13.1->eland[pytorch])
Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 557.1/557.1 MB 2.4 MB/s eta 0:00:00
Collecting nvidia-cublas-cu11==11.10.3.66 (from torch<2.0,>=1.13.1->eland[pytorch])
Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 MB 2.5 MB/s eta 0:00:00
Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch<2.0,>=1.13.1->eland[pytorch])
Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.0/21.0 MB 51.8 MB/s eta 0:00:00
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch<2.0,>=1.13.1->eland[pytorch]) (67.7.2)
Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch<2.0,>=1.13.1->eland[pytorch]) (0.42.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch]) (3.13.1)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch]) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch]) (2023.6.3)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch]) (2.31.0)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch])
Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 77.2 MB/s eta 0:00:00
Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch]) (0.4.1)
Collecting accelerate>=0.20.3 (from transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch])
Downloading accelerate-0.26.1-py3-none-any.whl (270 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 270.9/270.9 kB 27.9 MB/s eta 0:00:00
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.20.3->transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch]) (5.9.5)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (2023.6.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib>=3.6->eland[pytorch]) (1.16.0)
Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk->sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (8.1.7)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (1.3.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch]) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers[torch]<=4.33.2,>=4.31.0->eland[pytorch]) (3.6)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch]) (3.2.0)
INFO: pip is looking at multiple versions of torchvision to determine which version is compatible with other requirements. This could take a while.
Collecting torchvision (from sentence-transformers<=2.2.2,>=2.1.0->eland[pytorch])
Downloading torchvision-0.16.2-cp310-cp310-manylinux1_x86_64.whl (6.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 90.7 MB/s eta 0:00:00
Downloading torchvision-0.16.1-cp310-cp310-manylinux1_x86_64.whl (6.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 61.2 MB/s eta 0:00:00
Downloading torchvision-0.16.0-cp310-cp310-manylinux1_x86_64.whl (6.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.9/6.9 MB 63.5 MB/s eta 0:00:00
Downloading torchvision-0.15.2-cp310-cp310-manylinux1_x86_64.whl (6.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 91.1 MB/s eta 0:00:00
Downloading torchvision-0.15.1-cp310-cp310-manylinux1_x86_64.whl (6.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 59.8 MB/s eta 0:00:00
Downloading torchvision-0.14.1-cp310-cp310-manylinux1_x86_64.whl (24.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.2/24.2 MB 14.2 MB/s eta 0:00:00
Building wheels for collected packages: sentence-transformers
Building wheel for sentence-transformers (setup.py) ... done
Created wheel for sentence-transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125923 sha256=95fc17059701a3d5ccf9b50cb5433913fd2faedbcbabb91f493ef9b0c29c2e45
Stored in directory: /root/.cache/pip/wheels/62/f2/10/1e606fd5f02395388f74e7462910fe851042f97238cbbd902f
Successfully built sentence-transformers
Installing collected packages: tokenizers, sentencepiece, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cublas-cu11, elastic-transport, nvidia-cudnn-cu11, elasticsearch, transformers, torch, eland, torchvision, accelerate, sentence-transformers
Attempting uninstall: tokenizers
Found existing installation: tokenizers 0.15.1
Uninstalling tokenizers-0.15.1:
Successfully uninstalled tokenizers-0.15.1
Attempting uninstall: transformers
Found existing installation: transformers 4.35.2
Uninstalling transformers-4.35.2:
Successfully uninstalled transformers-4.35.2
Attempting uninstall: torch
Found existing installation: torch 2.1.0+cu121
Uninstalling torch-2.1.0+cu121:
Successfully uninstalled torch-2.1.0+cu121
Attempting uninstall: torchvision
Found existing installation: torchvision 0.16.0+cu121
Uninstalling torchvision-0.16.0+cu121:
Successfully uninstalled torchvision-0.16.0+cu121
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchaudio 2.1.0+cu121 requires torch==2.1.0, but you have torch 1.13.1 which is incompatible.
torchdata 0.7.0 requires torch==2.1.0, but you have torch 1.13.1 which is incompatible.
torchtext 0.16.0 requires torch==2.1.0, but you have torch 1.13.1 which is incompatible.
Successfully installed accelerate-0.26.1 eland-8.12.0 elastic-transport-8.12.0 elasticsearch-8.12.0 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 sentence-transformers-2.2.2 sentencepiece-0.1.99 tokenizers-0.13.3 torch-1.13.1 torchvision-0.14.1 transformers-4.33.2
(The error at the end is about torch packages that we don't use, and can be ignored.)
That said, I'm intrigued by this line in your output:
ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11
This suggests that pip was not using Python 3.10, otherwise Requires-Python >=3.7,<3.11 would be satisfied. You can use !python --version to check that your Python version is correct and !pip --version to make sure that pip is using that version. Sometimes, python -m pip ... is required to make sure pip uses the correct Python version. If that fixes it for you, we could update the notebook.
from eland.
maxhniebergall
commented on May 27, 2024
Thanks for trying to reproduce this @pquentin, I ran the version commands and indeed it seems that I was not running the correct version of python.
pyenv: python: command not found
The python command exists in these Python versions:
3.8.16
3.9.16
3.10.9
3.10.10
3.11.4
Note: See 'pyenv help global' for tips on allowing both
python2 and python3 to be found.
pip 23.2.1 from [/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip) (python 3.11)`
I used VS code to run this juptyer notebook, and used the UI element in VS code to select the python version/kernel, and it seems that selecting the kernel in the UI didn't translate to the python and pip calls in the jupyter notebook, so it seems like this is actually a bug in VS code.
from eland.
Related Issues (20)
- elastic/eland Docker hub image is missing. HOT 4
- `eland_import_hub_model --task-type text_classification` fails HOT 2
- Make access_token an optional parameter in TransformerModel HOT 1
- Simplify model import code
- Improve compatibility testing of supported NLP models HOT 1
- Reduce transitive dependencies from sentence-transformers
- Add model JinaBertForMaskedLM to the supported list. HOT 1
- Investigate model test failures in CI
- `RandomForestClassifier` model cannot be uploaded HOT 7
- A possible issue with eland.Dataframe.value_counts(), the statistical information is missing some values
- Is there a way to return _scores when using es_query or es_match?
- Cannot append fields of type "dense vectorfield type
- Cannot append fields of type "dense_vector" to an existing index HOT 1
- why openai/clip-vit-base-patch32 model not support ! HOT 2
- support adding cert path to eland_import_hub_model script HOT 4
- Feedback 🗣️
- Failed to import huggingface model: cardiffnlp/twitter-roberta-base-sentiment HOT 1
- Upgrade PyTorch to version 2.1.2 and
- Support Python 3.11 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eland.