RAG: Problems resolving dependencies on Google Colab about cratedb-examples HOT 3 CLOSED

Observations

@hammerhead reported that he experienced the problems in the area of pueblo[fileio]. As of its v0.0.9, this is the list of dependencies of this extra:

fileio = [
  "fsspec[abfs,dask,gcs,git,github,http,s3,smb]<2024.3",
  "pathlibfs<0.6",
  "python-magic<0.5",
  "yarl<1.10",
]

-- https://github.com/pyveci/pueblo/blob/v0.0.9/pyproject.toml#L95-L100

On the other hand, this is the list of dependencies of current development head, which deviates in the version of fsspec, which, in turn, also pulls in dask, but without specifying any version.

fileio = [
  "fsspec[abfs,dask,gcs,git,github,http,s3,smb]<2024.4",
  "pathlibfs<0.6",
  "python-magic<0.5",
  "yarl<1.10",
]

-- https://github.com/pyveci/pueblo/blob/71ff638/pyproject.toml

Thoughts

I don't see any immediate problems with the combination of dependencies, but the devil may well be in the details.

from cratedb-examples.

Trial-and-error version pinning

Both of those unpinned dependencies had recent releases:

• https://pypi.org/project/google-cloud-aiplatform/#history
• https://pypi.org/project/langchain-google-vertexai/#history

Given that the local topic/machine-learning/llm-langchain/requirements.txt was updated on Mon Apr 15 2024, only the update to google-cloud-aiplatform 1.48.0 might be relevant.

@hammerhead: Can I ask you to try / play around with those commands on Google Colab, in order to find out about the dependency in question which is causing the package solver to detour into infinity?

This command should cause the symptom you are observing, right?

pip install --upgrade https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt

What about that?

pip install --upgrade https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt "google-cloud-aiplatform<1.48"

from cratedb-examples.

@hammerhead reported that he experienced the problems in the area of pueblo[fileio].

This patch gets rid of the dependency completely.

• GH-428

from cratedb-examples.