Comments (14)
I am the MicroMamba.jl team! π The job of that package is very simple - to give you a working micromamba installation isolated from the rest of your system, so it's not the place for these changes.
However it will be simple to add a "SystemConda" backend to CondaPkg.jl to use a pre-installed Conda (and its root prefix) instead of using MicroMamba.jl. It would be activated by setting the env var JULIA_CONDAPKG_BACKEND=SystemConda
.
from condapkg.jl.
Currently there's no option to do this - CondaPkg always creates a Conda environment per Julia project. Of course this behaviour could be changed given a good reason.
With the Conda distribution of Julia, are you allowed to use multiple Julia projects per Conda environment? I'm not sure it makes sense to tie these all to the same Conda environment, since the point of projects is to have independent dependencies.
I'm also a bit wary of special-casing the behaviour of CondaPkg depending on which Julia distribution one is using.
from condapkg.jl.
In the 1.7 branch, via the above activate.sh script, we create a new Julia depot per conda environment. What we basically need are two settings:
- An environmental variable to indicate the location of the conda executable, via JuliaPy/Conda.jl#216 this will
CONDA_JL_CONDA_EXE
. Notably this could either point toconda
ormamba
- An environmental variable to indicate the prefix of the conda environment. This is currently
CONDA_JL_HOME
One option would be just to use the same variables as Conda.jl. Another might be to use the more modern Preferences.jl.
from condapkg.jl.
Ok, I'll think about adding an "external" backend to allow the user to use a pre-existing Conda environment, triggered by an environment variable.
Such a backend would differ quite a bit from the default behaviour of CondaPkg. First, it would use the same Conda environment for all Julia projects. Second (and less important), it would never remove unneeded packages. As such I'm not convinced you should set this as the default behaviour. That's up to you though.
from condapkg.jl.
I'm not sure it makes sense to tie these all to the same Conda environment, since the point of projects is to have independent dependencies.
I'm also a bit wary of special-casing the behaviour of CondaPkg depending on which Julia distribution one is using
I think what we are really advocating for here is just to avoid reinstalling conda/mamba. So I don't think you should change anything in your setup, except to use the conda environment already existing as base. From there, you can make as many conda/julia envs as you want (we also added stacking support).
Does that make sense? In my view, this is part is not necessary:
Such a backend would differ quite a bit from the default behaviour of CondaPkg. First, it would use the same Conda environment for all Julia projects.
How does your CondaPkg handle the base conda environment? Just have it take as the conda env where the julia from conda-forge (from julia-feedstock) is installed. So, let's say someone makes a conda env called jenv
, then does conda install julia
(or in one line, conda create -n jenv julia
). This will isolate julia inside the jenv
environment, also creating a julia environment/project called jenv
instead of v1.7
. I think what will make most sense is for you to take the conda env jenv
as your base
and then from there continue as you'd like --- all would be nested and isolated inside that $CONDA_PREFIX ad infinitum (unless the user specifies otherwise).
from condapkg.jl.
A more interesting question for me is how does your package handle shared dependencies? Say, conda install llvm
and the julia-vendored LLVM. There is now an increasing number of these, and that's a major headache in the julia-feedstock these days.
from condapkg.jl.
To be slightly more specific, I think only control-flow-type additions are needed around:
here:
here:
here:
The last part tells me you're essentially setting up a new base and relying on it to branch out. This could be changed by replacing $MAMBA_ROOT_PREFIX
to something like $CONDA_PREFIX
and condabin
to bin
. But it should only do so if it detects that we are inside a conda env to begin with, hence the environment variable suggested by @mkitti.
I guess though, you'd likely have to change a lot more because you've built this entire thing around this MAMBA (I am not sure where it comes from) so maybe it is better for us to target interop upstream with mamba and then it will trickle down here. Does that sound like a better idea, @cjdoris?
from condapkg.jl.
I think there's some confusion in terminology, because Conda conflates two separate things:
- The base environment, which is the default Conda environment.
- The root prefix, which is where Conda caches packages so that installing the same thing into N different environments doesn't download it N times.
In Conda (and Mamba I think), the base environment and the root prefix are the same directory. In MicroMamba there is no base environment: the root prefix is NOT typically a Conda environment.
Presently, CondaPkg.jl uses the MicroMamba executable and root prefix supplied by the MicroMamba.jl package. With this, it creates a Conda environment specific to the current Julia project.
One thing it would be easy to add would be the ability to configure CondaPkg.jl to use a different executable and root prefix (which I'll call the "backend"). I was always intending to add multiple backends to CondaPkg.jl but haven't yet.
For example, there could be a "SystemConda" backend which uses whichever conda
is already in the PATH, and the root prefix it is already configured with. Being Conda, the root prefix and the base environment are the same thing, but CondaPkg.jl doesn't care that the root prefix is an environment and doesn't use it. Except that CondaPkg.jl doesn't deactivate any existing Conda environment, so the project-specific environment it creates is effectively stacked on top of whichever environment was activated before launching Julia (usually the base environment, or more likely jenv
in your example above).
What I initially thought you were asking for was the ability to force CondaPkg.jl not to create its own project-specific environment, but instead use some fixed Conda environment. This pretty much goes against the central feature of CondaPkg.jl (separation of dependencies between projects) so I'm much less in favour of this.
from condapkg.jl.
Presently, CondaPkg.jl uses the MicroMamba executable and root prefix supplied by the MicroMamba.jl package. With this, it creates a Conda environment specific to the current Julia project.
Okay, I see. Thanks for explaining this --- obviously my response above was conflating micromamba with mamba π
Wouldn't be easier if we just talk to the MicroMamba.jl team and have them set their root prefix as jenv
above in the case when in a the conda-forge's julia? We would need to trigger an installation of micromamba or something as well...
from condapkg.jl.
I am the MicroMamba.jl team! π
Major plot twist π€―
from condapkg.jl.
OK it was simple so I did it.
You can now set JULIA_CONDAPKG_BACKEND=System
to use a pre-installed conda
(or mamba
or micromamba
) instead of the one from MicroMamba.jl.
Not released yet, but on the main branch if you want to try it out.
from condapkg.jl.
Closing since I added the backend. Let me know if you have more questions.
from condapkg.jl.
Thanks! Has this been released yet?
from condapkg.jl.
Good point, I've just triggered a release.
from condapkg.jl.
Related Issues (20)
- Not able to specify βextrasβ in `pip_add` ? HOT 1
- Set conda environment for a package HOT 2
- CondaPkg constantly uninstalling, reinstalling all packages HOT 4
- `conda-build` and `conda skeleton` HOT 3
- Automatically load conda packages from the dependend julia packages HOT 3
- To do: ignore indirect dependencies on CondaPkg
- How to make RCall.jl use a CondaPkg.jl environment? HOT 8
- add preferences to control allowed channels and priority
- freeze the environment by default when it is activated HOT 2
- add completions for rm, pip_rm and channel_rm HOT 1
- dependency overrides HOT 1
- [Bug] file is not passed to write_toml HOT 2
- only add channel if it is not already added
- using CondaPkg with Pluto leads to unnecesarry resolves HOT 7
- Post install/update hook HOT 3
- Feature Request: Support CONDA_CHANNELS to set conda channels HOT 1
- access denied.: "C:\Users\...\v1.9\.CondaPkg\env\DLLs\_ctypes.pyd.mamba_trash" HOT 1
- `pip_add` for URLs HOT 6
- `critical libmamba could not load prefix data` after conda add failed due to low memory HOT 1
- Make `--override-channels` optional? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from condapkg.jl.