To do: per-package channels about condapkg.jl HOT 7 CLOSED

You make a lot of good points, and I should go with what is most natural in Conda-land. I just find it weird that you can install the "same" (but actually different) package from different channels. But if that's how Conda works there's no point me trying to fight it.

So this proposal becomes: keep precisely the current behaviour, but allow the user to set all the fields of the Conda package spec, in particular the channel.

from condapkg.jl.

I agree that it's weird, but also just how conda works. The only alternatives I can think of involve

1. Calling conda install multiple times — once for each CondaPkg.toml. But that still involves ambiguity about the ordering, and would potentially install and reinstall the same packages, taking lots of time.
2. Trying to solve things yourself, which would require either using the conda API in complicated ways, or recreating that API in julia code — all in an effort to make CondaPkg behave differently from Conda itself.

So both for simplicity and consistency, I think your proposal sounds like the best option.

from condapkg.jl.

Ah, this is interesting. Instead of just the channel::package=ver syntax, it looks like you could support the whole range of MatchSpec capabilities by using package[channel='channel',version='ver'] syntax, with other key-value pairs as needed — and just some extra processing to resolve conflicting channels, as you've suggested. In particular, the difference between a package version '=0.53' and '==0.53' seems easier to deal with if you just put it in as a key-value pair.

It's not clear to me what exactly you mean by

merging two packages of the same name from different CondaPkg.toml files

Are you talking about just merging the channel spec for each package, or merging the whole package spec so that conda only sees each package once? For example, suppose one of the Project.toml dependencies in your example has this CondaPkg.toml:

[deps.bar]
channels = ["channel3", "channel4"]
md5 = "12345678901234567890123456789012"

When combined with your CondaPkg.toml above, you could merge the channels in the two bar specs in the julia code, but still supply them as separate specs to conda, so that the resulting command would look like this:

conda install foo[version='1'] bar[version='2',channel='channel3'] bar[channel='channel3',md5='12345678901234567890123456789012']

Conda has no trouble with the fact that bar is specified twice, and already has lots of complex logic to merge the specs (which you surely don't want to reproduce). Is this what you're intending?

from condapkg.jl.

You're absolutely right and in fact currently if a package appears in several CondaPkg.toml files with different versions then we specify the package several times. So indeed we could just allow the user to specify any of the spec fields and pass the specs on to Conda without changing them.

(Unfortunately Pip and Julia dependencies do need to be merged.)

So we could leave the current behaviour as is and just allow specifying more fields.

The main issue is with taking the union of the top-level channels from different toml files. It would be nice to guarantee that the packages in a file are installed from one of the specified channels, which isn't currently the case. To do this would require some logic with the top level channels and those on the package.

It would also be nice to let a package specify several channels it could come from.

from condapkg.jl.

I think the --channel CLI flags (especially when priority is turned off) are intended to say "You'll be able to find every package at least once by searching these channels, but I don't care where you get these packages." So in that sense, it's a non-restrictive statement, and taking the union would be fine. On the other hand, I would think of per-package channels as being more precise, and thus restrictive: "You must not install the package from anywhere else."

If you make the top-level channels option restrictive, you run into the problem that someone might say channels=["defaults", "bioconda"], just because they know their packages can be found in those channels, and don't want to bother checking if there are any other channels that might also work. But then I come along and specify that one of the packages they request must actually come from conda-forge, and now there's a conflict where there doesn't need to be.

Ultimately the problem is that conda itself doesn't support more complex channel specification, making it difficult for you to do so. But I'm also not sure it's really necessary. How often do people know that a package should only be installed from a limited set of channels, while being unable to decide on one channel to install from? Even the bioconda people recommend prioritizing conda-forge, for example. It seems like it would be fair for you to go the easiest route of just taking the union of all top-level channels, and if people turn up problems with that approach in the future, address it with more specific options.

from condapkg.jl.

@moble I've implemented this on the main branch now, I'd be grateful if you could try it out. You can now do CondaPkg.add("llvmlite", channel="numba") or equivalently pkg> conda add numba::llvmlite.

For now I've only added support for specifying the channel - supporting subdir and build is for another day.

from condapkg.jl.

Sorry, this fell off my radar for a bit. Yes, this works very nicely with the latest release. Thanks so much!

from condapkg.jl.