Support nix and guix by supporting multiple paths in janet's environment variables.

[amano-kenji]

On nix and guix, each package lives in its own tree.

For example,

• /gnu/store/xxxxxxxx-jpm-0.1.1/...
• /gnu/store/xxxxxxxx-janet-j3blocks-0.1.1/...

Without multiple paths in janet's environment variables, janet-j3blocks would have many symlinks to jpm, and janet-j3blocks-extra would have many symlinks to janet-j3blocks which in turn has many symlinks to jpm.

Practical Guix: Packaging Janet's jpm (Part 2) shows the difficulty of making jpm packages for nix and guix.

[sogaiu]

Please see this discussion for another method that was considered.

Update: Here something that points out the specific idea (though it's just an extract of the discussion mentioned).

---

BTW, thanks for posting the link.

[amano-kenji]

I just read jpm man page. If JANET_PATH took a (colon-separated) list of paths, I think nix and guix would be able to take jpm modules from various packages.

But, if this change takes place, JANET_MODPATH won't be able to default to JANET_PATH.

[sogaiu]

I don't know why no one seems to be commenting on the proposal in this discussion:

For this use case, I don't think setting JANET_PATH would be sufficient since it only contains one directory. I think what I need is a search path that supports a colon separated list of directories where modules can be found. In order to get that, I'm thinking of adding a Guix specific patch to Janet in our package that add such a search path via a GUIX_JANET_PATH environment variable. We've used similar strategies with other software before to good effect.

AFAICT, the idea there was proposed by someone who had knowledge of both guix and nix. The quote above even has a link to a case where a similar approach was tried with alleged success.

There was code discussed and both andrewchambers [1] and @bakpakin reviewed the idea and seemed to think it could work:

andrewchambers:

Looks pretty good to me in that case. Another approach might be to make a symlink farm to create a virtual JANET_PATH.

I think the 'Nix' way would be to have a nix function that takes in a list of janet packages and returns a janet with a wrapper that sets JANET_PATH to these paths.

bakpakin:

The proposed patch looks pretty good to me. However, ...

(elided modification suggestion that appeared to be accepted by proposer)

I think it would be better for people to consider that idea first (or the additional alternative -- see [1] below), before suggesting a change to janet like this.

IMO, there ought to be clearly spelled out reasons why the earlier proposal (or its alternative) won't / doesn't work, before asking for changes on janet's side.

---

[1] andrewchambers also suggested another approach which AFAIU does not involve modification to janet.

[amano-kenji]

I just ran a thought experiment in my head. Having a long colon-separated list of paths in JANET_PATH will be slow because every module will go through $JANET_PATH once.

For transitive dependencies, transitive symlinks are natural. Transitive symlinks would be still faster than a long list of search paths. However, if guix just copies symlinks intead of making symlinks to symlinks to symlinks to symlinks, we can replace transitive symlinks with direct symlinks. Direct symlinks are the fastest as long as they point to the right paths.

I don't know whether symlinks are problematic in practice. One potential problem of direct symlink farms is disk usage.

[amano-kenji]

Adam Faiz wrote

Search paths are preferred because their purpose is to provide a list of places to look for extensions/plugins/mods of a package. Essentially, search paths are a way to provide runtime dependencies that are optional.

Packages usually link directly to their dependencies at compile time, except for scripting programming languages(which aren't expected to have linking functionality). Propagated inputs are used for the transitive dependencies you mention, but they should only be used for required dependencies that have to be installed with the package for it to detect them.

Packages shouldn't need to have symlinks in their output to another package when it could just directly reference it instead. More information on how native-inputs, inputs and propagated-inputs are different is explained in the Guix manual:
https://guix.gnu.org/manual/devel/en/html_node/package-Reference.html#index-inputs_002c-of-packages

[amano-kenji]

IMO, there ought to be clearly spelled out reasons why the earlier proposal (or its alternative) won't / doesn't work, before asking for changes on janet's side.

What proposal? You need to be clear. List them out in clear language.