Including project libraries in odig's documentation collection

[hcarty]

I have a project with multiple libraries defined in it. I can generated their documentation with dune build @doc and I can, using a local opam switch, generate documentation for all of the project's dependencies using odig odoc.

Does odig have a way to link and bundle all of this documentation together? The project-local libraries are not separately packaged for opam so they aren't installable under the local opam switch. What I would really like is to be able to do something equivalent to

dune build @doc
<insert magic here>
odig odoc
gh-pages-amend $(odig cache path)/html doc

and have all of the local library documentation and all of the opam dependency documentation properly linked and exposed for users to browse.

[dbuenzli]

Does odig have a way to link and bundle all of this documentation together?

No at the moment I think the simplest is that you make the cmti available files in some way in $(opam var lib) (symlinks may do) under a special package name and try to odig the result.

In any case given the dependency order this is rather a job for dune than for odig. You won't be able to link things after the html has been generated by dune. The dune generation needs to see the odoc files of the dependencies and you would need something like ocaml/odoc#27.

[hcarty]

I think the simplest is that you make the cmti available files in some way in $(opam var lib) (symlinks may do) under a special package name and try to odig the result.

This works! Thanks for the suggestion. I basically did the following:

cd _opam/lib
for l in ../../_build/default/libraries/*; do
  base=$(basename $l)
  ln -s $l/src/.$base.objs/byte $base
done

It's not particularly elegant or easily reversible, but it gave me the results I was looking for in the short term.

[dbuenzli]

I'm not sure I understand the build structure here.

But isn't there a clean directory prefix that has one lib per directory with its build artefacts. If that is the case one thing that could be done is to turn the various directory configuration of odig conf into search paths.

You could then simply export ODIG_LIBDIR=_build/...:$(opam var lib); odig odoc.

In fact I always thought I'd eventually need to do something along these lines to support @lpw25's namespaces via the OCAML_NAMESPACES variable.

But I find his current spec for the variable a bit too complex --- @lpw25 I have the impression that since symlinks now exists on all platforms you could simply have a regular search path instead with the first level of directories in the search paths defining the names (also the overriding order you mandated is counter intuitive to the order used e.g. by gcc's C_INCLUDE_PATH, LIBRARY_PATH which follow the order of PATH, first match from the left wins).

[hcarty]

In this repo there are a few libraries with their source under libraries/$(libname)/src/. dune builds their *.cmti files under _build/default/libraries/$libname/src/.$libname.obs/byte/. odig didn't seem to find the *.cmti files unless they were in a top-level directory under $(opam var lib).

I didn't know about ODIG_LIBDIR - thank you for pointing it out. Will odig recursively search for a source of *.cmti files under the directories listed or do they need to be at a specific depth?

[dbuenzli]

I didn't know about ODIG_LIBDIR - thank you for pointing it out. Will odig recursively search for a source of *.cmti files under the directories listed or do they need to be at a specific depth?

It should. You can always see which packages it picks up and their information by invoking odig pkg --long. In this particular consult the doc-cobjs field (or directly invoke odig show doc-cobjs --long).

[lpw25]

I have the impression that since symlinks now exists on all platforms you could simply have a regular search path instead with the first level of directories in the search paths defining the names

The reason to have an additional layer of names is that the top level of names is inherently contested. If we try to make it be the package names then all OCaml packaging systems must agree on globally unique names for every package in order to inter-operate. Whereas if the top level of names is instead the names of the packing systems themselves (e.g. dune, opam, ocamlfind, debian) then only those must be unique -- the names of packages need only be unique within each packaging systems not globally.

also the overriding order you mandated is counter intuitive to the order used e.g. by gcc's C_INCLUDE_PATH, LIBRARY_PATH which follow the order of PATH, first match from the left wins

That does seem like a mistake -- could you add an issue to the repo to track it.

[dbuenzli]

I don't think I grasp your point. From a developer perspective don't I want to hard-code packaging system names in my build systems and/or sources.

I rather expect OCAML_NAMESPACES to union (with shadowing) sources of packages rooted in different paths in the following sense: If OCAML_NAMESPACES=p1:p2:p3 looking up for module N consists in looking up for N in the first matching directory path (p1, p2 or p3) from left to right.

And I think what you want could still be achieved with such a simpler scheme by doing a:

mkdir p1 && cd p1
ln -s PATH/TO/DEBIAN/LIB debian
ln -s $(opam var lib) opam
export OCAML_NAMESPACES=p1:$OCAML_NAMESPACES

This would make the lookup procedure easier to understand and implement for other tools (like odig).

That does seem like a mistake -- could you add an issue to the repo to track it.

lpw25/namespaces#8

[dbuenzli]

I proposed to standardize an OCAMLPATH upstream to solve that problem.