Why l3build needs some spring cleaning

[jlaurens]

Some time ago, l3build was made available to the public, and according to questions on TXS, package writers do use it. I bet that people will use it more and more for 2 reasons: the \Huge testing facilities and the necessary transition to LaTeX3.

Actually, there is at least one minor bug and some potential problems but none of them impact the usage of l3build as it was designed for: building latex. However, their impact on 3rd party code may be real, causing weird behaviors difficult to debug. All this is not easy to evaluate because the whole source code is still very intricate.

There are at least 4 points where l3build can be improved or enhanced:

1. User interface
2. code style
3. code design
4. unit testing

User interface

Here are the limited limitations I can think of

1. l3build allows custom targets, but its internals are publicly exposed for that, some dedicated function would help.
2. it does not allow custom options without hacking, some public dedicated function would help too
3. it may allow to turn switches off: something like --quiet [true], --quiet false, --debug [on], --debug off...

All this will not break existing l3build usage.

code style

We can see how different pieces of code reflect the style of their contributors.
Unfortunately, there is some lack of consistency that makes the code difficult to read and understand,
therefore difficult to maintain.
Some basic rules would help a lot on that respect. Fortunately, these are just cosmetic changes.

code design

When run l3buildbasically

• defines all its functions by loading many different files in a precise order
• loads the custom build.lua
• initializes the variables globally
• then possibly reads a config filebuild-...lua
• performs the target action.

During these steps, quite any object is kept in the global environment which is not recommended
for various very good reasons. The only variable and functions publicly available to build*.lua should be
exactly the ones documented in l3build.dtx, no less and no more (except intentionally hidden private features of course).

The status of all global variables is quite difficult to guess. We understand that some of them are readonly because the code seems to make the implicit assumption that they won't ever be modified. But clearly stating the readonly status and preventing modifications would help a lot. For example, one of the modules should be dedicated to OS related stuff and would be readonly.

Code separation is also a good practice. Splitting the code into different modules is definitely the way to go. The next step is to minimize module dependencies, in particular, cross dependencies should really be avoided. For that, some functions should rely more on formal arguments than global variables. Moreover require/provide techniques would help to manage module dependencies.

Unit testing

One of the main problems of unit testing is code coverage: any reachable piece of code should be tested. Testing can be made directly on l3build by analyzing the results of a command run in some source directory. Many different source examples are required to cover the different situations. This is provided. On the other hand, one can use unit testing with the help of a standard Lua library. This is a flexible and complementary approach we should definitely tend to adopt.

Is it possible to make all these changes without breaking anything? Definitely yes, but this is the subject of a forthcoming discussion.

In the meantime comments and suggestions are welcome.

[josephwright]

I'm new to GitHub Discussions so will do my best to get replies here right :) Ideally one would branch into separate threads, but I don't see how that's done ... the equivalent of 'XXXX [Was: YYYY]' is not obvious to me.

Actually, there is at least one minor bug and some potential problems but none of them impact the usage of l3build as it was designed for: building latex. However, their impact on 3rd party code may be real, causing weird behaviors difficult to debug. All this is not easy to evaluate because the whole source code is still very intricate.

I know you are aware, but for the record here: the main aim of l3build remains to support LaTeX Project code, so there are some compromises. I'll also mention in other replies that there is quite a bit of 'history' involved. That said, the aim in making it a generic tool was two-fold

• To as far as possible make it easy to use by simply setting variables, rather than writing Lua functions
• To allow users who do need specific functions to have some prototypes to work from

Now, I know it's not perfect: I learned a lot of Lua writing the early versions of l3build, but I'm still no expert, and I would design some stuff differently if starting from scratch. Improvements are very welcome. I do note, though, that most people who as about l3build are mainly not quite getting which variables to use, rather than running into more major issues.

[josephwright]

User interface

Here are the limited limitations I can think of

1. `l3build` allows custom targets, but its internals are publicly exposed for that, some dedicated function would help.

I moved the targets to using a (documented) table rather than a simple hard-coded list in a main() function to help with that. It's also why I tried to document the functions which implement the main targets. I guess I could do with a bit more detail here.

2. it does not allow custom options without hacking, some public dedicated function would help too

True. At the moment, I've not worked out how to give a nice interface here. Ideas very welcome.

3. it may allow to turn switches off: something like `--quiet [true]`, `--quiet false`, `--debug [on]`, `--debug off`...

I see the point: should be easy enough to set up. That said, no one has ever asked for this ...

[josephwright]

code style

We can see how different pieces of code reflect the style of their contributors.
Unfortunately, there is some lack of consistency that makes the code difficult to read and understand,
therefore difficult to maintain.
Some basic rules would help a lot on that respect. Fortunately, these are just cosmetic changes.

The original code was all written by me and was self-consistent if not following a 'typical' Lua style. I agree that as the tool has grown there have been more variations in style. I'd be happy to go though some cosmetic refactoring, agreeing a rule, applying it to the current code, documenting it.

Does anyone know of any Lua linters?

[jlaurens]

I use VSCode with built in support for lua syntax highlighting and linting.

[josephwright]

Ah: I was thinking more something we could run either locally or on Travis-CI as part of commits. Probably itself a Lua script ...

[josephwright]

code design

When run l3buildbasically

* defines all its functions by loading many different files in a precise order

* loads the custom `build.lua`

* initializes the variables globally

* then possibly reads a config file`build-...lua`

* performs the target action.

A bit of background before moving to replying. The original design had a single file which was accessed by using texlua build.lua <target>. We split the file into parts, and also got things set up as a proper 'tool', but the general structure remains. In many ways, we'd have preferred to use DocStrip to take the separate sources and make a single .lua file, but of course that won't work if you want to use l3build from the get go to bootstrap.

We did deprecate the texlua build.lua form but haven't totally dropped it: perhaps that is a place to start?

During these steps, quite any object is kept in the global environment which is not recommended
for various very good reasons. The only variable and functions publicly available to build*.lua should be
exactly the ones documented in l3build.dtx, no less and no more (except intentionally hidden private features of course).

We moved some way toward that by documenting any of them ;) I agree this can and should be tightened up.

The status of all global variables is quite difficult to guess. We understand that some of them are readonly because the code seems to make the implicit assumption that they won't ever be modified. But clearly stating the readonly status and preventing modifications would help a lot.

I'm not completely clear what you mean!

For example, one of the modules should be dedicated to OS related stuff and would be readonly.

You mean os_...? I guess if someone wanted to alter them, it might make life 'interesting', but they should still do the same things. I don't think we ever say to set them in the docs, do we?

Code separation is also a good practice. Splitting the code into different modules is definitely the way to go. The next step is to minimize module dependencies, in particular, cross dependencies should really be avoided. For that, some functions should rely more on formal arguments than global variables. Moreover require/provide techniques would help to manage module dependencies.

Here I'm a bit wary, but would welcome input from proper Lua programmers (@zauguin, @cereda for example). As I said earlier, the file split is mainly for convenient. To me, everything is in one 'module': it's not like l3build file can/should be loaded by anything else.

[jlaurens]

1. Well, not telling people to do something is not really the same as telling people not to do something... Anyways the os_ stuff is just one example, from the programmer point of view the same questions arise for all global variables: when are they defined, when are they used, when are they modified? For example, main is an undocumented global variable set just before the dofilestep. A user can inadvertently break l3build just by defining this variable for her own purpose. The main problem is obvious, but aren't there other ways to make l3build malfunction?

2. It is more about overall code design and organization than proper lua coding. You can see l3build as a big module... with a submodule to test, a submodule to typeset, a submodule to upload... They all share the global variables, most of them being defined by l3build-variables.lua. But others are not (apart from missed local). Some variables are not shared by different submodules, they are just exposed to the public. They "should" be exposed to the public by this very submodule.

When l3build will need more capabilities (like minifying or uglifying for just itself), you'll have to properly modify l3build-variables.lua, l3build-arguments.lua and l3build.lua. Wouldn't it be nice to only have to modify the last one, and in a simple way, of course.

[davidcarlisle]

l3build really should allow customisation of internals for unforeseen needs. if we packaged everything up as a module and only exposed the documented interface, then extension is a lot harder. You seem to imply for example that main shouldn't be exposed but it is intentionally exposed to allow it to be given custom definitions eg in latex2e/build.lua admittedly you could have a module and expose main through its public interface, but it is hard to know in advance everything that needs to be customised. Whether main is just exposed as a global variable or via a module interface, if someone defines it incorrectly then things break, that can't really be helped.

[josephwright]

I agree that not everything that is global needs to be, and also that some things are not documented that are needed as public/global.

On the specifics of main(), the original texlua build.lua approach required we had main = main or stdmain to allow customisation. That's no longer the case so we could I think drop that, keep just main() and deprecate the stdmain() business.

[josephwright]

Unit testing

One of the main problems of unit testing is code coverage: any reachable piece of code should be tested. Testing can be made directly on l3build by analyzing the results of a command run in some source directory. Many different source examples are required to cover the different situations. This is provided. On the other hand, one can use unit testing with the help of a standard Lua library. This is a flexible and complementary approach we should definitely tend to adopt.

You might guess that I, at least, have basically no idea about how to do unit testing for Lua :) Without \tracingall or \showoutput it's to me rather awkward. I guess I would naturally print() to some file then check it against a reference one. That's almost certainly wrong, so some input is welcome.

[jlaurens]

I use LuaUnit with some ideas of mine to embed the tests in the source code. Very powerful and versatile. Consider for example
next foo.lua:

function Add( a, b ) return a + b end
--[[ LU
function TestAdd() LU.assertEquals( Add( 2, 2 ), 4 ) return
--]]

My script will turn foo.lua into foo.test.lua and make the test. If Add( 2, 2 ) would return anything but 4, it would be reported.

[FrankMittelbach]

Am 21.01.21 um 14:30 schrieb Joseph Wright:

We did deprecate the |texlua build.lua| form but haven't totally dropped it: perhaps that is a place to start?

just one comment on this one here: I would not drop that support. currently l3build works on *any* TeX installation that has a luatex engine around (or a lua on the system). l3build as a tool is great, but it is restricted to distributions based on TL or miktex and that's it. So dropping that deprecated way means you drop support for anything not having l3build

[josephwright]

True but one can always do kpsewhich l3build.lua then texlua <path-to->l3build.lua I think

[josephwright]

Or on a *nix OS something like texlua $(kpsewhich l3build.lua) <target> (seems to work here on macOS 11).

[zauguin]

Or even luatex $(kpsewhich l3build.lua) <target> to be even more compatible with even more system which miss binaries.

[josephwright]

Does that work? Doesn't it need a \directlua wrapper plus \end?

[zauguin]

@josephwright It's a special case for lua files: If the file extension is .lua, then luatex behaves like texlua.

[davidcarlisle]

On Thu, 21 Jan 2021 at 22:56, LAURENS Jérôme ***@***.***> wrote: I use LuaUnit with some ideas of mine to embed the tests in the source code. Very powerful and versatile. Consider for example next foo.lua: function Add( a, b ) return a + b end --[[ LU function TestAdd() LU.assertEquals( Add( 2, 2 ), 4 ) return --]] My script will turn foo.lua into foo.test.lua and make the test. If Add( 2, 2 ) would return anything but 4, it would be reported.

Currently we have no dependencies on Lua libraries that are not included in luatex as shipped in texlive and miktex, I think that's a feature that we should preserve, so I don't think a library like LuaUnit is an option, really.

[jlaurens]

More than 3Gb seems quite big to develop l3build.
Can't we downsize the developer requirements ?

[josephwright]

More than 3Gb seems quite big to develop l3build.
Can't we downsize the developer requirements ?

Huh? Do you mean TL? That's a given for a TeX developer, plus of course one can install a much smaller set up than 'scheme full' - we did a lot of minimisation for testing on Travis-CI.

[josephwright]

More than 3Gb seems quite big to develop l3build.
Can't we downsize the developer requirements ?

Huh? Do you mean TL? That's a given for a TeX developer, plus of course one can install a much smaller set up than 'scheme full' - we did a lot of minimisation for testing on Travis-CI.

I think you probably can do scheme-infra-only, which is only a few megabytes.

[zauguin]

I guess we need at least scheme-minimal (otherwise there is not even luatex), but even then without docs and sources it's just 45MB.

[josephwright]

I guess we need at least scheme-minimal (otherwise there is not even luatex), but even then without docs and sources it's just 45MB.

scheme-infra-only plus tlmgr install luatex I think would do it.

[davidcarlisle]

On Fri, 22 Jan 2021 at 16:55, LAURENS Jérôme ***@***.***> wrote: @josephwright <https://github.com/josephwright> LuaUnit.lua is exactly what you are asking for. You just need to have a copy in the repository and that's it, and a lua executable of course. Whether tests are launched by texlua or lua does not matter.

Well yes and no. Especially when Lua has big changes such as when the integer type was added, what matters is that the code matches texlua in the targeted version of texlive. Whether the code works with the version of Lua you have locally (or even if you have a Lua install at all) is not important, so while it doesn't do any harm if people try it with a system Lua, the test requirement has to be that the code is correct for texlua. David

[zauguin]

This is especially important since Lua 5.4 got released and therefore system Lua and luatex Lua (Which will probably stay at 5.3 for the foreseeable future) will have significantly different versions.

[FrankMittelbach]

Bottom line, whatever lua is supported by the current texlua is set, there is no way around that because the main goal is to provide support for TeX code on any TeX installation without further dependencies. I don't think we can or should deviate from that maxime.

[jlaurens]

Btw, how does one know which lua is embedded in a given texlua?
I could not find it in the help page nor the manual.

[FrankMittelbach]

Btw, how does one know which lua is embedded in a given texlua?
I could not find it in the help page nor the manual.

strangely enough it is not given with --version but with --credits:

texlua --credits
This is LuaTeX, Version 1.12.0 (TeX Live 2020)

The LuaTeX team is Hans Hagen, Hartmut Henkel, Taco Hoekwater, Luigi Scarso.

LuaTeX merges and builds upon (parts of) the code from these projects:

tex       : Donald Knuth
etex      : Peter Breitenlohner, Phil Taylor and friends
omega     : John Plaice and Yannis Haralambous
aleph     : Giuseppe Bilotta
pdftex    : Han The Thanh and friends
kpathsea  : Karl Berry, Olaf Weber and others
lua       : Roberto Ierusalimschy, Waldemar Celes and Luiz Henrique de Figueiredo
metapost  : John Hobby, Taco Hoekwater, Luigi Scarso, Hans Hagen and friends
pplib     : Paweł Jackowski
fontforge : George Williams (partial)
luajit    : Mike Pall (used in LuajitTeX)

Compiled with libpng 1.6.37; using 1.6.37
Compiled with lua version 5.3.5
Compiled with mplib version 2.00
Compiled with zlib 1.2.11; using 1.2.11

Development id: 7306

[davidcarlisle]

@jlaurens You can get a version string via print(_VERSION) or print(lua.version) but what matters is not only the nominal version but which libraries are pre-loaded and also in some cases in the past (but I am not sure whether it is still the case) any changes to the source code of Lua they make while incorporating it into luatex.