--recommended does not work for previous SDK versions

[vmarkovtsev]

Using the latest docker image, I run

docker exec -it bblfshd bblfshctl driver install --recommended

This command silently returns and nothing happens. The logs stay silent, no drivers are installed.

Build information
  commit: v2.12.1
  date: 2019-04-10T16:05:44+0000

[vmarkovtsev]

--all works though.

[dennwc]

Is this a -drivers image?

[vmarkovtsev]

Nope, I followed https://docs.sourced.tech/babelfish/using-babelfish/getting-started

[creachadair]

Nope, I followed https://docs.sourced.tech/babelfish/using-babelfish/getting-started

Well, I guess at very least we should call out that distinction in our setup instructions.

[dennwc]

Interesting. Sounds like a bug, indeed.

[creachadair]

Repro (macOS):

$ docker volume create bblfshd-cache
$ docker run -d --name bblfshd --privileged -p 9432:9432 -v bblfshd-cache:/var/lib/bblfshd bblfsh/bblfshd
$ docker exec -it bblfshd bblfshctl driver install --recommended
$ docker exec -it bblfshd bblfshctl driver list
<empty>

My first attempt was to just switch to the drivers image:

$ docker run -d --name bblfshd --privileged -p 9432:9432 -v bblfshd-cache:/var/lib/bblfshd bblfsh/bblfshd:v2.12.0-drivers

But then even after docker exec -it bblfshd bblfshctl driver install --recommended the driver list was empty. As @vmarkovtsev pointed out, --all works in this latter case, though.

[creachadair]

Oh, and actually even without switching which image tag I used, --all worked fine (which in retrospect is probably what you meant).

[dennwc]

The workaround is to use -drivers images for now. It's the same as --recommended.

[dennwc]

OK, I know the reason now.

bblfshd uses either Github or a static driver list for discovery. To know what drivers are recommended, it checks the latest version of each driver and checks if the driver supports the same SDK version as bblfshd.

And recently we did a version bump for SDK and updated all the drivers to Go modules. Now old bblfshd considers all new driver versions incompatible because they use SDKv3, while bblfshd uses SDKv2.

Our mistake was that we did no version bump for drivers during the update. In theory, v2 and v3 drivers are compatible and have no differences in API. However, in practice, the driver discovery now has no way to know which driver uses which SDK version. It will have to scan all git tags to know that.

We will release a new bblfshd with SDKv3 in few days, which will fix this issue. Still, --recommended will not work for old bblfshd versions.

Thus, there is no easy fix for this, sorry. Installing driver with specific versions or using -drivers image is the only way right now.

[vmarkovtsev]

I confirm that it works now.

[dennwc]

@vmarkovtsev Awesome, thanks for checking it!

Although it works for the current version, we will face the same issue again when switching to v3 when the time will come.

So leaving the issue open to decide what a potential fix may be.

We keep exposing more information in the driver's manifest, so we should be able to correctly identify compatible ones after a few more changes.