Create separate KOLIBRI_HOME for Endless Key #119
Conversation
jprvita
force-pushed
the
ek-home-dir
branch
12 times, most recently
from
ac97123
to
1f64ece
Compare
We want to be able to ship different content and configuration for Kolibri and the Endless Key app. This commit adds support for installing the content and configuration for each app to separate locations, so they can be used side-by-side. #117
The Endless Key app needs a different set of plugins than traditional Kolibri. #117
When downloading Kolibri channels to our cache server, also include Endless Key channels. #117
jprvita
force-pushed
the
ek-home-dir
branch
from
1f64ece
to
ecf9599
Compare
Partially revert "Create separate KOLIBRI_HOME for Endless Key" This reverts the changes to hooks/image/62-kolibri-automatic-provision from commit 35694c0.
There was a problem hiding this comment.
I have some thoughts here, but I'm not sure how urgently we want to address them.
The one thing I definitely think we should change is we don't need these changes for the image/62-kolibri-automatic-provision hook to support Endless Key. That stuff just happens on its own with the current version of the Endless Key app, and I don't think we'll have images needing it to behave differently.
| # FIXME: enabling kolibri-explore-plugin fails because we are using a | ||
| # kolibri version that is too old. | ||
| #"kolibri_explore_plugin" |
There was a problem hiding this comment.
I think this is fine. It means Kolibri will need to execute some of extra database migrations on first run, but shouldn't be a problem.
One issue we are going to crash into here is that the Explore plugin provides its own applyexternaltags management command. We use that when importing content from our content collection files, and it's necessary so the Explore plugin knows how to populate the different sections in the Discovery page. Without those tags, the Discovery page is pretty bare.
I think that is going to be a bit of a puzzle to sort out, so I don't know if we want to do it here, but we'll want to do this all a bit differently in the near future. The good news is, I don't think we will need to put a Kolibri database in the image at all any more, so this issue about the Explore plugin and the old Kolibri version will probably be moot. All we need is to fetch the content files, put them somewhere (as in kolibri manage exportcontent), add a JSON file describing what that content is (the exportcontent command generates a content manifest file), and get the app to populate its database (with the correct content in the database and marked available) based on that file.
For Kolibri, that JSON file would be exactly what the exportcontent command generates. For Endless Key, we'd copy our own JSON files from endless-key-collections. Those ones include the information we need about external content tags.
There was a problem hiding this comment.
It's certainly a lot simpler to put the content directly in the Kolibri homedir rather than putting it somewhere else on disk and then coming up with a procedure to import it and then maybe delete the massive duplicated data.
When I was previously thinking about this, my understanding was that the content pack would still be picked at runtime. Then the explore plugin would very quickly run through the content downloader since all the content is already available. The last step would to apply the tags per the chosen content pack would still run.
There was a problem hiding this comment.
Another way to handle the whole mismatched versions thing is to just figure out the ones the flatpak has:
$ flatpak_site_packages=$(ls -d /var/lib/flatpak/app/org.endlessos.Key/current/active/files/lib/python*/site-packages)
$ PYTHONPATH="$flatpak_site_packages" pip list --format json > flatpak-packages.json
$ jq -r '.[] | select(.name == "kolibri") | .version' flatpak-packages.json
0.16.0b5
$ jq -r '.[] | select(.name == "kolibri-explore-plugin") | .version' flatpak-packages.json
6.47.0
Although for kolibri itself it would currently be awkward because we're install the betas directly from github releases instead of PyPI.
| # Preloaded content or the Endless Key app | ||
| # The Endless Key app can be pre-loaded with content from the Kolibri channels | ||
| # listed in this setting. | ||
| install_channels = |
There was a problem hiding this comment.
Is it intentional for the moment that we're going to do the old content nodes listed right in the image configuration? Another option here is we could add an install_content_collections option which lists names of collection files from endless-key-collections. The good news about those is Kolibri's importcontent management command already knows how to read them. The bad news is it would involve a whole new hook (and probably a Python script because that management command is a smidge limited at the moment).
| # FIXME: enabling kolibri-explore-plugin fails because we are using a | ||
| # kolibri version that is too old. | ||
| #"kolibri_explore_plugin" |
There was a problem hiding this comment.
Another way to handle the whole mismatched versions thing is to just figure out the ones the flatpak has:
$ flatpak_site_packages=$(ls -d /var/lib/flatpak/app/org.endlessos.Key/current/active/files/lib/python*/site-packages)
$ PYTHONPATH="$flatpak_site_packages" pip list --format json > flatpak-packages.json
$ jq -r '.[] | select(.name == "kolibri") | .version' flatpak-packages.json
0.16.0b5
$ jq -r '.[] | select(.name == "kolibri-explore-plugin") | .version' flatpak-packages.json
6.47.0
Although for kolibri itself it would currently be awkward because we're install the betas directly from github releases instead of PyPI.
|
|
||
| PLUGIN_LOCATION=`pip3 show kolibri-explore-plugin | grep Location | awk '{ print $2 }'` | ||
| APPS_LOCATION="${PLUGIN_LOCATION}/kolibri_explore_plugin" | ||
| COLLECTIONS_LOCATION="${PLUGIN_LOCATION}/kolibri_explore_plugin/static/collections/" |
There was a problem hiding this comment.
I don't think we need to do this. These bundles are only needed at runtime and aren't required if you're just running importchannel/importcontent.
There was a problem hiding this comment.
But don't we need to put them in place, so they can be found at runtime? Or do they already come with the Flatpak?
There was a problem hiding this comment.
The flatpak has them. Otherwise, how would it work if you just install the flatpak?
https://github.com/flathub/org.endlessos.Key/blob/master/modules/endless-key-collections.json
https://github.com/flathub/org.endlessos.Key/blob/master/modules/python3-kolibri-explore-plugin.json
There was a problem hiding this comment.
That makes sense.. I was confused by your earlier comment, but I guess things were not as well defined at that point.
There was a problem hiding this comment.
Sorry about that. I was totally wrong on what's needed here. In fact, the whole #117 issue could probably go away since we don't really need to create the homedir at all. What's actually needed is just preseeding content. Let's just keep going, but we may want to retitle the issue.
We want to be able to ship different content and configuration for
Kolibri and the Endless Key app. This commit adds support for installing
the content and configuration for each app to separate locations, so
they can be used side-by-side.
#117