-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Content downloader tests #897
Commits on Oct 25, 2023
-
jobs: Fix remotecontentimport node_ids semantics
As a convenience, the generateed `remotecontentimport` task was setting `node_ids` to an empty list if it wasn't specified. However, an empty list and `None` are 2 different cases. An empty list is used when no node IDs are to be imported, and `None` is used when all node IDs are to be imported. Even though we're not actually downloading all nodes here, fix the semantics and require that the thumbnails tasks explicitly request no nodes. This will be used later in testing.
Configuration menu - View commit details
-
Copy full SHA for 780f18c - Browse repository at this point
Copy the full SHA 780f18cView commit details
Commits on Oct 26, 2023
-
jobs: Return enqueued task from enqueue_next_background_task
This will be used in testing to check that a task was actually enqueued.
Configuration menu - View commit details
-
Copy full SHA for 1d8215c - Browse repository at this point
Copy the full SHA 1d8215cView commit details -
jobs: Lock database when enqueueing background task
The storage hook depends on the `BackgroundTask` containing the current job ID. However, it's possible the job will change state and the hook will run before the job ID has been saved. That would prevent the hook from properly synchronizing the updated job state. To prevent that, lock the database while the task is being enqueued until the job ID is saved.
Configuration menu - View commit details
-
Copy full SHA for ba915b9 - Browse repository at this point
Copy the full SHA ba915b9View commit details -
tests: Improve pytest failure logging
By default set the log level to INFO so there's useful information on failures. Task handling is multithreaded, so add the thread name to help debugging.
Configuration menu - View commit details
-
Copy full SHA for 3e97462 - Browse repository at this point
Copy the full SHA 3e97462View commit details -
ci: Run pytest tests with debug logging
The logs won't be shown if the tests pass, but if they don't the messages can be invaluable for debugging failures.
Configuration menu - View commit details
-
Copy full SHA for 4b95c14 - Browse repository at this point
Copy the full SHA 4b95c14View commit details -
Learning Equality obviously has no use for the test suite constantly telling it about an ephemeral deployment. More importantly, the ping task can hang and interfere with our own tasks being scheduled.
Configuration menu - View commit details
-
Copy full SHA for 9f20fa5 - Browse repository at this point
Copy the full SHA 9f20fa5View commit details -
tests: Generate fake channel databases
In order to exercise the collection downloader, we need channel data. The `create_channeldb_json.py` script and `db.json.template` file are used to create JSON files representing fake channels. The generated JSON files contain the data necessary to create sqlite channel databases with sqlalchemy as well as the content data inlined. 18 fake channels have been created and will be used in later commits.
Configuration menu - View commit details
-
Copy full SHA for 6c3180a - Browse repository at this point
Copy the full SHA 6c3180aView commit details -
tests: Add fixture to create content directory
The `create_contentdir` function takes the fake channel DB JSON files and creates a content directory with the channel databases and content files for testing. That's made available as the `contentdir` pytest fixture. A standalone script is provided as a convenience and for exercising the functionality outside of the test suite.
Configuration menu - View commit details
-
Copy full SHA for 1d4984b - Browse repository at this point
Copy the full SHA 1d4984bView commit details -
tests: Add content server fixture
This runs an HTTP server for the test content so that we can import channels and content during tests just like they were being imported from studio.
Configuration menu - View commit details
-
Copy full SHA for f59744b - Browse repository at this point
Copy the full SHA f59744bView commit details -
Provide test collections using all the packs in the current endless-key-collections release. The fake packs have very regular structures and use the fake channel data.
Configuration menu - View commit details
-
Copy full SHA for bf986a6 - Browse repository at this point
Copy the full SHA bf986a6View commit details -
tests: Add collectionviews API tests
These exercise all of the API endpoints that don't interact with the download manager.
Configuration menu - View commit details
-
Copy full SHA for c312ec3 - Browse repository at this point
Copy the full SHA c312ec3View commit details -
tests: Add fixtures for channel and content imports
This adds a few pytest fixtures for running channel and content import jobs. Beyond creating a facility and a facility user with appropriate permissions, 2 fixtures for handling interactions between Django and SQLAlchemy. Some of this is lifted from Kolibri and/or depends on Kolibri internals. We'll see how well they hold up over time.
Configuration menu - View commit details
-
Copy full SHA for d1e4e56 - Browse repository at this point
Copy the full SHA d1e4e56View commit details -
While here, add a few log messages to aid when debugging failures.
Configuration menu - View commit details
-
Copy full SHA for f3973d2 - Browse repository at this point
Copy the full SHA f3973d2View commit details
Commits on Oct 27, 2023
-
Add pytest-rerunfailures dev dependency
It seems that Kolibri's task worker can hang occasionally, so use pytest-rerunfailures to mark tests that can be retried on failure.
Configuration menu - View commit details
-
Copy full SHA for a73de4c - Browse repository at this point
Copy the full SHA a73de4cView commit details -
tests: Add download manager test
This is a basic smoketest that the collection downloader can be run with any collection specified. Unfortunately, Kolibri's task worker seems to hang sometimes during either the foreground or background downloading. Use pytest-rerunfailures's `flaky` mark to try again when that happens. Fixes: #778
Configuration menu - View commit details
-
Copy full SHA for f8563e9 - Browse repository at this point
Copy the full SHA f8563e9View commit details