Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Action inputs to dispatch n-runs of a single functional test #6297

Open
wants to merge 76 commits into
base: main
Choose a base branch
from

Conversation

carlydf
Copy link
Contributor

@carlydf carlydf commented Jul 16, 2024

What changed?

Add workflow dispatch options to the functional tests Github Action to allow us to run n-iterations of a single functional test with a configurable timeout.

Why?

To aid in the diagnosis and treatment of flaky tests.

How did you test it?

Tested in github actions.
Here is the action run normally, with no new input parameters: https://github.com/temporalio/temporal/actions/runs/9962219294

Here is the action run on one test multiple times: https://github.com/temporalio/temporal/actions/runs/9962244076

Potential risks

Our functional test pipeline could be broken by this PR, but we would notice that pretty immediately

Documentation

Is hotfix candidate?

@carlydf carlydf changed the title [DNR] Action to dispatch n-runs of a single functional test Action inputs to dispatch n-runs of a single functional test Jul 16, 2024
.github/workflows/run-tests.yml Outdated Show resolved Hide resolved
.github/workflows/run-tests.yml Outdated Show resolved Hide resolved
@rodrigozhou
Copy link
Collaborator

Can we create a separate workflow file for this purpose?
run-tests.yml is complicated enough that adding this seems added complexity unnecessarily.
Also, as you noted, running misc-checks is also unnecessary when running a single test.

@dnr
Copy link
Member

dnr commented Jul 20, 2024

Can we create a separate workflow file for this purpose? run-tests.yml is complicated enough that adding this seems added complexity unnecessarily. Also, as you noted, running misc-checks is also unnecessary when running a single test.

A separate workflow file that we run only rarely will rot and be likely to be broken when someone wants to use it. Integrating it into the main one makes it much more likely to be maintained and working

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants