[Bug] exclude-test-regex feature not working

[TylerHelmuth]

chainsaw version Version

v0.2.0

Description

I'm running into an issue where a value for --exclude-test-regex is recognized, but not honor. You can see https://github.com/open-telemetry/opentelemetry-helm-charts/actions/runs/9024018597/job/24797346933?pr=1179 that the configuration sees ExcludeTestRegex '.*multiple-configmaps.*', but the chainsaw/multiple-configmaps test was still run.

The test definition lives here: https://github.com/open-telemetry/opentelemetry-operator/tree/main/tests/e2e/multiple-configmaps

Steps to reproduce

See open-telemetry/opentelemetry-helm-charts#1179

Expected behavior

The test matching that regex would not be run

Screenshots

No response

Logs

No response

Slack discussion

No response

Troubleshooting

• I have searched other issues in this repository and mine is not recorded.

[TylerHelmuth]

It seems that the exact test name (--exclude-test-regex "chainsaw/multiple-configmaps") works.

[eddycharly]

This is the way it works in golang test framework (test name is split against /, and every part is matched with the corresponding part in the provided regexes).

Try with .*/.*multiple-configmaps.* it should work:

• .* will match with chainsaw
• .*multiple-configmaps.* will match with multiple-configmaps

I agree it's confusing but it's how golang testing package implements it 🤷

[TylerHelmuth]

Oh I didn't realize it was using Go's test framework, that's a good clarification to add to the flag description.

[eddycharly]

Oh I didn't realize it was using Go's test framework, that's a good clarification to add to the flag description.

Do you want to open a pr ?

[eddycharly]

Closing, not a bug.