Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail the test if dmesg after check fails #3261

Open
sbertramrh opened this issue Oct 2, 2024 · 3 comments
Open

Fail the test if dmesg after check fails #3261

sbertramrh opened this issue Oct 2, 2024 · 3 comments

Comments

@sbertramrh
Copy link

Currently with version 1.34, the test still passes when the dmesg after check fails.
I think the desired outcome is for the test to fail when dmesg after fails.
And if dmesg before fails the test should end in warning at least, but not pass.

e.g.

14:48:57             01:08:16 pass /general/time/timetests_upstream/timetests_upstream (on default-0) [1/1]
14:48:57             00:00:01     fail dmesg (before-test check)
14:48:57             00:00:01     fail dmesg (after-test check)
...
14:48:59     status: done
14:48:59     Write file '/var/tmp/tmt/run-085/general/time/timetests_upstream/plan/finish/step.yaml'.
14:48:59 
14:48:59 total: 1 test passed
@martinhoyer
Copy link
Collaborator

Duplicate of #3185
Being addressed by #3239, planned in next release.

@psss
Copy link
Collaborator

psss commented Oct 4, 2024

@sbertramrh, could you please have a look at #3239 and verify that the proposed implementation would cover your use case well? Thanks.

@sbertramrh
Copy link
Author

@sbertramrh, could you please have a look at #3239 and verify that the proposed implementation would cover your use case well? Thanks.

I found it hard to understand, but I think this part of the discussion is what is relevant?

        if interpret == ResultInterpret.XFAIL:
            if self.result == ResultOutcome.PASS and checks_failed:
                self.result = ResultOutcome.FAIL

If so then yes.

Also as mentioned in this ticket entry if the before test portion is the only part that fails for the check, the test should be self.result = ResultOutcome.WARN.

I will mention on the other ticket as well thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants