Fix issue 3243

[isaacnguyen0809]

Pull request (PR) checklist

Please check if your pull request fulfills the following requirements:

• The PR is submitted to the main branch.
• I've read the Contribution Guidelines and my PR doesn't break the rules.
• I've read and understand the Developer Guidelines.
• I confirm that I've run the code locally and everything works as expected.
• My PR includes only the necessary changes to fix the issue (i.e., no unnecessary files or lines of code are changed).
• 🎬 I've attached a screen recording of using the new code to the next paragraph (if applicable).

Description

• Let's check the integration first and if it's fine, i will start write the docs for it

What's changed?

• Migrate Paparzzi testing to Compose preveview screenshot testing
• Remove Paparazzi integration

Risk factors

...

Does this PR close any GitHub issues?

• Closes Migrate Paparazzi to Compose Preview Screenshot testing #3243

Troubleshooting CI failures ❌

Pull request checks failing? Read our CI Troubleshooting guide.

[ILIYANGERMANOV]

LGTM but we need to improve a few bits:

1. Build gradle setup must be just applying ivy.feature w/o additional configuration
2. Ideally, we should be able to apply @IvyPreviews to a regular preview in the main source set and it to work.
3. The screenshots must be both in Light & Dark mode

As a rule of thumb, the simpler the dev experience the better. In an ideal world, as a dev I want to get screenshot tests by just making a preview with @IvyPreview

@isaacnguyen0809 just to confirm, we don't use Git LFS, right? Because it wants us to pay

[ILIYANGERMANOV]

@isaacnguyen0809 can you also change the name of the GitHub Actions workflow to "Screenshot tests"

[isaacnguyen0809]

Done

[isaacnguyen0809]

@ILIYANGERMANOV I have just updated the CI docs. Please review the entire PR to see if any further adjustments are needed. Let me know.

[ILIYANGERMANOV]

@ILIYANGERMANOV I have just updated the CI docs. Please review the entire PR to see if any further adjustments are needed. Let me know.

Hey @isaacnguyen0809 the CI isn't passing. Fix that I guess we can merge

[isaacnguyen0809]

@ILIYANGERMANOV
After taking the time to check why the CI didn't pass Screenshots tests , I run it locally and there was no problem, but when running on github_action it didn't pass.
I modified the CI to run it on a module :screen:attributions and created an additional job to upload the reports file, this is the result.

https://github.com/Ivy-Apps/ivy-wallet/actions/runs/10005833216/job/27657304552?pr=3338

Artifact download URL: https://github.com/Ivy-Apps/ivy-wallet/actions/runs/10005833216/artifacts/1718465769

You can check reports in screenshotTest/preview/debug/index.html I don't know why it produces different results, even though it looks correct, could it be due to anti-aliasing?
If that's the reason, then I think we can't do anything more.

[ILIYANGERMANOV]

@isaacnguyen0809 I'm mobile only and can't investigate but here are a few reasons why it passes locally but fails on CI:

1. Flaky previews: common culprits LocalDate.now(), Locale.default() and all other device-related stuff
2. Again why do we need the damn screenshotDebug source set? I don't like it and it complicates matters without obvious benefit

Suggestions:

• Check the GitHub Actions logs and try to understand what's different?
• Is there a single screenshot test that passes successfully on the CI?
  • If yes, then the setup is correct and the failing tests are flaky and must be removed or fixed.
  • If no, the setup is broken

[ILIYANGERMANOV]

@isaacnguyen0809 to fix the CI can you try regenerating all screenshots? Also did you update the Paparazzi scripts in the scripts folder?

[isaacnguyen0809]

@isaacnguyen0809 to fix the CI can you try regenerating all screenshots? Also did you update the Paparazzi scripts in the scripts folder?

I have done it all. I also tried setting it up to run on one module to see where the problem was. And the result is like the link I attached above. I also just found a ticket that seems to be similar to the problem we are having https://issuetracker.google.com/issues/348590914?pli=1

You can see that they are almost the same but they will not be able to pass the test. So I think the problem is due to rendering on each environment. Maybe that's why we failed

[ILIYANGERMANOV]

@isaacnguyen0809 to fix the CI can you try regenerating all screenshots? Also did you update the Paparazzi scripts in the scripts folder?

I have done it all. I also tried setting it up to run on one module to see where the problem was. And the result is like the link I attached above. I also just found a ticket that seems to be similar to the problem we are having https://issuetracker.google.com/issues/348590914?pli=1

You can see that they are almost the same but they will not be able to pass the test. So I think the problem is due to rendering on each environment. Maybe that's why we failed

Got it. Paparazzi has a way to set a % difference tolerance. Can we do (the same with Google's thing? It would be a shame if they don't support it (sorry that I'm reading between the lines)

[isaacnguyen0809]

@ILIYANGERMANOV Maybe we have to wait for them to develop it, there's already a feature request about it haha. We probably have to postpone this PR

[ILIYANGERMANOV]

@ILIYANGERMANOV Maybe we have to wait for them to develop it, there's already a feature request about it haha. We probably have to postpone this PR

Is there any known workaround for this? Maybe we can somehow avoid it. If it's just this screen - feel free to tweak the UI or remove the test

[isaacnguyen0809]

It doesn't pass any tests, this is just me modifying the command to run CI on this module and get the report to check. But I'll try again tomorrow and see if there's a way. I have to go to sleep now haha. Btw, wish you happy weekend

[ILIYANGERMANOV]

It doesn't pass any tests, this is just me modifying the command to run CI on this module and get the report to check. But I'll try again tomorrow and see if there's a way. I have to go to sleep now haha. Btw, wish you happy weekend

Thanks for the amazing work! No rush^ Happy weekend, too!

[ILIYANGERMANOV]

I'll remove the "keep" feel free to reopen this awesome work once Google fixes the issue