Fix test-n-publish

[soininen]

This PR fixes the test-n-publish workflow.

• The skip tag did not work
• Packages cannot be published from reusable workflows, see Trusted publishing: Support for GitHub reusable workflows pypi/warehouse#11096

The publishing step still fails but that is because uploads to PyPI are currently disabled if I understood correctly.

[suvayu]

I think there's a bit of misunderstanding here, my fault really for not documenting 🤐

The difficulty with the test step is that we need to test the new wheels with it's new dependencies, and it could be one of the other wheels that need testing. Consider the following cases, you are releasing:

1. spinedb-api, spine-engine
   • spinedb-api, spine-engine: come from wheel
   • spine-items, spine-toolbox: come from PyPI
2. spine-engine
   • spine-engine: comes from wheel
   • spinedb-api, spine-items, spine-toolbox: come from PyPI
3. spinedb-api, spine-engine, spine-items, spine-toolbox; all come from wheels

So instead of figuring all of this out, I download all the wheels and install all of them for all packages, that way, when a package is excluded, it gets pulled in from PyPI.

And now to run the tests, we checkout the projects in a sub-directory, and slightly modify the test discovery command so that the $PWD is correct since we are one directory above.

<working directory>
├── dist
│   ├── Spine-Database-API
│   ├── spine-engine
│   ├── spine-items
│   └── Spine-Toolbox
└── <pkg-being-tested>

As for the reusable workflow issue, good catch! But I don't think we need to change anything in the repo. Our reusable workflow is in the same repo as the main calling workflow, so all we need to do is on PyPI approve publish-wheel.yml instead of test-n-publish.yml. I will have to fix the documentation for this.

Hopefully this makes sense 😬

[suvayu]

To fix the skip tag issue, I think what we can do is in the build step if skip is true, then add unconditional success step as the very last step (after upload). something like this maybe?

    - name: Signal skipping is also success
      if: ${{ inputs[matrix.pkg] }} == 'skip'
      run: true

There might be a cleaner more GH action way of doing this though, WDYT?

[soininen]

I have now added publish-wheel.yml as a publisher to PyPI for all the four projects so hopefully the reusable workflow works next time.

I am OK with trying something simpler that keeps the tests working as intended. I can always come back to this branch if I run into issues next time new versions get published :)

Anyway, PyPI is again accepting new uploads and this time I managed to get the publishing step to succeed. This conductor is pretty nifty once you get it running!

[suvayu]

I am OK with trying something simpler that keeps the tests working as intended.

I'm off until the 12, but I'll try to find some work time to implement this. It's mostly easy to implement, but cumbersome to test :-p

Anyway, PyPI is again accepting new uploads and this time I managed to get the publishing step to succeed.

I was wondering about this, if there's some way to retry after sometime, let's say a day. I think it should be possible because retention period on the artifacts is set to 7 days. I guess this isn't high priority since relaunching the workflow is simple. But maybe an informative error message would be useful.

This conductor is pretty nifty once you get it running!

Thanks a lot for patiently iterating :-p The next major piece would be if we can also generate your pyinstaller bundle, then I would consider this feature complete (for now :-p).

[suvayu]

close in favour of #27