Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threadpool improvements #298

Merged
merged 3 commits into from
Jun 1, 2023
Merged

Threadpool improvements #298

merged 3 commits into from
Jun 1, 2023

Conversation

wirew0rm
Copy link
Member

@wirew0rm wirew0rm commented Jun 1, 2023

Backport of changes from graph prototype. See original issue:
fair-acc/gnuradio4#67

Some noteworthy changes:

Also enables execution of emscripten unittests via ctest and unbreak failing emscripten tests.

@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:36 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:36 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:36 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:36 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:36 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:36 — with GitHub Actions Inactive
Backport of changes from graph prototype. See original issue:
fair-acc/gnuradio4#67

Some noteworthy changes:
- emscripten compatibility: std::jthread -> std::thread, ranges usages
- fix race condition on thread shutdown
- fix #243 (starting and stopping threads in ThreadPool)
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:46 — with GitHub Actions Inactive
Enables execution of emscripten unittests via ctest and unbreak failing
emscripten tests.
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:52 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:52 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:52 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:52 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:52 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 08:52 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 09:48 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 09:48 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 09:48 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 09:48 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 09:48 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 09:48 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 09:48 — with GitHub Actions Inactive
Copy link
Member

@RalphSteinhagen RalphSteinhagen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linked to fair-acc/gnuradio4#67

Thanks for enabling the WASM test work and following up on the ThreadPool adjustments. Nice!

Try to prevent negative hit counts in multithreaded instrumended code.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68080#c4
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 11:24 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 11:24 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 11:24 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 11:24 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 11:24 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 11:24 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 11:24 — with GitHub Actions Inactive
@wirew0rm wirew0rm temporarily deployed to configure coverage June 1, 2023 11:24 — with GitHub Actions Inactive
@codecov
Copy link

codecov bot commented Jun 1, 2023

Codecov Report

Patch coverage: 47.91% and project coverage change: +0.40 🎉

Comparison is base (418003b) 55.29% compared to head (3fb355a) 55.70%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #298      +/-   ##
==========================================
+ Coverage   55.29%   55.70%   +0.40%     
==========================================
  Files          68       65       -3     
  Lines        7216     7174      -42     
  Branches     2667     2613      -54     
==========================================
+ Hits         3990     3996       +6     
- Misses       1526     1527       +1     
+ Partials     1700     1651      -49     
Impacted Files Coverage Δ
src/client/include/ClientContext.hpp 55.81% <ø> (ø)
src/client/include/RestClientNative.hpp 45.41% <ø> (-0.53%) ⬇️
src/majordomo/include/majordomo/Settings.hpp 100.00% <ø> (ø)
src/core/include/Debug.hpp 43.58% <7.69%> (ø)
src/majordomo/include/majordomo/RestBackend.hpp 17.53% <8.10%> (-0.02%) ⬇️
src/core/include/ThreadAffinity.hpp 59.57% <14.28%> (ø)
src/majordomo/include/majordomo/Broker.hpp 53.10% <34.50%> (+5.74%) ⬆️
src/client/include/Client.hpp 42.42% <41.66%> (+4.50%) ⬆️
src/client/include/MockServer.hpp 58.44% <46.15%> (+1.68%) ⬆️
src/majordomo/include/majordomo/MockClient.hpp 51.51% <46.15%> (+8.43%) ⬆️
... and 5 more

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@wirew0rm wirew0rm merged commit bcbe993 into main Jun 1, 2023
@wirew0rm wirew0rm deleted the threadpoolImprovements branch June 1, 2023 11:53
@wirew0rm
Copy link
Member Author

wirew0rm commented Jun 1, 2023

Added -fprofile-update=atomic to the coverage compiler options, see documentation: https://gcc.gnu.org/onlinedocs/gcc-9.3.0/gcc/Instrumentation-Options.html, to prevent the problem of negative coverage line hits due to concurrent coverage counter updates as described in the issue linked from the error message:

 gcovr.gcov_parser.NegativeHits: Got negative hit value in gcov line 'branch  1 taken -1' caused by a bug in gcov tool, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68080. Use option --gcov-ignore-parse-errors with a value of negative_hits.warn, or negative_hits.warn_one.

https://github.com/fair-acc/opencmw-cpp/actions/runs/5142318355/jobs/9257131141#step:14:1360

Since this is not deterministic, I'm not completely sure that this will fix the issue. According to the documentation it gcc should automatically select atomic if -lpthread is used, but is not completely clear about whether this applies generally or only for -fprofile-update=maybe-atomic. If it reoccurs in the future the other possible problem would be unjoined threads which break coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[0pts] Fix starting and stopping threads in ThreadPool
2 participants