Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transfer and improve Transactions #99

Merged
merged 7 commits into from
Sep 29, 2023

Conversation

vimpostor
Copy link
Contributor

@vimpostor vimpostor commented Jun 26, 2023

I am opening this PR already, so that the API can be reviewed.

Some things had to be changed from the opencmw implementation to make it more generic. Specifically TimingCtx requires a predicate both for matching and for parsing, we can't hardcode the FAIR routines anymore.
And instead of hardcoding members for BPCID, SID, BPID and GID we allow to use a generic identifier map (type is pmtv::map_t, which is a std::mapfrom std::string to pmt).

The final thing that is still WIP is the port of CtxSetting and the reason why it is not easy to port, is that it needs to default-construct TimingCtx instances, which is a bit tricky to get right with the lambdas needed for TimingCtx.
The easy way out of this would be to use std::function instead, but I want to avoid it due to performance reasons.

At the bottom of test/qa_settings.cpp there is an example of the problem, where the compilation fails. The example in the "SettingBase constructors"_test works however, because the lambda types are matching. But if we use a different lambda, the type system will complain.

I tried to provide templated default predicates for the default-values in TimingCtx like this (and some variations of it) so that a default predicate is created specifically for every different templated type, but wasn't able to make the type system happy with it:

template<typename T>
auto defaultParse = [](auto) {};
template<typename T>
auto defaultMatch = [](auto, auto) { return true; };

Maybe someone has an idea how to tackle this problem for CtxSetting.

The buffer also needed to be ported from opencmw::RingBuffer to the graph-prototype circular buffer.
That's about it for all the major changes, although there are also a lot of other smaller changes when porting all the different components, for example the cache line size is not hardcoded to 64 anymore and instead we try to use std::hardware_destructive_interference_size when available.

Closes fair-acc/opencmw-cpp#191

@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 26, 2023 07:40 — with GitHub Actions Inactive
@RalphSteinhagen
Copy link
Member

The easy way out of this would be to use std::function instead, but I want to avoid it due to performance reasons.

Will have a look at this. The performance isn't that critical as I initially thought since this code would be only invoked if a tag arrives at the node which is (worst-case) only once every 100 ms. Maybe the use of std::function is OK in this case.

Anyway, having a good modular solution now is better than a perfect at an undefined moment in the future. 👍

Copy link
Member

@RalphSteinhagen RalphSteinhagen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many parts look familiar but I am a bit confused w.r.t. the usage...

The general idea was/is that these transaction-based settings are an alternative drop-in replacement for basic_settings<node> and that is compatible with the following interface:

https://github.com/fair-acc/graph-prototype/blob/8be673430f5fd17b561a8ab09f404aed2726f029/include/settings.hpp#L16-L77

I.e. from the user perspective, the user would set(..., ctx) a given property_map for a given block and SettingsCtx ctx and when a tag passes along where the tag matches the parameters in ctx and time, that the block settings would be swapped w.r.t. the matching ones from the transaction-based settings.

Maybe I am missing something.

Regarding performance -- compared to the previous implementation in OpenCMW -- we changed the settings-change philosophy a bit and only swap-in settings on matching tags which should make the worst-case scenario be something like one settings property_map->struct-reflection-data swap-in every 100 ms. I guess using std::function should be sufficient for that.

Some unnamed/unconstraint template parameters make the code a bit harder to interpret. Maybe it's worthwhile to add some concept-based constraints on these.

The general code structure and qa_tests are thorough! 👍

Did I mention that I like unit-tests.... ;-)

include/utils.hpp Outdated Show resolved Hide resolved
*/
template<std::movable T, std::movable U = T, std::equality_comparable TransactionToken = std::string, std::size_t N_HISTORY = 1024, typename TimeDiff = std::chrono::seconds, int timeOut = -1,
int timeOutTransactions = -1>
requires(fair::meta::is_power2_v<N_HISTORY> && N_HISTORY > 8)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still functionally needed or just a performance optimisation?

Copy link
Contributor Author

@vimpostor vimpostor Jun 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I transfered this requires from the opencmw implementation, I guess its purpose was just performance.

include/utils.hpp Outdated Show resolved Hide resolved
include/reader_writer_lock.hpp Outdated Show resolved Hide resolved
@vimpostor vimpostor temporarily deployed to configure coverage June 27, 2023 09:19 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 27, 2023 09:19 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 27, 2023 09:19 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 27, 2023 09:19 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 27, 2023 09:19 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 27, 2023 09:19 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 27, 2023 09:19 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage June 27, 2023 09:19 — with GitHub Actions Inactive
@vimpostor
Copy link
Contributor Author

@RalphSteinhagen Many thanks for the detailed review. I implemented the matching over multiple rounds as requested. My first thought was also that it feels a bit "workaroundy", but after implementing it, it honestly feels like the best solution.
This is also a great solution for usecases, where no hierarchical matching is needed - so they can just ignore the attempt argument completely.

As discussed above, could you also add the FAIR specific use-case and matcher that would implement the following scenario implemented fair-acc/opencmw-cpp#156

The test has been ported and added, it works flawlessy with the new attempt API.

This allows for hierarchical matching, where in the first round only
very refined matching is performed and then the condition becomes
increasingly looser.
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 28, 2023 16:00 — with GitHub Actions Inactive
Copy link
Member

@RalphSteinhagen RalphSteinhagen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vimpostor excellent! Thanks for making these quick changes! 👍

@vimpostor vimpostor temporarily deployed to configure coverage September 29, 2023 05:34 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 29, 2023 05:34 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 29, 2023 05:34 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 29, 2023 05:34 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 29, 2023 05:34 — with GitHub Actions Inactive
@vimpostor vimpostor temporarily deployed to configure coverage September 29, 2023 05:34 — with GitHub Actions Inactive
@RalphSteinhagen RalphSteinhagen merged commit 8a2f060 into fair-acc:main Sep 29, 2023
13 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[2pt,7pt] Event store: Evaluate the current action settings and extend if needed, transfer from OpenCMW to GR
2 participants