Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Add discrepancy testing for halld_recon #337

Open
nathanwbrei opened this issue Aug 9, 2024 · 0 comments
Open

CI: Add discrepancy testing for halld_recon #337

nathanwbrei opened this issue Aug 9, 2024 · 0 comments

Comments

@nathanwbrei
Copy link
Collaborator

nathanwbrei commented Aug 9, 2024

In JANA1, users use the janadump or hd_dump plugins to inspect data event-by-event. Right now, we are manually using hd_dump for discrepancy testing our halld_recon port.

In JANA2, we have several different tools that capture different pieces of the necessary functionality but definitely need reorganization:

  • The ported hd_dump
  • JInspector
  • regressiontest
  • JCsvWriter

We also should consider whether we can simply reuse ePIC's capybara tool for detecting discrepancies instead.

Requirements:

  • Experiment-agnostic. Leverages JObject::Summarize to incorporate users' preferred formatting and also be consistent with janaview.
  • Accessible from inside a debugger (without restarting the process)
  • Accessible when running in interactive mode (i.e. JApplicationInspector)
  • Outputs ASCII tables, pretty-printed JSON, or CSV, depending on user config
  • Outputs to a separate file, not to stdout, so that other log messages don't pollute our diff
  • Works with topologies that use multiple event levels (e.g. TimesliceExample).
  • Sorts outputs by event number so that output is deterministic even when multithreading is enabled

Nice-to-haves:

  • Works with PODIO data (without needing to know about the exact PODIO datamodel)

Obvious problems with the existing tools:

  • JCsvWriter only works on one collection at a time and can't be made into its own plugin due to how it uses templates. This is an easy fix, although JCsvWriter should probably be outright replaced by a subset of JInspector anyhow.
  • JEventSource implements nevents/nskip in a way which makes all of these tools very inefficient. Adding random file access and O(1) nskip will fix this.
  • regressiontest attempts to do a diff itself, which it probably shouldn't
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant