From 9c28aa30eb3a70c46b749f86379abaf9b27a3d8e Mon Sep 17 00:00:00 2001 From: Marco Donadoni Date: Wed, 7 Feb 2024 18:03:00 +0100 Subject: [PATCH] refactor(docs): move from reST to Markdown (#50) Convert docs from reStructuredText to Markdown so that the changelog file is compatible with Release Please. --- AUTHORS.md | 16 ++++ AUTHORS.rst | 17 ---- README.md | 217 ++++++++++++++++++++++++++++++++++++++++++++++ README.rst | 245 ---------------------------------------------------- 4 files changed, 233 insertions(+), 262 deletions(-) create mode 100644 AUTHORS.md delete mode 100644 AUTHORS.rst create mode 100644 README.md delete mode 100644 README.rst diff --git a/AUTHORS.md b/AUTHORS.md new file mode 100644 index 0000000..65c202a --- /dev/null +++ b/AUTHORS.md @@ -0,0 +1,16 @@ +# Authors + +The list of contributors in alphabetical order: + +- [Audrius Mecionis](https://orcid.org/0000-0002-3759-1663) +- [Daniel Prelipcean](https://orcid.org/0000-0002-4855-194X) +- [Diego Rodriguez](https://orcid.org/0000-0003-0649-2002) +- [Giuseppe Steduto](https://orcid.org/0009-0002-1258-8553) +- [Jan Okraska](https://orcid.org/0000-0002-1416-3244) +- [Lukas Heinrich](https://orcid.org/0000-0002-4048-7584) +- [Marco Donadoni](https://orcid.org/0000-0003-2922-5505) +- [Marco Vidal](https://orcid.org/0000-0002-9363-4971) +- [Rokas Maciulaitis](https://orcid.org/0000-0003-1064-6967) +- [Ronald Dobos](https://orcid.org/0000-0003-2914-000X) +- [Tibor Simko](https://orcid.org/0000-0001-7202-5803) +- [Vladyslav Moisieienkov](https://orcid.org/0000-0001-9717-0775) diff --git a/AUTHORS.rst b/AUTHORS.rst deleted file mode 100644 index 3892132..0000000 --- a/AUTHORS.rst +++ /dev/null @@ -1,17 +0,0 @@ -Authors -======= - -The list of contributors in alphabetical order: - -- `Audrius Mecionis `_ -- `Daniel Prelipcean `_ -- `Diego Rodriguez `_ -- `Giuseppe Steduto `_ -- `Jan Okraska `_ -- `Lukas Heinrich `_ -- `Marco Donadoni `_ -- `Marco Vidal `_ -- `Rokas Maciulaitis `_ -- `Ronald Dobos `_ -- `Tibor Simko `_ -- `Vladyslav Moisieienkov `_ diff --git a/README.md b/README.md new file mode 100644 index 0000000..b53b026 --- /dev/null +++ b/README.md @@ -0,0 +1,217 @@ +# REANA example - ATLAS RECAST + +[![image](https://github.com/reanahub/reana-demo-atlas-recast/workflows/CI/badge.svg)](https://github.com/reanahub/reana-demo-atlas-recast/actions) +[![image](https://img.shields.io/badge/discourse-forum-blue.svg)](https://forum.reana.io) +[![image](https://img.shields.io/github/license/reanahub/reana-demo-atlas-recast.svg)](https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/LICENSE) +[![image](https://www.reana.io/static/img/badges/launch-on-reana-at-cern.svg)](https://reana.cern.ch/launch?url=https%3A%2F%2Fgithub.com%2Freanahub%2Freana-demo-atlas-recast&name=reana-demo-atlas-recast) + +## About + +This [REANA](http://www.reana.io/) reproducible analysis example demonstrates a +[RECAST](https://arxiv.org/abs/1010.2506) analysis using [ATLAS](https://atlas.cern/) +Analysis Software Group stack. + +## Analysis structure + +Making a research data analysis reproducible basically means to provide "runnable +recipes" addressing (1) where is the input data, (2) what software was used to analyse +the data, (3) which computing environments were used to run the software and (4) which +computational workflow steps were taken to run the analysis. This will permit to +instantiate the analysis on the computational cloud and run the analysis to obtain (5) +output results. + +### 1. Input data + +The analysis takes the following inputs: + +- `dxaod` input ROOT file +- `did` dataset ID e.g. 404958 +- `xsec_in_pb` cross section in picobarn e.g. 0.00122 + +### 2. Analysis code + +The **event selection** code for this analysis example resides under the +[eventselection](eventselection) subdirectory. It uses the official analysis releases +prepared by the ATLAS Analysis Software Group (ASG). + +- `eventselection/CMakeLists.txt` +- `eventselection/MyEventSelection/CMakeLists.txt` +- `eventselection/MyEventSelection/MyEventSelection/MyEventSelectionAlg.h` +- `eventselection/MyEventSelection/Root/LinkDef.h` +- `eventselection/MyEventSelection/Root/MyEventSelectionAlg.cxx` +- `eventselection/MyEventSelection/util/myEventSelection.cxx` + +The **statistical analysis** code for this analysis example resides in +[statanalysis](statanalysis) subdirectory. It implements limit setting for outputs +produced by the event selection package. + +- `statanalysis/data/background.root` +- `statanalysis/data/data.root` +- `statanalysis/make_ws.py` +- `statanalysis/plot.py` +- `statanalysis/set_limit.py` + +Notes that `make_ws.py` script generates a HistFactory configuration based on signal, +data and background ROOT files. It performs a simple HistFactory-based fit based on a +single channel (consisting of two bins). + +### 3. Compute environment + +In order to be able to rerun the analysis even several years in the future, we need to +"encapsulate the current compute environment", for example to freeze the ATLAS software +version our analysis is using. We shall achieve this by preparing a +[Docker](https://www.docker.com/) container image for our analysis steps. + +The event selection stage uses official ATLAS `atlas/analysisbase` container on top of +which we add and build our custom code: + +```console +$ less eventselection/Dockerfile +FROM docker.io/atlas/analysisbase:21.2.51 +ADD . /analysis/src +WORKDIR /analysis/build +RUN source ~/release_setup.sh && \ + sudo chown -R atlas /analysis && \ + sudo chmod -R 775 /analysis && \ + sudo chmod -R 755 /home/atlas && \ + cmake ../src && \ + make -j4 +USER root +RUN sudo usermod -G root atlas +USER atlas +``` + +We can build our event selection analysis environment image and give it a name +`docker.io/reanahub/reana-demo-atlas-recast-eventselection`: + +```console +$ cd eventselection +$ docker build -t docker.io/reanahub/reana-demo-atlas-recast-eventselection . +``` + +The statistical analysis stage also extends `atlas/analysisbase` by the custom code: + +```console +$ less statanalysis/Dockerfile +FROM docker.io/atlas/analysisbase:21.2.51 +ADD . /code +RUN sudo sh -c "source /home/atlas/release_setup.sh && pip install hftools==0.0.6" +USER root +RUN sudo usermod -G root atlas && sudo chmod -R 755 /home/atlas && chmod -R 775 /code +USER atlas +``` + +We can build our statistical analysis environment image and give it a name +`docker.io/reanahub/reana-demo-atlas-recast-statanalysis`: + +```console +$ cd statanalysis +$ docker build -t docker.io/reanahub/reana-demo-atlas-recast-statanalysis . +``` + +We can upload both images to the DockerHub image registry: + +```console +$ docker push docker.io/reanahub/reana-demo-atlas-recast-eventselection +$ docker push docker.io/reanahub/reana-demo-atlas-recast-statanalysis +``` + +(Note that typically you would use your own username such as `johndoe` in place of +`reanahub`.) + +### 4. Analysis workflow + +This analysis example consists of a simple workflow where event selection is run first +and its output serve as an input for the statistical analysis. + +We shall use the [Yadage](https://github.com/yadage) workflow engine to express the +computational steps in a declarative manner: + +![](https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/docs/workflow.png){.align-center} + +The full analysis pipeline is defined in [workflow.yml](workflow/workflow.yml) and the +individual steps are defined in [steps.yml](workflow/steps.yml). + +### 5. Output results + +The analysis produces several pre-fit and post-fit plots: + +![](https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/statanalysis/example_results/pre.png){.align-center} + +![](https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/statanalysis/example_results/post.png){.align-center} + +The limit plot: + +![](https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/statanalysis/example_results/limit.png){.align-center} + +The limit data is also stored in JSON format for both an entire µ-scan as well as for +µ=1. + +## Running the example on REANA cloud + +There are two ways to execute this analysis example on REANA. + +If you would like to simply launch this analysis example on the REANA instance at CERN +and inspect its results using the web interface, please click on the following badge: + +```{=html} + + + +

+``` + +If you would like a step-by-step guide on how to use the REANA command-line client to +launch this analysis example, please read on. + +We start by creating a [reana.yaml](reana.yaml) file describing the above analysis +structure with its inputs, code, runtime environment, computational workflow steps and +expected outputs: + +```yaml +version: 0.3.0 +inputs: + parameters: + did: 404958 + xsec_in_pb: 0.00122 + dxaod_file: https://recastwww.web.cern.ch/recastwww/data/reana-recast-demo/mc15_13TeV.123456.cap_recast_demo_signal_one.root +workflow: + type: yadage + file: workflow/workflow.yml +outputs: + files: + - outputs/statanalysis/fitresults/pre.png + - outputs/statanalysis/fitresults/post.png + - outputs/statanalysis/fitresults/limit.png + - outputs/statanalysis/fitresults/limit_data.json +``` + +We can now install the REANA command-line client, run the analysis and download the +resulting plots: + +```console +$ # create new virtual environment +$ virtualenv ~/.virtualenvs/reana +$ source ~/.virtualenvs/reana/bin/activate +$ # install REANA client +$ pip install reana-client +$ # connect to some REANA cloud instance +$ export REANA_SERVER_URL=https://reana.cern.ch/ +$ export REANA_ACCESS_TOKEN=XXXXXXX +$ # create new workflow +$ reana-client create -n myanalysis +$ export REANA_WORKON=myanalysis +$ # upload input code, data and workflow to the workspace +$ reana-client upload +$ # start computational workflow +$ reana-client start +$ # ... should be finished in about a minute +$ reana-client status +$ # list workspace files +$ reana-client ls +$ # download output results +$ reana-client download +``` + +Please see the [REANA-Client](https://reana-client.readthedocs.io/) documentation for +more detailed explanation of typical `reana-client` usage scenarios. diff --git a/README.rst b/README.rst deleted file mode 100644 index 17442d7..0000000 --- a/README.rst +++ /dev/null @@ -1,245 +0,0 @@ -============================== - REANA example - ATLAS RECAST -============================== - -.. image:: https://github.com/reanahub/reana-demo-atlas-recast/workflows/CI/badge.svg - :target: https://github.com/reanahub/reana-demo-atlas-recast/actions - -.. image:: https://badges.gitter.im/Join%20Chat.svg - :target: https://gitter.im/reanahub/reana?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge - -.. image:: https://img.shields.io/github/license/reanahub/reana-demo-atlas-recast.svg - :target: https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/LICENSE - -.. image:: https://www.reana.io/static/img/badges/launch-on-reana-at-cern.svg - :target: https://reana.cern.ch/launch?url=https%3A%2F%2Fgithub.com%2Freanahub%2Freana-demo-atlas-recast&name=reana-demo-atlas-recast - -About -===== - -This `REANA `_ reproducible analysis example demonstrates -a `RECAST `_ analysis using `ATLAS -`_ Analysis Software Group stack. - -Analysis structure -================== - -Making a research data analysis reproducible basically means to provide -"runnable recipes" addressing (1) where is the input data, (2) what software was -used to analyse the data, (3) which computing environments were used to run the -software and (4) which computational workflow steps were taken to run the -analysis. This will permit to instantiate the analysis on the computational -cloud and run the analysis to obtain (5) output results. - -1. Input data -------------- - -The analysis takes the following inputs: - -- ``dxaod`` input ROOT file -- ``did`` dataset ID e.g. 404958 -- ``xsec_in_pb`` cross section in picobarn e.g. 0.00122 - -2. Analysis code ----------------- - -The **event selection** code for this analysis example resides under the -`eventselection `_ subdirectory. It uses the official analysis -releases prepared by the ATLAS Analysis Software Group (ASG). - -- ``eventselection/CMakeLists.txt`` -- ``eventselection/MyEventSelection/CMakeLists.txt`` -- ``eventselection/MyEventSelection/MyEventSelection/MyEventSelectionAlg.h`` -- ``eventselection/MyEventSelection/Root/LinkDef.h`` -- ``eventselection/MyEventSelection/Root/MyEventSelectionAlg.cxx`` -- ``eventselection/MyEventSelection/util/myEventSelection.cxx`` - -The **statistical analysis** code for this analysis example resides in -`statanalysis `_ subdirectory. It implements limit setting for -outputs produced by the event selection package. - -- ``statanalysis/data/background.root`` -- ``statanalysis/data/data.root`` -- ``statanalysis/make_ws.py`` -- ``statanalysis/plot.py`` -- ``statanalysis/set_limit.py`` - -Notes that ``make_ws.py`` script generates a HistFactory configuration based on -signal, data and background ROOT files. It performs a simple HistFactory-based -fit based on a single channel (consisting of two bins). - -3. Compute environment ----------------------- - -In order to be able to rerun the analysis even several years in the future, we -need to "encapsulate the current compute environment", for example to freeze the -ATLAS software version our analysis is using. We shall achieve this by preparing -a `Docker `_ container image for our analysis steps. - -The event selection stage uses official ATLAS ``atlas/analysisbase`` container -on top of which we add and build our custom code: - -.. code-block:: console - - $ less eventselection/Dockerfile - FROM docker.io/atlas/analysisbase:21.2.51 - ADD . /analysis/src - WORKDIR /analysis/build - RUN source ~/release_setup.sh && \ - sudo chown -R atlas /analysis && \ - sudo chmod -R 775 /analysis && \ - sudo chmod -R 755 /home/atlas && \ - cmake ../src && \ - make -j4 - USER root - RUN sudo usermod -G root atlas - USER atlas - -We can build our event selection analysis environment image and give it a name -``docker.io/reanahub/reana-demo-atlas-recast-eventselection``: - -.. code-block:: console - - $ cd eventselection - $ docker build -t docker.io/reanahub/reana-demo-atlas-recast-eventselection . - -The statistical analysis stage also extends ``atlas/analysisbase`` by the custom -code: - -.. code-block:: console - - $ less statanalysis/Dockerfile - FROM docker.io/atlas/analysisbase:21.2.51 - ADD . /code - RUN sudo sh -c "source /home/atlas/release_setup.sh && pip install hftools==0.0.6" - USER root - RUN sudo usermod -G root atlas && sudo chmod -R 755 /home/atlas && chmod -R 775 /code - USER atlas - -We can build our statistical analysis environment image and give it a name -``docker.io/reanahub/reana-demo-atlas-recast-statanalysis``: - -.. code-block:: console - - $ cd statanalysis - $ docker build -t docker.io/reanahub/reana-demo-atlas-recast-statanalysis . - -We can upload both images to the DockerHub image registry: - -.. code-block:: console - - $ docker push docker.io/reanahub/reana-demo-atlas-recast-eventselection - $ docker push docker.io/reanahub/reana-demo-atlas-recast-statanalysis - -(Note that typically you would use your own username such as ``johndoe`` in -place of ``reanahub``.) - -4. Analysis workflow --------------------- - -This analysis example consists of a simple workflow where event selection is run -first and its output serve as an input for the statistical analysis. - -We shall use the `Yadage `_ workflow engine to -express the computational steps in a declarative manner: - -.. figure:: https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/docs/workflow.png - :alt: workflow.png - :align: center - -The full analysis pipeline is defined in `workflow.yml `_ -and the individual steps are defined in `steps.yml `_. - -5. Output results ------------------ - -The analysis produces several pre-fit and post-fit plots: - -.. figure:: https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/statanalysis/example_results/pre.png - :alt: pre.png - :align: center - -.. figure:: https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/statanalysis/example_results/post.png - :alt: post.png - :align: center - -The limit plot: - -.. figure:: https://raw.githubusercontent.com/reanahub/reana-demo-atlas-recast/master/statanalysis/example_results/limit.png - :alt: limit.png - :align: center - -The limit data is also stored in JSON format for both an entire µ-scan as well -as for µ=1. - -Running the example on REANA cloud -================================== - -There are two ways to execute this analysis example on REANA. - -If you would like to simply launch this analysis example on the REANA instance -at CERN and inspect its results using the web interface, please click on -the following badge: - -.. raw:: html - - - - -

- -If you would like a step-by-step guide on how to use the REANA command-line -client to launch this analysis example, please read on. - -We start by creating a `reana.yaml `_ file describing the above -analysis structure with its inputs, code, runtime environment, computational -workflow steps and expected outputs: - -.. code-block:: yaml - - version: 0.3.0 - inputs: - parameters: - did: 404958 - xsec_in_pb: 0.00122 - dxaod_file: https://recastwww.web.cern.ch/recastwww/data/reana-recast-demo/mc15_13TeV.123456.cap_recast_demo_signal_one.root - workflow: - type: yadage - file: workflow/workflow.yml - outputs: - files: - - outputs/statanalysis/fitresults/pre.png - - outputs/statanalysis/fitresults/post.png - - outputs/statanalysis/fitresults/limit.png - - outputs/statanalysis/fitresults/limit_data.json - -We can now install the REANA command-line client, run the analysis and download -the resulting plots: - -.. code-block:: console - - $ # create new virtual environment - $ virtualenv ~/.virtualenvs/reana - $ source ~/.virtualenvs/reana/bin/activate - $ # install REANA client - $ pip install reana-client - $ # connect to some REANA cloud instance - $ export REANA_SERVER_URL=https://reana.cern.ch/ - $ export REANA_ACCESS_TOKEN=XXXXXXX - $ # create new workflow - $ reana-client create -n myanalysis - $ export REANA_WORKON=myanalysis - $ # upload input code, data and workflow to the workspace - $ reana-client upload - $ # start computational workflow - $ reana-client start - $ # ... should be finished in about a minute - $ reana-client status - $ # list workspace files - $ reana-client ls - $ # download output results - $ reana-client download - -Please see the `REANA-Client `_ -documentation for more detailed explanation of typical ``reana-client`` usage -scenarios.