Flat data model fails several minor manual tests

[hslh]

What version of EMX2 are you using (see footer)
v10.109.5-SNAPSHOT on preview server 4114

Describe the bug
I went through the manual tests for the flat data model on the preview server for #4114 . This is the resulting list of things to fix.

• ‘Variables’ is missing from the top ribbon for a network of networks. See $testCatalogue/testNetworkofNetworks. (test 1 step 8) #4234
• Eleanor design: Network page within a network of networks used to show information on Partners, Cohorts/Collections, Variables and Data Sources (or is that now Datasets?).
  See: $testCatalogue/testNetworkofNetworks/networks/testNetwork1. (test 1 step 11/test 2 step 15)
• Eleanor design: ‘About’-page for a network has only website and description, most content is missing (e.g. partners, funding, variables, cohorts).
  See: $testCatalogue/ssr-catalogue/testNetworkofNetworks/about-catalogue. (test 1 step 13/test 2 step 11)
• Eleanor design: On cohort page, the ‘Partners’-section is missing. The ‘Available Data & Samples’-section is there, but it is missing from the sidebar. ’Networks’-section is called ‘Networks’ in sidebar, but ‘Part of Networks’ in the main content screen.
  See: $testCatalogue/ssr-catalogue/testNetwork1/cohorts/testCohort2. (test 3 step 6b/test 5 step 11, 53/test 9 step 26)
  ------ [ ] remove Organisations as a separate heading on the cohort detail page and absorb it into Contributors (as it is in prod) - OR IS THIS A NEW DIVISION OF THE DATA FOR THE ROR ORGANISATIONS? TO BE DISCUSSED.
  ------ [ ] make the heading/menu item for Networks / Part of Networks consistent
  ------ [ ] correct test plan
• fix(catalogue): the ‘repeats’-icon is missing from variables (e.g. the little +10).  #4244
• On variable page, the ‘Definition’-section is called ‘Description’ in the sidebar and ‘Harmonisation Details Per Collection’ is called ‘Harmonisation Details Per Data Source’ in sidebar.
  See: $testCatalogue/ssr-catalogue/testNetwork1/variables/testVarRepeats_-testNetwork1-cdm_1-testNetwork1?keys={"name":"testVarRepeats_","collection":{"id":"testNetwork1"},"dataset":{"name":"cdm_1","collection":{"id":"testNetwork1”}}}. (test 4 step 10)
• fix(catalogue): On variable page, when you click a source variable under ‘Harmonisation Details Per Collection’, nothing happens, no dialog pops up.  #4246
• On cohort page, in the ‘General Design’-section, the ‘Cohort Design’, in this case ‘longitudinal’, isn’t displayed.
  See: $testCatalogue/ssr-catalogue/all/cohorts/testCohort2. (test 5 step 18)
• fix(catalogue): On variables overview page, filtering on variable name ‘asthma’ and selecting the 4 test cohorts doesn’t display any variables.  #4247
• On the variable page for variable ‘asthma_’, there are no numberings anymore, which makes the harmonisation status unclear. It’s just a list of status indicators with the name of the cohort.
  See: $testCatalogue/ssr-catalogue/all/variables/asthma_-LifeCycle-outcome-LifeCycle?keys={"name":"asthma_","collection":{"id":"LifeCycle"},"dataset":{"name":"outcome","collection":{"id":"LifeCycle”}}}. (test 7 step 13)
  EDIT: Pelagie is no longer presented as a column with the repeated variables in the rows, but it's listed in each row next to the harmonisation symbol, without the name/number of the repeat being displayed.
• On the network variables page, the harmonisation matrix is displayed even though the selected topic and cohort have no mappings.
  See: /ssr-catalogue/testNetwork1/variables?page=1&conditions=[{"id":"topics","conditions":[{"name":"Language"},{"name":"Socio-demographic+and+economic+characteristics"}]},{"id":"collections","conditions":[{"name":"testCohort2"}]}]&view=harmonisation . (test 7 step 40, 46)
  EDIT: in test plan 7 step 38 we just have the 'language' filter active --> 4 variables, all cohorts shown, testcohort2 has no mappings at all. Add the filter on testcohort2 --> randomly 1 variable (testVarNoRepeats) is shown (and not all 4), while the list should be empty. And in test plan 7 step 46 we have no filters active other than 1 cohort, testcohort2. 0 variables should be displayed but we get 2 seemingly random variables displayed, with no mappings (testVarNoRepeats and testVarRepeats_).
• On the catalogue contents page, the button on the top ribbon shows ‘Collections (0)’, even though there are 105 collections (the button works fine, though). See: $testCatalogue/ssr-catalogue/all/collections. (test 8 step 11) #4256

ISSUES FOUND IN VERSION v11.11.1-SNAPSHOT

• On the details page for a network (eg in preview: https://emx2.dev.molgenis.org/catalogue-demo/ssr-catalogue/testNetwork1/networks/testNetwork1) the 'chapter' headings on the left hand side are completely different to what we had before. (See comment below) (test 2 step 15) #4254
• Pop-up for 'about statuses' in the harmonisation matrix (eg on a variable detail page such as this in preview) needs to be brought in line with the new labels for what used to be labelled 'cohorts'. Test plan 4 step 11. See comment below. Plus test plan 4 step 16 (slightly different text here).
• The harmonisation matrix and the info per cohort don't match, for some reason. testvarrepeats in preview (here) gives different results in the matrix to what you see below when you click on testcohort3 in the harmonisation details per data source. Test plan 4 step 12, test plan 6 step 6. See comment below.
  --> test data fixen en dan controleren
• The contact page (behind button 'Contact' on the cohort detail page) doesn't seem to have the updates we made in the big release in July, i.e. pop-up with "Name of test cohort 2", Contact, Name entry field, Email entry field, Organisation entry field, Topic dropdown, Message entry field, "or contact us at: [email protected]", Send button" Test plan 5, step 6
  --> testen in acc (emailsettings nodig)
• The start/end year of cohort testcohort2 should be '1955-ongoing', not 'not available' Test plan 5, step 24, and test plan 9 step 10 (different years for cohort 1 but the same issue)
  --> bewust veranderd ivm DCAT
• The headings on the left hand side of the cohort page are missing 'Variables & Topics' in between Contributors and Available Data & Samples. eg on https://emx2.dev.molgenis.org/catalogue-demo/ssr-catalogue/all/cohorts/testCohort2 Test plan 5, step 11
• The text under 'Variables & Topics' is still a placeholder text and needs to be updated or removed. Test plan 5 (no test step for this new block yet)
  --> Blok Variables & Topics wordt eerst weggehaald en daarna goed gedefinieerd en gespecificeerd.
• The comorbidity items in the pop-up for a subcohort are presented in random order but should be in alphabetical/numerical order. Test plan 5 step 45
• When clicking on the arrow or the network name (rather than 'website') for information about a network to which a cohort is linked (Eg name for test network2) the link opens in the same window rather than in a new window. (We decided to have all external links open in a new window to avoid 'losing' the catalogue in your tabs) Clicking on 'website' gives the correct behaviour. Test plan 5, around step 53, test plan 9 around step 27.
• Data access fee and prelinked fields not shown under Access conditions (bug https://github.com/orgs/molgenis/projects/15/views/1?filterQuery=data+access+fee&pane=issue&itemId=68839761) Test plan 5 step 57 (compare test plan 9 step 28, where they ARE shown because they are 'true').
• Linkage options field is also not shown under Access conditions (but this is not a true/false field). Test plan 5 step 57 and test plan 9 step 28.

[EleanorHyde-UMCG]

Going through the test plans again in preview to see if I can tick off any of the points made above (running into test data issues, though, for some points). I've found something else which I will add here below and at the bottom of the list above:

• On the details page for a network (eg in preview: https://preview-emx2-pr-4178.dev.molgenis.org/catalogue-demo/ssr-catalogue/testNetwork1/networks/testNetwork1) the 'chapter' headings on the left hand side are completely different to what we had before. What we had before was:

ACRONYM FOR TESTNETWORK1
Description
Partners
Funding & Acknowledgements
Cohorts
Data Sources
Variables

What we have now is:
ACRONYM FOR TESTNETWORK1
Description
General Design
Population
Organisations
Datasets
Networks
Publications
Funding & Acknowledgements

Items like 'general design' and 'population' sound more like the cohort details page, is this deliberate? Not sure we can make such sweeping statements about networks, which are often a combination of various sets of populations / designs etc.

[EleanorHyde-UMCG]

This item has been partially solved:

• On cohort page, the ‘Partners’-section is missing. The ‘Available Data & Samples’-section is there, but it is missing from the sidebar. ’Networks’-section is called ‘Networks’ in sidebar, but ‘Part of Networks’ in the main content screen.

See: $testCatalogue/ssr-catalogue/testNetwork1/cohorts/testCohort2. (test 3 step 6b/test 5 step 11, 53/test 9 step 26)

Before the big release in July, we absorbed Partners into Contributors (but unfortunately omitted to adjust this test plan 3). However, Organisations should also have been absorbed into Contributors. What we currently have in prod is:

Description
General Design
Population
Contributors
Available Data & Samples
Subpopulations
Collection Events
Datasets
Networks
Publications
Access Conditions
Funding & Acknowledgements
Documentation

So what still needs to be fixed for this item is:

• remove Organisations as a separate heading on the cohort detail page and absorb it into Contributors (as it is in prod)
• make the heading/menu item for Networks / Part of Networks consistent ('part of networks' is new and may have been decided upon within the flat data model group?)

@BrendaHijmans

[EleanorHyde-UMCG]

Pop-up for 'about statuses' in the harmonisation matrix (eg on a variable detail page such as this in preview) needs to be brought in line with the new labels for what used to be labelled 'cohorts'. The current text is this:

Pop up should show 'completed: cohort was able to fully map to the harmonised variables'; 'partially: cohort was able to partially map to the harmonised variable'; 'No data: no harmonisation data is available'.

Test plan 4 step 11

I've added a new item in the list above.

[EleanorHyde-UMCG]

The harmonisation matrix and the info per cohort don't match, for some reason. testvarrepeats in preview (here) gives different results in the matrix to what you see below when you click on testcohort3 in the harmonisation details per data source. Test plan 4 step 12, test plan 6 step 6.

Have added this to the list above.

[EleanorHyde-UMCG]

Data banks and data sources are included in the harmonisation matrix for a network. Do we expect them to have harmonised variables? (test plan 4 step 15, test plan 7 step 33)

@BrendaHijmans

[EleanorHyde-UMCG]

The number of variables for a network is incorrect (I suspect a change made for the network-of-networks works has something to do with this). The test plan has been changed since the big release in July, also for the network of networks, to include another variable ga_lmp (under testcohort2) but this is not picked up in preview.

Overview page of network --> number of variables should be 8, not 7. Clicking on the variables should also give 8 variables, including ga_lmp, which has been mapped by testcohort2 (which belongs to testnetwork1).

Test plan 4, step 5

Let's discuss and check what should be happening here (I wasn't involved in the change of test data). @BrendaHijmans ?

[EleanorHyde-UMCG]

The contact page (behind button 'Contact') doesn't seem to have the updates we made in the big release in July, i.e. pop-up with "Name of test cohort 2", Contact, Name entry field, Email entry field, Organisation entry field, Topic dropdown, Message entry field, "or contact us at: [email protected]", Send button"

Test plan 5, step 6

Have added to list above.

[EleanorHyde-UMCG]

The start/end year of cohort testcohort2 should be '1955-ongoing', not 'not available' Test plan 5, step 24, and test plan 9 step 10 (different years for cohort 1 but the same issue)

Have added to list above.

[EleanorHyde-UMCG]

The headings on the left hand side of the cohort page are missing 'Variables & Topics' in between Contributors and Available Data & Samples. eg on https://preview-emx2-pr-4178.dev.molgenis.org/catalogue-demo/ssr-catalogue/all/cohorts/testCohort2
Test plan 5, step 11

Have added to the list above.

[EleanorHyde-UMCG]

The text under 'Variables & Topics' is still a placeholder text and needs to be updated or removed. Test plan 5 (no test step for this new block yet)

Have added to the list above.

[EleanorHyde-UMCG]

What do we expect to happen when pressing the 'Harmonised variables' button on the cohort page? eg testcohort2 And why does testcohort2 have 299 variables? Need an explanation of what's changed here :-) @BrendaHijmans

[EleanorHyde-UMCG]

The comorbidity items in the pop-up for a subcohort are presented in random order but should be in alphabetical/numerical order. Test plan 5 step 45

Have added to the list above.

[EleanorHyde-UMCG]

When clicking on the arrow or the network name (rather than 'website') for information about a network to which a cohort is linked (Eg name for test network2) the link opens in the same window rather than in a new window. (We decided to have all external links open in a new window to avoid 'losing' the catalogue in your tabs) Clicking on 'website' gives the correct behaviour. Test plan 5, around step 53, test plan 9 around step 27.

Have added this to the list.

[EleanorHyde-UMCG]

Data access fee and prelinked fields not shown under Access conditions (bug https://github.com/orgs/molgenis/projects/15/views/1?filterQuery=data+access+fee&pane=issue&itemId=68839761) Test plan 5 step 57 (compare test plan 9 step 28, where they ARE shown because they are 'true').

Linkage options field is also not shown (but this is not a true/false field). Test plan 5 step 57 and test plan 9 step 28.

Have added to list.

[EleanorHyde-UMCG]

When browsing 'all resources', there's no difference in the text under 'Cohort studies' and 'Studies' (both have "cohorts & biobanks" as explanatory text). Is this deliberate?

@BrendaHijmans

[EleanorHyde-UMCG]

The harmonisation matrix and the info per cohort don't match, for some reason. testvarrepeats in preview (here) gives different results in the matrix to what you see below when you click on testcohort3 in the harmonisation details per data source. Test plan 4 step 12, test plan 6 step 6.

Have added this to the list above.

Fixed by fixing the test data. The migration script meant for production data worked differently to what was intended on the (randomly filled-in) test repeated variable, meaning the number of repeats was smaller than meant to be, which in turn meant that fewer repeats were shown in the harmonisation matrix. No software fix required.