chore(emx2): spark to javalin migration #4165

harmbrugge · 2024-08-26T19:57:58Z

What are the main changes you did:

Migrated the API code from Spark to Javalin

how to test:
Please test all features of emx2 to see if everything behaves as expected. Everything should work as before.

todo:

migrate api code to javalin
add proxy test

# Conflicts: # backend/molgenis-emx2-fairdatapoint/src/main/java/org/molgenis/emx2/fairdatapoint/FAIRDataPointCatalog.java # backend/molgenis-emx2-fairdatapoint/src/main/java/org/molgenis/emx2/fairdatapoint/FAIRDataPointDistribution.java # backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/CsvApi.java # backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/ExcelApi.java # backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/GroupPathMapper.java # backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/MolgenisWebservice.java # backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/RDFApi.java # backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/TaskApi.java # backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/ZipApi.java

...d/molgenis-emx2-beacon-v2/src/main/java/org/molgenis/emx2/beaconv2/requests/BeaconQuery.java

connoratrug · 2024-09-18T12:17:17Z

...olgenis-emx2-beacon-v2/src/test/java/org/molgenis/emx2/beaconv2/Beaconv2_EntryTypesTest.java

    when(request.url()).thenReturn("http://localhost:8080/api/beacon");
    when(request.attribute("specification")).thenReturn("beacon");

    return request;
  }

  @Test
-  public void testEntryTypes() {
-    Request request = mockRequest();
+  @Disabled


why is it disabled ? ( i think a annotation allows to add a reason )

connoratrug · 2024-09-18T12:17:32Z

backend/molgenis-emx2-beacon-v2/src/test/java/org/molgenis/emx2/beaconv2/Beaconv2_MapTest.java

    when(request.url()).thenReturn("http://localhost:8080/api/beacon");
    when(request.attribute("specification")).thenReturn("beacon");

    return request;
  }

  @Test
-  public void testMap() {
+  @Disabled


connoratrug · 2024-09-18T12:19:13Z

...end/molgenis-emx2-beacon-v2/src/test/java/org/molgenis/emx2/beaconv2/vp/BeaconVpMapTest.java

    when(request.url()).thenReturn("http://localhost:8080/api/beacon_vp");
    when(request.attribute("specification")).thenReturn("beacon_vp");

    return request;
  }

  @Test
-  public void testMap() {
+  @Disabled


connoratrug · 2024-09-18T12:24:14Z

backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/AnalyticsApi.java


  private AnalyticsApi() {
    // hide constructor
  }

-  public static void create() {
+  public static void create(Javalin app) {


in general it's bad practice modify the param passed in a method, guess the same thing sort of happend via the static post/get/.. imports, so i guess the is sort of a step in the right direction, is this the recommended way to configure the routes in javlin ?

connoratrug · 2024-09-18T12:25:28Z

backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/AnalyticsApi.java

-    response.type("application/json");
-    MolgenisSession session = sessionManager.getSession(request);
-    String schemaName = sanitize(request.params(SCHEMA));
+  private static void listSchemaTriggers(Context ctx) {


why change form function to method, makes hard to test , reuse compose

connoratrug · 2024-09-18T12:26:01Z

backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/AnalyticsApi.java

-        actionTransformer.transform(request.body(), CreateTriggerAction.class);
-    MolgenisSession session = sessionManager.getSession(request);
-    String schemaName = sanitize(request.params(SCHEMA));
+  private static void addTrigger(Context ctx) {


keeps a function

connoratrug · 2024-09-18T12:26:09Z

backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/AnalyticsApi.java

-    var action = actionTransformer.transform(request.body(), UpdateTriggerAction.class);
-    MolgenisSession session = sessionManager.getSession(request);
-    String schemaName = sanitize(request.params(SCHEMA));
+  private static void updateTrigger(Context ctx) {


dtroelofsprins

Did some functional tests:
Successful:

Create Directory schema from template including demo data
Schema editor
DataCatalogueTemplate
Fair data hub template
Data upload (zipfile with several csv's) using UI
Other templates STILL NEED TO BE TESTED

Not working properly:

Create ERN Dashboard from template when going to the schema a white screen is shown, should go to /tables/#
Directory App => footer is broken, pictures not shown (f.e. logo)
Manually adding data => FAILS WITH ERROR: Parsing of graphql variables failed. Should be an object with each graphql variable a key. argument "content" is null: argument "content" is null
Manually updating data => FAILS WITH ERROR Parsing of graphql variables failed. Should be an object with each graphql variable a key. argument "content" is null: argument "content" is null
Run a script on the server => Cannot create a script due to saving error (Running the hello world script works)
Run a script locally using the EMX2-pyclient =>
- create schema using template WORKS
- downloading real BBMRI-ERIC biobanks fails with error UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 8688: invalid continuation byte => using the same script, with the same data on emx2.dev.molgenis.org works fine.

dtroelofsprins

Did some functional tests:
Successful:

Create Directory schema from template including demo data
Schema editor
Use DataCatalogueTemplate
Fair data hub template
Data upload (zipfile with several csv's) using UI
Other templates => tested several, but not all
Schedule a script
Failure email when script fails => NOT POSSIBLE on preview server

Fixed:

Directory App => footer is broken, pictures not shown (f.e. logo)
Manually adding data
Manually updating data
Create ERN Dashboard from template when going to the schema a white screen is shown, should go to /tables/#
Run a script on the server
Run a script locally using the EMX2-pyclient =>
- create schema using template WORKS
- downloading real BBMRI-ERIC biobanks

New issues (probably not tested before):

Directory App => Load json settings does not have the proper layout
CatalogueTemplate => SSR Catalogue shows white screen => Might also be due to recent changes to the catalogue that it's not working properly

sonarcloud · 2024-09-25T07:26:52Z

Quality Gate failed

Failed conditions
72.2% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

harmbrugge added 2 commits August 26, 2024 17:05

started migration to javalin

ac75e53

migration to javalin

c0f057a

harmbrugge marked this pull request as draft August 26, 2024 19:58

harmbrugge added 16 commits August 27, 2024 14:21

changed api classes to javalin code

d8455df

different paths for jars

3e07554

added needed graphql mappings

f968f7f

Added css and static routes

34fadfa

Updated beacon api

4e0bb47

updated tests

e25f66e

handling of static files

d3de6cf

fix: correct return types in zip api

7a5cbcf

updated beacon test with javalin context

f5f0781

updated tests

080896d

migration to javalin

15f00d8

spark to javalin migration

5b0d426

spark to javalin migration

27735dd

spark to javalin migration

19d2c80

spark to javalin migration

ecbac76

harmbrugge requested review from connoratrug, mswertz and dtroelofsprins September 18, 2024 07:15

harmbrugge added 2 commits September 18, 2024 10:14

spark to javalin: ignore trailing slashes

128cf10

spark to javalin: simplified static file mapping

48232ac

connoratrug reviewed Sep 18, 2024

View reviewed changes

...d/molgenis-emx2-beacon-v2/src/main/java/org/molgenis/emx2/beaconv2/requests/BeaconQuery.java Show resolved Hide resolved

connoratrug reviewed Sep 18, 2024

View reviewed changes

dtroelofsprins requested changes Sep 19, 2024

View reviewed changes

harmbrugge marked this pull request as ready for review September 19, 2024 12:24

spark to javalin: graphql api fix

87f00a9

dtroelofsprins requested changes Sep 23, 2024

View reviewed changes

harmbrugge added 2 commits September 24, 2024 10:17

spark to javalin: oidc fix

a99c899

spark to javalin: index file fix

1d1a1b0

harmbrugge requested review from BrendaHijmans and Fernananas September 25, 2024 07:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(emx2): spark to javalin migration #4165

chore(emx2): spark to javalin migration #4165

harmbrugge commented Aug 26, 2024 •

edited

Loading

connoratrug Sep 18, 2024

connoratrug Sep 18, 2024

connoratrug Sep 18, 2024

connoratrug Sep 18, 2024

connoratrug Sep 18, 2024

connoratrug Sep 18, 2024

connoratrug Sep 18, 2024

dtroelofsprins left a comment •

edited by harmbrugge

Loading

dtroelofsprins left a comment

sonarcloud bot commented Sep 25, 2024

chore(emx2): spark to javalin migration #4165

Are you sure you want to change the base?

chore(emx2): spark to javalin migration #4165

Conversation

harmbrugge commented Aug 26, 2024 • edited Loading

connoratrug Sep 18, 2024

Choose a reason for hiding this comment

connoratrug Sep 18, 2024

Choose a reason for hiding this comment

connoratrug Sep 18, 2024

Choose a reason for hiding this comment

connoratrug Sep 18, 2024

Choose a reason for hiding this comment

connoratrug Sep 18, 2024

Choose a reason for hiding this comment

connoratrug Sep 18, 2024

Choose a reason for hiding this comment

connoratrug Sep 18, 2024

Choose a reason for hiding this comment

dtroelofsprins left a comment • edited by harmbrugge Loading

Choose a reason for hiding this comment

dtroelofsprins left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Sep 25, 2024

Quality Gate failed

harmbrugge commented Aug 26, 2024 •

edited

Loading

dtroelofsprins left a comment •

edited by harmbrugge

Loading