Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add script to generate OAI models #27

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,7 @@ openapitools.json
*.log
funnel-work-dir/
funnel/

package.json
package-lock.json
node_modules/
5 changes: 1 addition & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,7 @@ TBA

## Building

First, clone the repository, and then run the following command to automatically generate models using OpenAPI specifications:
```
bash ./build-models.sh
```
**Developer note:** If you want to update any of the [OpenAPI](https://www.openapis.org/) specifications supported by this tool or extend it to support any other OpenAPI-backed services (e.g., not currently supported GA4GH API-based services or your own services), please refer to our [documentation for auto-generating OpenAPI models](docs/autogenerate_OAI_models.md).

To build the project:
```
Expand Down
52 changes: 52 additions & 0 deletions docs/autogenerate_OAI_models.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
## Motivation
You will only need to use the script to build models when any of the GA4GH API's [OpenAPI](https://www.openapis.org/) specs are updated. The models are being used to return the result of any API call in the correct format, in the correct structs.

## Prerequisites and Dependencies:
1. OpenAPI Generator CLI: Make sure the directory `~/bin/openapitools` is empty, as OpenAPI is being installed in this script, which might conflict with your existing directory. Then run the following script to install OpenAPI generator CLI.
```sh
mkdir -p ~/bin/openapitools
curl https://raw.githubusercontent.com/OpenAPITools/openapi-generator/master/bin/utils/openapi-generator-cli.sh > ~/bin/openapitools/openapi-generator-cli
chmod u+x ~/bin/openapitools/openapi-generator-cli
export PATH=$PATH:~/bin/openapitools/
```
For more information, and for looking at alternate ways to install OpenAPI Generator CLI, see the documentation at https://openapi-generator.tech/docs/installation/

2. Java Development Kit (JDK)
The OpenAPI generator CLI requires Java to run. Ensure that you have the JDK installed on your system.
You can install the JDK using the following command:
- On Ubuntu/Debian:
```sh
sudo apt-get update
sudo apt-get install default-jdk
```
- On Fedora:
```sh
sudo dnf install java-11-openjdk
```
- On macOS (using Homebrew):
```sh
brew install openjdk@11
```

## Usage

### Option 1
1. Clone the repository:
```sh
git clone https://github.com/elixir-cloud-aai/ga4gh-sdk.git
```
2. Update the specs of the APIs you want to change in [`/utils/build-models.sh`](command:_github.copilot.openRelativePath?%5B%7B%22scheme%22%3A%22file%22%2C%22authority%22%3A%22%22%2C%22path%22%3A%22%2Fhome%2Faarav%2Fdev%2Fga4gh-sdk%2Futils%2Fbuild_models_wrapper.sh%22%2C%22query%22%3A%22%22%2C%22fragment%22%3A%22%22%7D%5D "/home/aarav/dev/ga4gh-sdk/utils/build_models_wrapper.sh"):
```sh
declare -A SPECS
SPECS["serviceinfo"]="new serviceinfo specs"
SPECS["tes"]="new tes openapi specs"
```
3. Run the following command to automatically generate models using OpenAPI specifications:
```sh
bash ./utils/build_models_wrapper.sh
```

### Option 2
To run the script, use the following command:
```sh
./build_models.sh <OPENAPI_SPEC_PATH> <API_NAME> <DESTINATION_DIR>
84 changes: 84 additions & 0 deletions utils/build_models.sh
uniqueg marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#!/bin/bash

# Usage:
# -----
# To run the script, use the following command:
# ./script.sh <OPENAPI_SPEC_PATH> <API_NAME> <DESTINATION_DIR>
#
# Example:
# ./script.sh https://example.com/path/to/openapi.yaml myapi lib/src/clients/myapi/
#
# Parameters:
# - OPENAPI_SPEC_PATH: The URL or file path to the OpenAPI specification.
# - API_NAME: The name of the API.
# - DESTINATION_DIR: The directory where the generated models should be copied.
#
# The script will generate Rust models from the OpenAPI specification and place them in the specified destination directory.

# Ensure the correct number of arguments
if [ "$#" -ne 3 ]; then
echo "Usage: $0 <OPENAPI_SPEC_PATH> <API_NAME> <DESTINATION_DIR>"
exit 1
fi

# Get the parameters
OPENAPI_SPEC_PATH="$1"
API_NAME="$2"
DESTINATION_DIR="$3"

# Define constants
SED_RULE="s/^use\s\+crate::models\s*;/#![allow(unused_imports)]\n#![allow(clippy::empty_docs)]\nuse crate::clients::$API_NAME::models;/"
TEMP_OUTPUT_DIR=$(mktemp -d) # Define the temporary output directory for the OpenAPI generator

# Exit immediately if a command exits with a non-zero status.
set -euo pipefail

generate_openapi_models() {
# Remove the temporary directory at the end of the script
trap 'rm -rf "$TEMP_OUTPUT_DIR"' EXIT

openapi-generator-cli version

# Run the OpenAPI generator CLI using the JAR file
openapi-generator-cli generate -g rust \
-i "$OPENAPI_SPEC_PATH" \
-o "$TEMP_OUTPUT_DIR" \
--additional-properties=useSingleRequestParameter=true

# Check if the generation was successful
if [ $? -ne 0 ]; then
echo "OpenAPI generation failed. Check the verbose output for details."
exit 1
fi

# Remove the openapitools.json file
rm -f "$TEMP_OUTPUT_DIR/openapitools.json"

echo "TEMP_OUTPUT_DIR is $TEMP_OUTPUT_DIR"

# Modify the import statements in each generated file
find "$TEMP_OUTPUT_DIR" -name '*.rs' > /dev/null
if [ $? -ne 0 ]; then
echo "Error: 'find' command failed."
exit 1
fi

for file in $(find "$TEMP_OUTPUT_DIR" -name '*.rs'); do
if [[ "$OSTYPE" == "darwin"* ]]; then
# macOS (BSD) sed syntax
sed -i '' "$SED_RULE" "$file"
else
# Linux (GNU) sed syntax
sed -i "$SED_RULE" "$file"
fi
done

rm -rf "$DESTINATION_DIR/models"
mkdir -p "$DESTINATION_DIR"
cp -r "$TEMP_OUTPUT_DIR/src/models" "$DESTINATION_DIR"

echo "OpenAPI generation complete. Models copied to $DESTINATION_DIR"
}

# Call the function to generate models
generate_openapi_models "$OPENAPI_SPEC_PATH" "$API_NAME" "$DESTINATION_DIR"
33 changes: 33 additions & 0 deletions utils/build_models_wrapper.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/bash

# Define the list of OpenAPI specifications and their corresponding API names and destination directories
declare -A SPECS

# *Developer note*: Add entries to the SPECS associative array as shown in the examples below.
# Each entry should map an API name to a URL and a destination directory.
#
# Example:
# SPECS["serviceinfo"]="https://raw.githubusercontent.com/ga4gh-discovery/ga4gh-service-info/develop/service-info.yaml lib/src/clients/serviceinfo/"
# SPECS["tes"]="https://raw.githubusercontent.com/ga4gh/task-execution-schemas/develop/openapi/task_execution_service.openapi.yaml lib/src/clients/tes/"
#
# To add a new entry, follow this format:
# SPECS["<api_name>"]="<url> <destination_directory>"
#
# Replace <api_name> with the name of the API, <url> with the URL to the OpenAPI specification,
# and <destination_directory> with the path to the directory where the models should be generated.
#
# Example of adding a new entry:
# SPECS["newapi"]="https://example.com/path/to/newapi.yaml lib/src/clients/newapi/"


# Directory of the build script
BUILD_SCRIPT="./utils/build_models.sh"

# Iterate over the specifications and call the build script for each
for API_NAME in "${!SPECS[@]}"; do
SPEC_URL=$(echo ${SPECS[$API_NAME]} | awk '{print $1}')
DESTINATION_DIR=$(echo ${SPECS[$API_NAME]} | awk '{print $2}')

echo "Generating models for $API_NAME from $SPEC_URL into $DESTINATION_DIR"
$BUILD_SCRIPT "$SPEC_URL" "$API_NAME" "$DESTINATION_DIR"
done