Skip to content

Commit

Permalink
Change architecture to microservices (#194)
Browse files Browse the repository at this point in the history
* Implement missing OTAC RPCs

* Remove TODO Command

* Add labels field for otacs.

We need this property in the future

* Implement missing ac service functions

* Refactor gRPC errors

* Introduce general proto message definitions

* Change proto package name

* Add dbconfig proto type definitions

* Refactoring

* Expose HobbyfarmCRD function

* Add crd config for dbconfig service

* Add Dockerfile for dbconfig service

* Refactoring crd generation

* Fix mistake

* Remove update of labels and restricted bind value.

Since resources contain an owner reference, labels and restricted bind values should not be updated but rather inferred from the resource object's owner reference.

* Implement dbconfig service

* Minor changes

* Add vm claim service

* Add vmset service

* Add progress service

* Remove CRD installation of migrated services

* Remove unused function

* Add new grpc error helper functions

* Fix error and add progress svc go module

* Improve error handling for not found errors

* Introduce caching for get/list and improve error handling

* Implement caching for get/list

* Add generic functions for Get and OwnerReferences

* log error for unspecified IDs

* Use generic getter function

Improving code quality...

* implement and use util function to delete hf resources

further improving code quality ...

* Implement generic unmarshall function

* Fix integer typing

* Add generic function to convert maps of type map[string] int/uint32

* Apply map conversion

* Remove redundant imports

* Add docker files

* Implement internal part of vm template service

* Rm CRDs, add generic func for CRD installation

The CRDs removed here are migrated towards their respective microservice

* Use generic function to generate CRDs

* Implement vm service

* move labels from util to labels package

* Add GrpcBadRequestError

* Add .proto definitions for vm service

* Implement internal scheduled event service

* Update proto definitions, use scheduled event service

* Update proto go gen version

* Add internal course and scenario service

* Update proto definitions

* Add internal environment service

* Use correct label package and generic func to install CRDs

* Use correct labels and generic func to install CRDs

* Use correct label package

* Use generic function to install CRDs

* Add internal session service

* Use session client, correct labels and generic CRD installer

* Use generic CRD installer

* Use correct labels package

* Use generic CRD installer

* Update to latest proto gen go version

* Fix retrieval of setting values

* Add new services to go.work

* Update proto gen go version

* Add missing microservices to MicroService type

* Minor fix

* Add deletion time stamp

* Add deletion time stamp

* Update function signatures

* Add uid in proto definitions

* Add uid to PreparedListSetting

* Add uid

* Return (list/get) annotations for environments and vms

* provide ID to vm service

* Add vmclaim controller

* Simplify code

* remove todo comment

* Update proto-gen version

* Implement vm controller

* Add parameter to optionally pass workqueue directly to controller

* Add deletion timestamp for vms

* Fix typo

* Add util function to add items to workqueue

* Add util function to remove finalizers

* Add wrapper type for string slices

* Add parameter to enable the option updating finalizers on vms

* Update error message

* Add controller logic for vm sets

* Remove old controller logic

* Add controller logic to scheduledevent service

* Rename scheduled event controller

* Use new scheduled event controller logic

* Remove old scheduled event controller logic

* Add session controller logic to session service

* Remove old session controller logic

* Implement terraform service

* Make GrpcNotFoundError function a generic function

* Return NotFoundError if to be deleted item is not found

* Add terraform service

* Add terraform client to vm svc and its controller

* Readd ContainsFinalizer utility function

* Add option to update secret name for vm

* Update tf service to return resource id after state creation

* Fix typo

* Add utility function to watch/verify object deletion

* Add proto file change to allow vm updates for secret name

* Move terraform controller logic into vm controller

* Remove old controller logic

* Add vm tasks to protobuf specification

* change property types

* Add vm tasks to scenario service

* Add missing route to authn service

* Remove old authserver logic

* Update package names for protobuf modules

* Simplify api server creation for user service

* Add function to safely extract grpc error details

* Add function to convert []*generalpb.StringMap to []map[string]string

* Simplify and improve grpc error handling

* Migrate course api server logic

* Start api server via course service

* Minor fix

* Rename rbac import name

* Rename function name. Add conversion functions.

* Migrate environment api server logic

* Start environment api logic in environment service

* Migrate progress api server logic

* Run go mod tidy

* Add comment explaining why progress api server is using progress k8s client directly

* Rename imports

* move course utility func to util module

This is needed because the AppendDynamicScenariosByCategories function is also used inside the scenario service

* Improve error handling

* Remove redundant return statements

* Simplify if expression

* Add utility functions to filter for sessions/scheduled events

* Change VMTask property name

* Add copy function to grpc server

* Migrate scenario API server logic

* Make struct map conversion generic

* Rename function

* Add max duration param for grpc OTAC creation func

* Migrate and apply scheduled event api server logic

* minor refactoring

* Rename field name

* Change return type

* Change return type

* Implement the option to update a collection of progress data

* Add comment

* Migrate session api server logic

* Change proto message field name

* Run go.mod tidy

* Migrate vm claim server logic

* Implement vmset api server logic

* Refactor wait group logic

* Run go.mod tidy

* Migrate vm api server logic

* Migrate vm template api logic

* Refactor shell service

* Refactor gargantua main.go and remove legacy code

* Add jobs for new services

* Run go.mod tidy

* Fix index out of bounds error

* Bug fixes

* Set WorkScheduler in controllers

* Fix typos/bugs

* Minor refactoring

* Minor bug fix

* Use fixed version for protoc-gen-go-grpc module

* Fix deletion of vm sets for finished scheduled events
  • Loading branch information
PhilipAB committed Jun 21, 2024
1 parent e49f4b9 commit c50d1df
Show file tree
Hide file tree
Showing 236 changed files with 40,955 additions and 11,432 deletions.
72 changes: 72 additions & 0 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,21 +31,93 @@ jobs:
path: ./v3/services/conversionsvc
image: conversion-service
secrets: inherit
build-course-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/coursesvc
image: course-service
secrets: inherit
build-dbconfig-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/dbconfigsvc
image: dbconfig-service
secrets: inherit
build-environment-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/environmentsvc
image: environment-service
secrets: inherit
build-progress-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/progresssvc
image: progress-service
secrets: inherit
build-rbac-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/rbacsvc
image: rbac-service
secrets: inherit
build-scenario-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/scenariosvc
image: scenario-service
secrets: inherit
build-scheduledevent-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/scheduledeventsvc
image: scheduledevent-service
secrets: inherit
build-session-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/sessionsvc
image: session-service
secrets: inherit
build-setting-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/settingsvc
image: setting-service
secrets: inherit
build-terraform-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/terraformsvc
image: terraform-service
secrets: inherit
build-user-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/usersvc
image: user-service
secrets: inherit
build-vmclaim-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/vmclaimsvc
image: vmclaim-service
secrets: inherit
build-vmset-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/vmsetsvc
image: vmset-service
secrets: inherit
build-vm-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/vmsvc
image: vm-service
secrets: inherit
build-vmtemplate-service:
uses: ./.github/workflows/build.yaml
with:
path: ./v3/services/vmtemplatesvc
image: vmtemplate-service
secrets: inherit
2 changes: 1 addition & 1 deletion generate-protos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ docker run --rm \
-c "apt-get update && apt-get install -y protobuf-compiler && \
export PATH=\$PATH:/go/bin && \
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest && \
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest && \
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.3.0 && \
find ${PROTO_DIR} -type f -name '*.proto' -exec bash -c ' \
protoc -I ${PROTO_DIR} \
--go_out=./protos \
Expand Down
12 changes: 12 additions & 0 deletions go.work
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,16 @@ use (
./v3/services/rbacsvc
./v3/services/settingsvc
./v3/services/usersvc
./v3/services/dbconfigsvc
./v3/services/vmclaimsvc
./v3/services/vmsetsvc
./v3/services/progresssvc
./v3/services/vmtemplatesvc
./v3/services/vmsvc
./v3/services/sessionsvc
./v3/services/scheduledeventsvc
./v3/services/environmentsvc
./v3/services/scenariosvc
./v3/services/coursesvc
./v3/services/terraformsvc
)
Loading

0 comments on commit c50d1df

Please sign in to comment.