Styx uses Google Cloud Datastore and Google Cloud Bigtable for storing data about workflow state and executions and Google Container Engine (GKE) for container orchestration.
The connection to these external services uses Service Accounts and Application Default Credentials (ADC) to authenticate.
Because of the typical containers that we run at Spotify and the fact that they need to resolve resources in our own data centers, the k8s cluster needs some tweaking. These manual tweaks are temporary workarounds for features that are currently lacking in GKE.
A Datastore namespace will be used to store application and scheduler state.
The Bigtable cluster can be created from the cloud console and used as is. For the typical workloads that Styx generate, a basic 3 node cluster is sufficient.
TBA