Can Dex server run in HA setup in K8s

[pdrastil]

Hi, I'm trying to investigate if Dex server can run in HA deployment on K8s for Argo CD project. I've seen in your helm chart repository that user can configure multiple replicas, however Argo CD documentation states that Dex uses internal in-memory database and it's not possible to run more than 1 replica because of two or more instances would have inconsistent data. Can someone from maintainers confirm / deny if HA should work or not?

[nabokihms]

Hello! Thank you for the question.

Dex can run in ha mode by using external storage (Kubernetes, MySQL, Postgres or etcd). In theory, there might be a problem with cleanup or keys rotation jobs, but I have not seen one.

[sagikazarmark]

@nabokihms I've been thinking about moving those cleanups to a separate process for some time now. The idea is that the user can disable the "automatic" cleanup and run a separate command eg. in a cron job. That way it remains backward compatible. WDYT?

[nabokihms]

I like the approach. The only concerns are about details, e.g., running it using the CronJob controller on top of Kubernetes (as for me, it seems better to stick to the separate Deployment controller).

Anyway, implementing this is reasonable. After that, we will be able to say without a doubt that Dex can run in HA mode.

[sagikazarmark]

Well, we can certainly run it as a "worker" like process that does the scheduling internally. I can see the pros, but I don't see why using Kubernetes cron is worse. Can you elaborate?

We'll have to check how the cleanup jobs are scheduled anyway: if they are not periodic (ie. GC-like) Cron will indeed not be sufficient.

[nabokihms]

About CronJob, we faced that because of missing runs, it can be stuck, which requires manual actions to bring it alive. It looks like it is resolved in newer Kubernetes versions.

[dllegru]

Hi there! Is this still an issue? Would like to be able to run dex in HA with argo-cd as it is the sole component not HA enabled yet.
I'm reading the documentation of argo and still states a single instance can be run due the in-memory dabase constraint.