Flyte Project isolation

[robert-ulbrich-mercedes-benz]

Motivation

Flyte allows having multiple tenants by using projects and domains. Flyte also comes with an authentication that only
allows access to Flyte if the user is successfully authenticated. Once the user is successfully authenticated, she has full access to all projcets and domains in Flyte e.g. submit a workflow in any Flyte project.

Functional description

In order to protect data from unauthorized access, it makes sense to isolate the Flyte projects from each other.
That way a user can only submit workflows in the project he or she is assigned to. If trying to register a launchplan,
submitting a Flyte workflow or a task in a project the user is not assigned to, will cause an error.

Implementation idea

The projects a user is assigned to, will be identified by reading a claim from an identity token that is issued in the course
of the authentication process. The claim to be read for this purpose is configured in Flyte's configuration file. Also a mapping
between the claim's value and the actual project to be assigned to is maintained in the Flyte configuration.

Limits

So this approach only aims on isolating single Flyte tenants from each other and there is no intention to introduce a full blown
IAM concept including user roles. So any member of a Flyte project still has full access to their project's launch plans data etc.

[kumare3]

IMO using oauth2 for AuthZ is usually frowned upon and leads to problems. Maintaining claims and projects in configuration is also hard. Flyte has API's to create projects, this becomes hard. Also returning all claims will slow down authentication. I am not an expert at this, but seems like we need to think way more deeply about this

[robert-ulbrich-mercedes-benz]

I agree, that it won't be really easy. But I do not understand, why it is hard to maintain claims and project configuration in configuration. Of course we should also add fields to the API to configure a mapping between claims (and their values) and the projects to create.

Can you also please explain how returning claims slows down authentication? I guess the list of claims and their values for projects can be kept in memory and iterating over them with the identity token, I personally do not see a big performance degradation risk, as least as there not thousands of different projects.

[corleyma]

It would be interesting to consider providing some kind of integration between FlyteAdmin and OpenFGA.

and/or, providing some pluggable AuthZ interface in flyteadmin that allows users to implement their own external authz service could be a good compromise.

[sorushsaghari]

I appreciate the discussion around this topic. Maybe I’m missing something, but why not consider adding a simple mechanism, like using a configuration file, to match a field (such as a group) with the OAuth2 provider? This could be used to control access to projects and domains, similar to how Argo CD manages access control with OAuth2 providers, as described in their documentation: Argo CD User Management with Auth0.

This approach could be sufficient for most use cases, providing a straightforward and effective way to manage access without the need for a full-blown IAM solution or complex configurations. However, if there are specific challenges or considerations with this approach that I might be overlooking, I’d love to hear your thoughts!

[robert-ulbrich-mercedes-benz]

Hi @sorushsaghari,
what you are suggesting is exactly the plan. By the way: We already implemented that project isolation and would like to contribute it back to upstream, but there was no advance on this topic in the recent months.

It is fairly simple: you configure fields and their values you expect on an ID or access token in order to provide access to a certain project. Works pretty well for us so far.

[robert-ulbrich-mercedes-benz]

I would also like to keep it simple for the beginning and not to integrate OpenFGA which might be a good product, but still is far from being a standard for authentication and authorization. Going further here than isolation of single projects makes it much harder to implement authentication and authorization.