Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for dynamic credentials during get/put #322

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

lrstanley
Copy link
Member

This PR add support for using dynamically generated credentials inside of the get and put step. This is primarily beneficial when one is unable to generate long-lived credentials. For our specific usecase, any usage of AWS credentials must be short lived (less than 90min), and the way those credentials are generated does not integrate well with this resource.

I didn't add duplicate sections for those parameters under get and put definitions in the readme, as the ideal in most situations is for people to use the source configuration.

One item I'm not quite sure about is in relation to #96 (comment) (@vito) -- I am not sure if a resource is only used as a put (and the subsequent get), if this breaks the global shared resource model. Thoughts? Is there a better way around this issue?

@lrstanley lrstanley marked this pull request as ready for review September 12, 2022 22:43
@xtremerui xtremerui force-pushed the master branch 2 times, most recently from 779ef34 to 09bdeda Compare October 26, 2022 21:20
@taylorsilva taylorsilva self-requested a review February 23, 2023 15:47
@taylorsilva
Copy link
Member

So this workflow only works when you're putting an image. It would never work if your job is only trying to get an image since the check step has no way of getting credentials passed in like a get or put step would. This feels a bit half-baked here because of that.

With rotating credentials you'd be resetting the resource configuration every time new credentials are passed into the resource, which would break the version history on the resource (if you care about that).

The only workaround I can think of at the moment is to have some other job fetch credentials and update the secret store you have concourse connected to with the new credentials every time the old one is about to expire. wdyt?

@lrstanley
Copy link
Member Author

Yeah, I've been thinking about how to get those credentials into check steps in an elegant way for awhile, with no luck. Unfortunately, unless some of the future RFCs are pushed forward, I don't know if it'll be possible.

We had a few hundred users/pipelines using the approach of a job that runs every few hours, generates the credentials, write them to vault, so they can be pulled by concourse dynamically. The problem with that approach is that it's drastically more complex, users have trouble understanding the workflow, and it sends quite a few more requests to Vault (we already peak 40mil+ requests daily to our Vault clusters, which are used just for Concourse).

We are currently now using this PRs approach via my feature branch for all of those usecases, which seems to simplify getting this setup for most folks. It's definitely not perfect, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants