[Feature] make it possible to attibute S3 GET/List requests to specific queries.

[antonysouthworth-halter]

Is this your first time submitting a feature request?

• I have searched the existing issues, and I could not find an existing issue for this feature

Describe the feature

When trying to trace down the origins of S3 GET requests it is currently quite difficult to attribute a given GET request with a specific Athena query.

We already set the User-Agent for all requests (#49) which is awesome and means you can tell that they are coming from the adapter; my proposal is to take that a step further and potentially include some kind of identifier for the exact query being run? Unfortunately this can't be the query execution ID from Athena but perhaps we could inject the ClientRequestToken into the User-Agent header value for the StartQueryExecution call?

That way the ClientRequestToken of the original StartQueryExecution call will show up in CloudTrail logs for the S3 GET requests and therefore provide a lineage chain from StartQueryExecution call to GetObject request and back again (check the responseElements of the CloudTrail log for the StartQueryExecution request to get the queryExecutionId).

Never mind, see comment. Also it doesn't need to be the ClientRequestToken it can just be anything.

Describe alternatives you've considered

Haven't really thought of any. Ideas welcome!

Who will this benefit?

Folks trying to use the adapter at any kind of scale.

Are you interested in contributing this feature?

Potentially? There's a decent chance I just try to do this on a local fork.

Anything else?

No response

[antonysouthworth-halter]

🙃 Not sure how I ended up thinking that the User-Agent is passed from Athena through to S3 GET, it's clear as day that the User-Agent for the GETs is just athena.amazonaws.com. Must have got some wires crossed somewhere.

Regardless if we had some way of attributing the requests that would be good!

[nicor88]

Here it's where we set the user-agent for the calls done from boto3 to Athena. But given the fact that the caller for the S3 calls is athena itself, I'm unsure if the information that you need are passed down to S3. It will be indeed amazing to have in the S3 trails more informations about the caller, for example the query execution id to track down who did the Get/List.

Shall we raise this as a feedback for the athena team itself? I'm usure that we can do much here.