[plugin] request collapsing (service aggregation) #7350
Replies: 48 comments 2 replies
-
|
more: http://microservices.io/patterns/apigateway.html this is traditionally referred to as "batching proxy". basically client sends a multi-part request, representing all the requests it wants to get, and the proxy completes those requests and responds with a multi-part response. in the case of kong the multi-part request can be configured in the plugin / kong side, and the response is a multi-part response. |
Beta Was this translation helpful? Give feedback.
-
|
@ahmadnassri Would it be possible to transform the response before passing it back to the client and also targeting devices? It's understood that everything is still in the idea phase. |
Beta Was this translation helpful? Give feedback.
-
|
@lindo-jmm depends on what you mean by "targeting devices" ? |
Beta Was this translation helpful? Give feedback.
-
|
I would love Kong do that but I wondering if its responsibility. https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcQ35pXZMfL-YNeA67_FbT8uxvhgz1_1ONpXiTddYzjn9mnbycd3bU-IS11ZbQ there need a edge service in our api to compose load balance e fault tolerance enhance |
Beta Was this translation helpful? Give feedback.
-
|
In order to start working on this plugin, could you provide some examples of requests you would like to collapse, and how the response of those requests should be aggregated in one output? |
Beta Was this translation helpful? Give feedback.
-
|
@thefosk there are two design patterns here that I know of: simple:plugin definition defines multiple upstream paths to query, and respond back with a single multi-part response that includes all results (headers + bodies) an upstream timeout configuration is critical here, (preferably a timeout per upstream). complex
tl;dr:
detailed example GET /my/collapsed/api HTTP/1.1
Host: localhost:8000
Accept: multipart/form-data
Connection: keep-alive
Content-Type: multipart/form-data; boundary=---------------------------xxx
Content-Length: 554
-----------------------------xxx
Host: api.foo.com
Content-Type: application/json
Content-Length: 14
{"foo": "bar"}
-----------------------------xxx
Host: api.foo.com
Content-Type: application/xml
<xml><foo>bar</foo></xml>
-----------------------------xxx
Host: api.foo.com
Content-Type: text/html
<!DOCTYPE html><title>Content of a.html.</title>
-----------------------------xxx--
outgoing response is a multipart response detailing each of the upstream results |
Beta Was this translation helpful? Give feedback.
-
|
I think this requires some more details before implementing. Eg. the example As for implementation; This will have the side effect that the sub requests can only exploit To make it play nicely with other plugins, this plugin would probably have to How to deal with errors?
How to pass parameters in the request? does every upstream get all parameters? What http methods should it support? Configuration: {
main_name = "some name", -- equals the names below, but for the main request
sub_requests = { -- array of upstream request targets
{
name = "upstream2",
timeout = 10,
url = "http://some/where/",
},{
name = "upstream3",
timeout = 10,
url = "http://some/where/",
},
},
response = "application/json" -- response type of the combined response (json or url)
header_prefix = "X-Kong-batch-", -- for url encoded responses
}JSON response: Multipart response: Does the above make sense? |
Beta Was this translation helpful? Give feedback.
-
|
Is there any progress? |
Beta Was this translation helpful? Give feedback.
-
|
+1 for the above use case, only change I want to request is to please add support for additional message formats such as JSON, XML etc
|
Beta Was this translation helpful? Give feedback.
-
|
Hi guys, I've made something close to that: |
Beta Was this translation helpful? Give feedback.
-
Any idea , if we have this plugin available now? |
Beta Was this translation helpful? Give feedback.
-
@carnei-ro can you add a license to your repo? |
Beta Was this translation helpful? Give feedback.
-
|
@Tieske I moved from bitbucket to github, please, use the link: https://github.com/carnei-ro/kong-aggregator. It has MIT license. |
Beta Was this translation helpful? Give feedback.
-
|
May these help ? |
Beta Was this translation helpful? Give feedback.
-
|
@subnetmarco @Tieske Any plans to implement this officially in Kong (Enterprise or OSS)? Or is this something the community could develop with Go plugins? |
Beta Was this translation helpful? Give feedback.
-
|
@subnetmarco @Tieske I would like to have any news about this implementation. |
Beta Was this translation helpful? Give feedback.
-
|
waiting for this awesome feature |
Beta Was this translation helpful? Give feedback.
-
|
We are also waiting for this feature |
Beta Was this translation helpful? Give feedback.
-
|
Hey guys, this is a really old thread. I made that plugin a long time ago. I could update it if you tell me some uses cases…. |
Beta Was this translation helpful? Give feedback.
-
|
Could you please point me to the plugin? If there was an easy way to allow Kong to make request to multiple upstream services APIs to aggregate responses similar to Tyk's compose API feature. This is a key feature of what API gateways should provide https://docs.microsoft.com/en-us/azure/architecture/patterns/gateway-aggregation |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/carnei-ro/kong-aggregator It is a really old and not maintained code |
Beta Was this translation helpful? Give feedback.
-
|
Is there an agreed upon way to implement this? I'm also very interested in this feature My use case for example is for a user to send a post request, with a json body. There would be some basic transformation using an array of data provided as a property. These requests would now all be forwarded on to the same microservice. The resulting requests would then be aggregated together into one array and sent back |
Beta Was this translation helpful? Give feedback.
-
|
is aggregator feature not implemented yet? |
Beta Was this translation helpful? Give feedback.
-
|
Krakend already supports this feature https://www.krakend.io/docs/backends/, even in sequential way https://www.krakend.io/docs/endpoints/sequential-proxy/ |
Beta Was this translation helpful? Give feedback.
and others
-
Example: https://github.com/Netflix/Hystrix/wiki/How-To-Use#Collapsing
Beta Was this translation helpful? Give feedback.
All reactions