diff --git a/plugins/wasm-cpp/extensions/basic_auth/README.md b/plugins/wasm-cpp/extensions/basic_auth/README.md index 02dc764f4d..6a39fd1481 100644 --- a/plugins/wasm-cpp/extensions/basic_auth/README.md +++ b/plugins/wasm-cpp/extensions/basic_auth/README.md @@ -1,16 +1,30 @@ -

- English | 中文 -

+--- +title: Basic 认证 +keywords: [higress,basic auth] +description: Basic 认证插件配置参考 +--- -# 功能说明 +## 功能说明 `basic-auth`插件实现了基于 HTTP Basic Auth 标准进行认证鉴权的功能 -# 配置字段 +## 运行属性 -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ----------- | --------------- | -------- | ------ | ---------------------------------------------------- | -| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | -| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表，用于对请求进行鉴权 | +插件执行阶段：`认证阶段` +插件执行优先级：`320` + +## 配置字段 + +**注意：** + +- 在一个规则里，鉴权配置和认证配置不可同时存在 +- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 + +### 认证配置 + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | -------- | ------ | ---------------------------------------------------- | +| `global_auth` | bool | 选填（**仅实例级别配置**） | - | 只能在实例级别配置，若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制，若不配置则仅当没有域名和路由配置时全局生效（兼容老用户使用习惯）。 | +| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | `consumers`中每一项的配置字段说明如下： @@ -19,51 +33,49 @@ | `credential` | string | 必填 | - | 配置该consumer的访问凭证 | | `name` | string | 必填 | - | 配置该consumer的名称 | -`_rules_` 中每一项的配置字段说明如下： +### 鉴权配置（非必需） | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的路由名称 | -| `_match_domain_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的域名 | | `allow` | array of string | 必填 | - | 对于符合匹配条件的请求，配置允许访问的consumer名称 | -**注意：** -- 若不配置`_rules_`字段，则默认对当前网关实例的所有路由开启认证； -- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 - -# 配置示例 +## 配置示例 -## 对特定路由或域名开启认证和鉴权 +### 全局配置认证和路由粒度进行鉴权 以下配置将对网关特定路由或域名开启 Basic Auth 认证和鉴权，注意凭证信息中的用户名和密码之间使用":"分隔，`credential`字段不能重复 + +在实例级别做如下插件配置： + ```yaml -# 使用 _rules_ 字段进行细粒度规则配置 consumers: - credential: 'admin:123456' name: consumer1 - credential: 'guest:abc' name: consumer2 -_rules_: -# 规则一：按路由名称匹配生效 - - _match_route_: - - route-a - - route-b - allow: - - consumer1 -# 规则二：按域名匹配生效 - - _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer2 +global_auth: false +``` + +对 route-a 和 route-b 这两个路由做如下配置： + +```yaml +allow: +- consumer1 +``` + +对 *.example.com 和 test.com 在这两个域名做如下配置: + +```yaml +allow: +- consumer2 ``` -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许`name`为`consumer1`的调用者访问，其他调用者不允许访问； +若是在控制台进行配置，此例指定的 `route-a` 和 `route-b` 即在控制台创建路由时填写的路由名称，当匹配到这两个路由时，将允许`name`为`consumer1`的调用者访问，其他调用者不允许访问； -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将允许`name`为`consumer2`的调用者访问，其他调用者不允许访问。 +此例指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将允许`name`为`consumer2`的调用者访问，其他调用者不允许访问。 -### 根据该配置，下列请求可以允许访问： +根据该配置，下列请求可以允许访问： **请求指定用户名密码** @@ -77,7 +89,7 @@ curl -H 'Authorization: Basic YWRtaW46MTIzNDU2' http://xxx.hello.com/test 认证鉴权通过后，请求的header中会被添加一个`X-Mse-Consumer`字段，在此例中其值为`consumer1`，用以标识调用方的名称 -### 下列请求将拒绝访问： +下列请求将拒绝访问： **请求未提供用户名密码，返回401** ```bash @@ -93,22 +105,10 @@ curl -u admin:abc http://xxx.hello.com/test curl -u guest:abc http://xxx.hello.com/test ``` -## 网关实例级别开启 - -以下配置未指定`_rules_`字段，因此将对网关实例级别开启 Basic Auth 认证 - -```yaml -consumers: -- credential: 'admin:123456' - name: consumer1 -- credential: 'guest:abc' - name: consumer2 -``` - -# 相关错误码 +## 相关错误码 -| HTTP 状态码 | 出错信息 | 原因说明 | -| ----------- | ------------------------------------------------------------------------------ | ---------------------- | +| HTTP 状态码 | 出错信息 | 原因说明 | +| ----------- |--------------------------------------------------------------------------------| ---------------------- | | 401 | Request denied by Basic Auth check. No Basic Authentication information found. | 请求未提供凭证 | -| 401 | Request denied by Basic Auth check. Invalid username and/or password | 请求凭证无效 | -| 403 | Request denied by Basic Auth check. Unauthorized consumer | 请求的调用方无访问权限 | +| 401 | Request denied by Basic Auth check. Invalid username and/or password. | 请求凭证无效 | +| 403 | Request denied by Basic Auth check. Unauthorized consumer. | 请求的调用方无访问权限 | diff --git a/plugins/wasm-cpp/extensions/basic_auth/README_EN.md b/plugins/wasm-cpp/extensions/basic_auth/README_EN.md index 3b07963380..b5cc71b98b 100644 --- a/plugins/wasm-cpp/extensions/basic_auth/README_EN.md +++ b/plugins/wasm-cpp/extensions/basic_auth/README_EN.md @@ -1,117 +1,99 @@ -

- English | 中文 -

- -# Description -`basic-auth` plugin implements the function of authentication based on the HTTP Basic Auth standard. - -# Configuration Fields - -| Name | Type | Requirement | Default Value | Description | -| ----------- | --------------- | -------- | ------ | ---------------------------------------------------- | -| `consumers` | array of object | Required | - | Caller of the service for authentication of requests | -| `_rules_` | array of object | Optional | - | Configure access permission list for specific routes or domains to authenticate requests | - -Filed descriptions of `consumers` items: - -| Name | Type | Requirement | Default Value | Description | -| ------------ | ------ | ----------- | ------------- | ------------------------------------- | -| `credential` | string | Required | - | Credential for this consumer's access | -| `name` | string | Required | - | Name of this consumer | - -Configuration field descriptions for each item in `_rules_` are as follows: - -| Field Name | Data Type | Requirement | Default | Description | -| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | One of `_match_route_` or `_match_domain_` | - | Configure the routes to match for request authorization | -| `_match_domain_` | array of string | One of `_match_route_` , `_match_domain_` | - | Configure the domains to match for request authorization | -| `allow` | array of string | Required | - | Configure the consumer names allowed to access requests that match the match condition | - +--- +title: Basic Authentication +keywords: [higress,basic auth] +description: Basic authentication plugin configuration reference +--- +## Function Description +The `basic-auth` plugin implements authentication and authorization based on the HTTP Basic Auth standard. + +## Operation Attributes +Plugin execution stage: `Authentication Phase` +Plugin execution priority: `320` + +## Configuration Fields **Note:** - -- If the `_rules_` field is not configured, authentication is enabled for all routes of the current gateway instance by default; -- For authenticated requests, `X-Mse-Consumer` field will be added to the request header to identify the name of the caller. - -# Configuration Samples - -## Enable Authentication and Authorization for specific routes or domains - -The following configuration will enable Basic Auth authentication and authorization for specific routes or domains of the gateway. Note that the username and password in the credential information are separated by a ":", and the `credential` field cannot be repeated. - - - +- In one rule, authentication configurations and authorization configurations cannot coexist. +- For requests that pass authentication, the request header will include an `X-Mse-Consumer` field to identify the caller's name. + +### Authentication Configuration +| Name | Data Type | Requirements | Default Value | Description | +| ------------- | ---------------- | ------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `global_auth` | bool | Optional (**instance-level only**) | - | Can only be configured at the instance level. If set to true, the authentication mechanism will take effect globally; if set to false, it will only take effect for the configured domains and routes. If not configured, it will only take effect globally when there are no domain and route configurations (compatible with old user habits). | +| `consumers` | array of object | Required | - | Configures the service callers for request authentication. | + +Each configuration field in `consumers` is described as follows: +| Name | Data Type | Requirements | Default Value | Description | +| ------------ | --------- | ------------ | ------------- | ------------------------------- | +| `credential` | string | Required | - | Configures the access credentials for this consumer. | +| `name` | string | Required | - | Configures the name of this consumer. | + +### Authorization Configuration (Optional) +| Name | Data Type | Requirements | Default Value | Description | +| ---------------- | ---------------- | ---------------------------------------------------- | -------------- | -------------------------------------------------------- | +| `allow` | array of string | Required | - | Configures the consumer names allowed to access for matching requests. | + +## Configuration Example +### Global Authentication and Route Granularity Authorization +The following configuration will enable Basic Auth authentication and authorization for specific routes or domains of the gateway. Note that the username and password in the credential information are separated by ":", and the `credential` field cannot be duplicated. + +Make the following plugin configuration at the instance level: ```yaml -# use the _rules_ field for fine-grained rule configuration. consumers: - credential: 'admin:123456' name: consumer1 - credential: 'guest:abc' name: consumer2 -_rules_: -# rule 1: match by the route name. - - _match_route_: - - route-a - - route-b - allow: - - consumer1 -# rule 2: match by the domain. - - _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer2 +global_auth: false ``` -In this sample, `route-a` and `route-b` specified in `_match_route_` are the route names filled in when creating gateway routes. When these two routes are matched, the caller with `name` as `consumer1` is allowed to access, and other callers are not allowed to access. -The `*.example.com` and `test.com` specified in `_match_domain_` are used to match the domain name of the request. When the domain name is matched, the caller with `name` as `consumer2` is allowed to access, and other callers are not allowed to access. +For routes `route-a` and `route-b`, configure as follows: +```yaml +allow: +- consumer1 +``` +For the domains `*.example.com` and `test.com`, configure as follows: +```yaml +allow: +- consumer2 +``` -### According to this configuration, the following requests are allowed: +If configured in the console, the specified `route-a` and `route-b` refer to the route names filled in when creating the routes in the console. When matching these two routes, callers with the name `consumer1` will be allowed access, while other callers will not. -**Requests with specified username and password** +The specified `*.example.com` and `test.com` are used to match the request domain. When a match is found, callers with the name `consumer2` will be allowed access, while other callers will not. +Based on this configuration, the following requests may be allowed access: +**Request with specified username and password** ```bash -# Assuming the following request will match with route-a -# Use -u option of curl to specify the credentials +# Assuming the following request matches the route-a route +# Using curl's -u parameter to specify curl -u admin:123456 http://xxx.hello.com/test -# Or specify the Authorization request header directly with the credentials in base64 encoding +# Or directly specify the Authorization request header with the username and password encoded in base64 curl -H 'Authorization: Basic YWRtaW46MTIzNDU2' http://xxx.hello.com/test ``` -A `X-Mse-Consumer` field will be added to the headers of the request, and its value in this example is `consumer1`, used to identify the name of the caller when passed authentication and authorization. +After successful authentication, the request header will have an added `X-Mse-Consumer` field, which in this case is `consumer1` to identify the caller's name. -### The following requests will be denied: - -**Requests without providing username and password, returning 401** +The following requests will be denied access: +**Request without username and password, returns 401** ```bash curl http://xxx.hello.com/test ``` -**Requests with incorrect username or password, returning 401** + +**Request with incorrect username and password, returns 401** ```bash curl -u admin:abc http://xxx.hello.com/test ``` -**Requests matched with a caller who has no access permission, returning 403** + +**Caller matched by username and password has no access, returns 403** ```bash -# consumer2 is not in the allow list of route-a +# consumer2 is not in the allow list for route-a curl -u guest:abc http://xxx.hello.com/test ``` -## Enable basic auth for gateway instance - -The following configuration does not specify the `_rules_` field, so Basic Auth authentication will be effective for the whole gateway instance. - -```yaml -consumers: -- credential: 'admin:123456' - name: consumer1 -- credential: 'guest:abc' - name: consumer2 -``` - -# Error Codes - -| HTTP Status Code | Error Info | Reason | -| ----------- | ------------------------------------------------------------------------------ | ---------------------- | -| 401 | Request denied by Basic Auth check. No Basic Authentication information found. | Credentials not provided in the request | -| 401 | Request denied by Basic Auth check. Invalid username and/or password | Invalid username and/or password | -| 403 | Request denied by Basic Auth check. Unauthorized consumer | Unauthorized consumer | \ No newline at end of file +## Related Error Codes +| HTTP Status Code | Error Message | Reason Description | +| ---------------- | ------------------------------------------------------------------------------------- | -------------------------------- | +| 401 | Request denied by Basic Auth check. No Basic Authentication information found. | Request did not provide credentials. | +| 401 | Request denied by Basic Auth check. Invalid username and/or password. | Request credentials are invalid. | +| 403 | Request denied by Basic Auth check. Unauthorized consumer. | The caller making the request does not have access. | diff --git a/plugins/wasm-cpp/extensions/bot_detect/README.md b/plugins/wasm-cpp/extensions/bot_detect/README.md index a9e5e98bca..11311313dd 100644 --- a/plugins/wasm-cpp/extensions/bot_detect/README.md +++ b/plugins/wasm-cpp/extensions/bot_detect/README.md @@ -1,11 +1,21 @@ -

- English | 中文 -

+--- +title: Bot 拦截 +keywords: [higress,bot detect] +description: Bot 拦截插件配置参考 +--- + + +## 功能说明 -# 功能说明 `bot-detect`插件可以用于识别并阻止互联网爬虫对站点资源的爬取 -# 配置字段 +## 运行属性 + +插件执行阶段：`授权阶段` +插件执行优先级：`310` + + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- | -------- | -------- | @@ -33,9 +43,9 @@ (CSimpleSpider|Cityreview Robot|CrawlDaddy|CrawlFire|Finderbots|Index crawler|Job Roboter|KiwiStatus Spider|Lijit Crawler|QuerySeekerSpider|ScollSpider|Trends Crawler|USyd-NLP-Spider|SiteCat Webbot|BotName\/\$BotVersion|123metaspider-Bot|1470\.net crawler|50\.nu|8bo Crawler Bot|Aboundex|Accoona-[A-z]{1,30}-Agent|AdsBot-Google(?:-[a-z]{1,30}|)|altavista|AppEngine-Google|archive.{0,30}\.org_bot|archiver|Ask Jeeves|[Bb]ai[Dd]u[Ss]pider(?:-[A-Za-z]{1,30})(?:-[A-Za-z]{1,30}|)|bingbot|BingPreview|blitzbot|BlogBridge|Bloglovin|BoardReader Blog Indexer|BoardReader Favicon Fetcher|boitho.com-dc|BotSeer|BUbiNG|\b\w{0,30}favicon\w{0,30}\b|\bYeti(?:-[a-z]{1,30}|)|Catchpoint(?: bot|)|[Cc]harlotte|Checklinks|clumboot|Comodo HTTP\(S\) Crawler|Comodo-Webinspector-Crawler|ConveraCrawler|CRAWL-E|CrawlConvera|Daumoa(?:-feedfetcher|)|Feed Seeker Bot|Feedbin|findlinks|Flamingo_SearchEngine|FollowSite Bot|furlbot|Genieo|gigabot|GomezAgent|gonzo1|(?:[a-zA-Z]{1,30}-|)Googlebot(?:-[a-zA-Z]{1,30}|)|Google SketchUp|grub-client|gsa-crawler|heritrix|HiddenMarket|holmes|HooWWWer|htdig|ia_archiver|ICC-Crawler|Icarus6j|ichiro(?:/mobile|)|IconSurf|IlTrovatore(?:-Setaccio|)|InfuzApp|Innovazion Crawler|InternetArchive|IP2[a-z]{1,30}Bot|jbot\b|KaloogaBot|Kraken|Kurzor|larbin|LEIA|LesnikBot|Linguee Bot|LinkAider|LinkedInBot|Lite Bot|Llaut|lycos|Mail\.RU_Bot|masscan|masidani_bot|Mediapartners-Google|Microsoft .{0,30} Bot|mogimogi|mozDex|MJ12bot|msnbot(?:-media {0,2}|)|msrbot|Mtps Feed Aggregation System|netresearch|Netvibes|NewsGator[^/]{0,30}|^NING|Nutch[^/]{0,30}|Nymesis|ObjectsSearch|OgScrper|Orbiter|OOZBOT|PagePeeker|PagesInventory|PaxleFramework|Peeplo Screenshot Bot|PlantyNet_WebRobot|Pompos|Qwantify|Read%20Later|Reaper|RedCarpet|Retreiver|Riddler|Rival IQ|scooter|Scrapy|Scrubby|searchsight|seekbot|semanticdiscovery|SemrushBot|Simpy|SimplePie|SEOstats|SimpleRSS|SiteCon|Slackbot-LinkExpanding|Slack-ImgProxy|Slurp|snappy|Speedy Spider|Squrl Java|Stringer|TheUsefulbot|ThumbShotsBot|Thumbshots\.ru|Tiny Tiny RSS|Twitterbot|WhatsApp|URL2PNG|Vagabondo|VoilaBot|^vortex|Votay bot|^voyager|WASALive.Bot|Web-sniffer|WebThumb|WeSEE:[A-z]{1,30}|WhatWeb|WIRE|WordPress|Wotbox|www\.almaden\.ibm\.com|Xenu(?:.s|) Link Sleuth|Xerka [A-z]{1,30}Bot|yacy(?:bot|)|YahooSeeker|Yahoo! Slurp|Yandex\w{1,30}|YodaoBot(?:-[A-z]{1,30}|)|YottaaMonitor|Yowedo|^Zao|^Zao-Crawler|ZeBot_www\.ze\.bz|ZooShot|ZyBorg)(?:[ /]v?(\d+)(?:\.(\d+)(?:\.(\d+)|)|)|) ``` -# 配置示例 +## 配置示例 -## 放行原本命中爬虫规则的请求 +### 放行原本命中爬虫规则的请求 ```yaml allow: - ".*Go-http-client.*" @@ -44,7 +54,7 @@ allow: 若不作该配置，默认的 Golang 网络库请求会被视做爬虫，被禁止访问 -## 增加爬虫判断 +### 增加爬虫判断 ```yaml deny: - "spd-tools.*" @@ -56,24 +66,3 @@ deny: curl http://example.com -H 'User-Agent: spd-tools/1.1' curl http://exmaple.com -H 'User-Agent: spd-tools' ``` - -## 对特定路由或域名开启 -```yaml -# 使用 _rules_ 字段进行细粒度规则配置 -_rules_: -# 规则一：按路由名称匹配生效 -- _match_route_: - - route-a - - route-b -# 规则二：按域名匹配生效 -- _match_domain_: - - "*.example.com" - - test.com - allow: - - ".*Go-http-client.*" -``` -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将使用此段配置； -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将使用此段配置； -配置的匹配生效顺序，将按照 `_rules_` 下规则的排列顺序，匹配第一个规则后生效对应配置，后续规则将被忽略。 - - diff --git a/plugins/wasm-cpp/extensions/bot_detect/README_EN.md b/plugins/wasm-cpp/extensions/bot_detect/README_EN.md index 83ff3b6fee..1fbaf1a6a6 100644 --- a/plugins/wasm-cpp/extensions/bot_detect/README_EN.md +++ b/plugins/wasm-cpp/extensions/bot_detect/README_EN.md @@ -1,22 +1,26 @@ -

- English | 中文 -

- -# Description -`bot-detect` plugin can be used to identify and prevent web crawlers from crawling websites. - -# Configuration Fields - -| Name | Type | Requirement | Default Value | Description | -| -------- | -------- | -------- | -------- | -------- | -| allow | array of string | Optional | - | A regular expression to match the User-Agent request header and will allow access if the match hits | -| deny | array of string | Optional | - | A regular expression to match the User-Agent request header and will block the request if the match hits | -| blocked_code | number | Optional | 403 | The HTTP status code returned when a request is blocked | -| blocked_message | string | Optional | - | The HTTP response Body returned when a request is blocked | - -If field `allow` and field `deny` are not configured at the same time, the default logic to identify crawlers will be executed. By configuring the `allow` field, requests that would otherwise hit the default logic can be allowed. The judgement can be extended by configuring the `deny` field - -The default set of crawler judgment regular expressions is as follows： +--- +title: Bot Detect +keywords: [higress, bot detect] +description: Bot detect plugin configuration reference +--- +## Function Description +The `bot-detect` plugin can be used to identify and block internet crawlers from accessing site resources. + +## Running Properties +Plugin Execution Phase: `Authorization Phase` +Plugin Execution Priority: `310` + +## Configuration Fields +| Name | Data Type | Required | Default Value | Description | +| ----------------- | ------------------- | --------------| --------------| ---------------------------------------------------------- | +| allow | array of string | Optional | - | Regular expressions to match the User-Agent request header; requests matching will be allowed to access. | +| deny | array of string | Optional | - | Regular expressions to match the User-Agent request header; requests matching will be blocked. | +| blocked_code | number | Optional | 403 | HTTP status code returned when a request is blocked. | +| blocked_message | string | Optional | - | HTTP response body returned when a request is blocked. | + +The `allow` and `deny` fields can both be left unconfigured, in which case the default crawler identification logic will be executed. Configuring the `allow` field can allow requests that would otherwise hit the default crawler identification logic. Configuring the `deny` field can add additional crawler identification logic. + +The default crawler identification regular expression set is as follows: ```bash # Bots General matcher 'name/0.0' @@ -33,45 +37,23 @@ The default set of crawler judgment regular expressions is as follows： (CSimpleSpider|Cityreview Robot|CrawlDaddy|CrawlFire|Finderbots|Index crawler|Job Roboter|KiwiStatus Spider|Lijit Crawler|QuerySeekerSpider|ScollSpider|Trends Crawler|USyd-NLP-Spider|SiteCat Webbot|BotName\/\$BotVersion|123metaspider-Bot|1470\.net crawler|50\.nu|8bo Crawler Bot|Aboundex|Accoona-[A-z]{1,30}-Agent|AdsBot-Google(?:-[a-z]{1,30}|)|altavista|AppEngine-Google|archive.{0,30}\.org_bot|archiver|Ask Jeeves|[Bb]ai[Dd]u[Ss]pider(?:-[A-Za-z]{1,30})(?:-[A-Za-z]{1,30}|)|bingbot|BingPreview|blitzbot|BlogBridge|Bloglovin|BoardReader Blog Indexer|BoardReader Favicon Fetcher|boitho.com-dc|BotSeer|BUbiNG|\b\w{0,30}favicon\w{0,30}\b|\bYeti(?:-[a-z]{1,30}|)|Catchpoint(?: bot|)|[Cc]harlotte|Checklinks|clumboot|Comodo HTTP\(S\) Crawler|Comodo-Webinspector-Crawler|ConveraCrawler|CRAWL-E|CrawlConvera|Daumoa(?:-feedfetcher|)|Feed Seeker Bot|Feedbin|findlinks|Flamingo_SearchEngine|FollowSite Bot|furlbot|Genieo|gigabot|GomezAgent|gonzo1|(?:[a-zA-Z]{1,30}-|)Googlebot(?:-[a-zA-Z]{1,30}|)|Google SketchUp|grub-client|gsa-crawler|heritrix|HiddenMarket|holmes|HooWWWer|htdig|ia_archiver|ICC-Crawler|Icarus6j|ichiro(?:/mobile|)|IconSurf|IlTrovatore(?:-Setaccio|)|InfuzApp|Innovazion Crawler|InternetArchive|IP2[a-z]{1,30}Bot|jbot\b|KaloogaBot|Kraken|Kurzor|larbin|LEIA|LesnikBot|Linguee Bot|LinkAider|LinkedInBot|Lite Bot|Llaut|lycos|Mail\.RU_Bot|masscan|masidani_bot|Mediapartners-Google|Microsoft .{0,30} Bot|mogimogi|mozDex|MJ12bot|msnbot(?:-media {0,2}|)|msrbot|Mtps Feed Aggregation System|netresearch|Netvibes|NewsGator[^/]{0,30}|^NING|Nutch[^/]{0,30}|Nymesis|ObjectsSearch|OgScrper|Orbiter|OOZBOT|PagePeeker|PagesInventory|PaxleFramework|Peeplo Screenshot Bot|PlantyNet_WebRobot|Pompos|Qwantify|Read%20Later|Reaper|RedCarpet|Retreiver|Riddler|Rival IQ|scooter|Scrapy|Scrubby|searchsight|seekbot|semanticdiscovery|SemrushBot|Simpy|SimplePie|SEOstats|SimpleRSS|SiteCon|Slackbot-LinkExpanding|Slack-ImgProxy|Slurp|snappy|Speedy Spider|Squrl Java|Stringer|TheUsefulbot|ThumbShotsBot|Thumbshots\.ru|Tiny Tiny RSS|Twitterbot|WhatsApp|URL2PNG|Vagabondo|VoilaBot|^vortex|Votay bot|^voyager|WASALive.Bot|Web-sniffer|WebThumb|WeSEE:[A-z]{1,30}|WhatWeb|WIRE|WordPress|Wotbox|www\.almaden\.ibm\.com|Xenu(?:.s|) Link Sleuth|Xerka [A-z]{1,30}Bot|yacy(?:bot|)|YahooSeeker|Yahoo! Slurp|Yandex\w{1,30}|YodaoBot(?:-[A-z]{1,30}|)|YottaaMonitor|Yowedo|^Zao|^Zao-Crawler|ZeBot_www\.ze\.bz|ZooShot|ZyBorg)(?:[ /]v?(\d+)(?:\.(\d+)(?:\.(\d+)|)|)|) ``` -# Configuration Samples - -## Release Requests that would otherwise Hit the Crawler Rules +## Configuration Example +### Allowing Requests That Hit the Crawler Rules ```yaml allow: - ".*Go-http-client.*" ``` -Without this configuration, the default Golang web library request will be treated as a crawler and access will be denied. +If this configuration is not made, requests from the default Golang network library will be treated as crawlers and blocked. - -## Add Crawler Judgement +### Adding Crawler Identification ```yaml deny: - "spd-tools.*" ``` -According to this configuration, the following requests will be denied: - +With this configuration, the following requests will be blocked: ```bash curl http://example.com -H 'User-Agent: spd-tools/1.1' curl http://exmaple.com -H 'User-Agent: spd-tools' ``` - -## Only Enabled for Specific Routes or Domains -```yaml -# Use _rules_ field for fine-grained rule configurations -_rules_: -# Rule 1: Match by route name -- _match_route_: - - route-a - - route-b -# Rule 2: Match by domain -- _match_domain_: - - "*.example.com" - - test.com - allow: - - ".*Go-http-client.*" -``` -In the rule sample of `_match_route_`, `route-a` and `route-b` are the route names provided when creating a new gateway route. When the current route names matches the configuration, the rule following shall be applied. -In the rule sample of `_match_domain_`, `*.example.com` and `test.com` are the domain names used for request matching. When the current domain name matches the configuration, the rule following shall be applied. -All rules shall be checked following the order of items in the `_rules_` field, The first matched rule will be applied. All remained will be ignored. diff --git a/plugins/wasm-cpp/extensions/custom_response/README.md b/plugins/wasm-cpp/extensions/custom_response/README.md index bdf48f7b28..e59863441d 100644 --- a/plugins/wasm-cpp/extensions/custom_response/README.md +++ b/plugins/wasm-cpp/extensions/custom_response/README.md @@ -1,11 +1,19 @@ -

- English | 中文 -

+--- +title: 自定义应答 +keywords: [higress,customn response] +description: 自定义应答插件配置参考 +--- -# 功能说明 + +## 功能说明 `custom-response`插件支持配置自定义的响应，包括自定义 HTTP 应答状态码、HTTP 应答头，以及 HTTP 应答 Body。可以用于 Mock 响应，也可以用于判断特定状态码后给出自定义应答，例如在触发网关限流策略时实现自定义响应。 -# 配置字段 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`910` + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- | -------- | -------- | @@ -14,9 +22,9 @@ | body | string | 选填 | - | 自定义 HTTP 应答 Body | | enable_on_status | array of number | 选填 | - | 匹配原始状态码，生成自定义响应，不填写时，不判断原始状态码 | -# 配置示例 +## 配置示例 -## Mock 应答场景 +### Mock 应答场景 ```yaml status_code: 200 @@ -38,7 +46,7 @@ Content-Length: 17 {"hello":"world"} ``` -## 触发限流时自定义响应 +### 触发限流时自定义响应 ```yaml enable_on_status: @@ -58,27 +66,3 @@ Location: https://example.com 从而实现基于浏览器 302 重定向机制，将限流后的用户引导到其他页面，比如可以是一个 CDN 上的静态页面。 如果希望触发限流时，正常返回其他应答，参考 Mock 应答场景配置相应的字段即可。 - -## 对特定路由或域名开启 -```yaml -# 使用 _rules_ 字段进行细粒度规则配置 -_rules_: -# 规则一：按路由名称匹配生效 -- _match_route_: - - route-a - - route-b - body: "{\"hello\":\"world\"}" -# 规则二：按域名匹配生效 -- _match_domain_: - - "*.example.com" - - test.com - enable_on_status: - - 429 - status_code: 200 - headers: - - Content-Type=application/json - body: "{\"errmsg\": \"rate limited\"}" -``` -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将使用此段配置； -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将使用此段配置； -配置的匹配生效顺序，将按照 `_rules_` 下规则的排列顺序，匹配第一个规则后生效对应配置，后续规则将被忽略。 diff --git a/plugins/wasm-cpp/extensions/custom_response/README_EN.md b/plugins/wasm-cpp/extensions/custom_response/README_EN.md index dcab23f7e3..365696e20a 100644 --- a/plugins/wasm-cpp/extensions/custom_response/README_EN.md +++ b/plugins/wasm-cpp/extensions/custom_response/README_EN.md @@ -1,84 +1,54 @@ -

- English | 中文 -

+--- +title: Custom Response +keywords: [higress, custom response] +description: Custom response plugin configuration reference +--- +## Function Description +The `custom-response` plugin supports the configuration of custom responses, including custom HTTP response status codes, HTTP response headers, and HTTP response bodies. It can be used for Mock responses or for providing custom responses based on specific status codes, such as implementing custom responses when triggering the gateway rate-limiting policy. -# Description -`custom-response` plugin implements a function of sending custom responses, including custom HTTP response status codes, HTTP response headers and HTTP response body, which can be used in the scenarios of response mocking and sending a custom response for specific status codes, such as customizing the response for rate-limited requests. +## Running Attributes +Plugin Execution Phase: `Authentication Phase` -# Configuration Fields +Plugin Execution Priority: `910` -| Name | Type | Requirement | Default Value | Description | +## Configuration Fields +| Name | Data Type | Requirements | Default Value | Description | | -------- | -------- | -------- | -------- | -------- | | status_code | number | Optional | 200 | Custom HTTP response status code | -| headers | array of string | Optional | - | Custom HTTP response header. Key and value shall be separated using `=`. | +| headers | array of string | Optional | - | Custom HTTP response headers, keys and values separated by `=` | | body | string | Optional | - | Custom HTTP response body | -| enable_on_status | array of number | Optional | - | The original response status code to match. Generate the custom response only the actual status code matches the configuration. Ignore the status code match if left unconfigured. | - -# Configuration Samples - -## Mock Responses +| enable_on_status | array of number | Optional | - | Match original status codes to generate custom responses; if not specified, the original status code is not checked | +## Configuration Example +### Mock Response Scenario ```yaml status_code: 200 headers: - Content-Type=application/json - Hello=World body: "{\"hello\":\"world\"}" - ``` - -According to the configuration above, all the requests will get the following custom response: - +With this configuration, the request will return the following custom response: ```text HTTP/1.1 200 OK Content-Type: application/json Hello: World Content-Length: 17 - {"hello":"world"} ``` - -## Send a Custom Response when Rate-Limited - +### Custom Response on Rate Limiting ```yaml -enable_on_status: +enable_on_status: - 429 status_code: 302 headers: - Location=https://example.com ``` - -When rate-limited, normally gateway will return a status code of `429` . Now, rate-limited requests will get the following custom response: - +When the gateway rate limiting is triggered, it generally returns the `429` status code, and the request will return the following custom response: ```text HTTP/1.1 302 Found Location: https://example.com ``` +This achieves the goal of redirecting users who have been rate-limited to another page based on the browser's 302 redirect mechanism, which could be a static page on a CDN. -So based on the 302 redirecting mechanism provided by browsers, this can redirect rate-limited users to other pages, for example, a static page hosted on CDN. - -If you'd like to send other responses when rate-limited, please add other fields into the configuration, referring to the Mock Responses scenario. - -## Only Enabled for Specific Routes or Domains -```yaml -# Use _rules_ field for fine-grained rule configurations -_rules_: -# Rule 1: Match by route name -- _match_route_: - - route-a - - route-b - body: "{\"hello\":\"world\"}" -# Rule 2: Match by domain -- _match_domain_: - - "*.example.com" - - test.com - enable_on_status: - - 429 - status_code: 200 - headers: - - Content-Type=application/json - body: "{\"errmsg\": \"rate limited\"}" -``` -In the rule sample of `_match_route_`, `route-a` and `route-b` are the route names provided when creating a new gateway route. When the current route names matches the configuration, the rule following shall be applied. -In the rule sample of `_match_domain_`, `*.example.com` and `test.com` are the domain names used for request matching. When the current domain name matches the configuration, the rule following shall be applied. -All rules shall be checked following the order of items in the `_rules_` field, The first matched rule will be applied. All remained will be ignored. +If you wish to return other responses normally when rate limiting is triggered, just refer to the Mock response scenario to configure the relevant fields accordingly. diff --git a/plugins/wasm-cpp/extensions/hmac_auth/README.md b/plugins/wasm-cpp/extensions/hmac_auth/README.md index 719c3d9ef1..dba464a877 100644 --- a/plugins/wasm-cpp/extensions/hmac_auth/README.md +++ b/plugins/wasm-cpp/extensions/hmac_auth/README.md @@ -1,13 +1,32 @@ -# 功能说明 +--- +title: HMAC 认证 +keywords: [higress,hmac auth] +description: HMAC 认证插件配置参考 +--- + +## 功能说明 `hmac-auth`插件实现了基于 HMAC 算法为 HTTP 请求生成不可伪造的签名，并基于签名实现身份认证和鉴权 -# 配置字段 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`330` + +## 配置字段 + +**注意：** + +- 在一个规则里，鉴权配置和认证配置不可同时存在 +- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 + +### 认证配置 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | ------------- | --------------- | -------- | ------ | ------------------------------------------------------------------------------------------------------------------- | +| `global_auth` | bool | 选填（**仅实例级别配置**） | - | 只能在实例级别配置，若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制，若不配置则仅当没有域名和路由配置时全局生效（兼容老用户使用习惯）。 | | `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | | `date_offset` | number | 选填 | - | 配置允许的客户端最大时间偏移，单位为秒，根据请求头`Date`解析客户端 UTC 时间，可用于避免请求重放；未配置时，不做校验 | -| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表，用于对请求进行鉴权 | + `consumers`中每一项的配置字段说明如下： @@ -17,24 +36,21 @@ | `secret` | string | 必填 | - | 配置用于生成签名的secret | | `name` | string | 必填 | - | 配置该consumer的名称 | -`_rules_` 中每一项的配置字段说明如下： +### 鉴权配置（非必需） -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的路由名称 | -| `_match_domain_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的域名 | -| `allow` | array of string | 必填 | - | 对于符合匹配条件的请求，配置允许访问的consumer名称 | +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `allow` | array of string | 选填(**非实例级别配置**) | - | 只能在路由或域名等细粒度规则上配置，对于符合匹配条件的请求，配置允许访问的 consumer，从而实现细粒度的权限控制 | -**注意：** -- 若不配置`_rules_`字段，则默认对当前网关实例的所有路由开启认证； -- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 -# 配置示例 +## 配置示例 -以下配置将对网关特定路由或域名开启 Hmac Auth 认证和鉴权，注意`key`字段不能重复 +### 全局配置认证和路由粒度进行鉴权 + +在实例级别做如下插件配置, 注意`key`字段不能重复: -## 对特定路由或域名开启 ```yaml +global_auth: false consumers: - key: appKey-example-1 secret: appSecret-example-1 @@ -42,34 +58,33 @@ consumers: - key: appKey-example-2 secret: appSecret-example-2 name: consumer-2 -# 使用 _rules_ 字段进行细粒度规则配置 -_rules_: -# 规则一：按路由名称匹配生效 -- _match_route_: - - route-a - - route-b - allow: - - consumer-1 -# 规则二：按域名匹配生效 -- _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer-2 ``` -每条匹配规则下的`allow`字段用于指定该匹配条件下允许访问的调用者列表； -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许`name`为`consumer-1`的调用者访问，其他调用者不允许访问； +route-a和route-b两个路由做如下插件配置： + +```yaml +allow: +- consumer1 +``` -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将允许`name`为`consumer-2`的调用者访问，其他调用者不允许访问； +在*.example.com和test.com两个域名做如下插件配置： -认证成功后，请求的header中会被添加一个`X-Mse-Consumer`字段，其值为调用方的名称，例如`consumer-1`。 +```yaml +allow: +- consumer2 +``` -## 网关实例级别开启 +若是在控制台进行配置，此例指定的route-a和route-b即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许name为consumer1的调用者访问，其他调用者不允许访问。 -以下配置将对网关实例级别开启 Hamc Auth 认证 +此例指定的*.example.com和test.com用于匹配请求的域名，当发现域名匹配时，将允许name为consumer2的调用者访问，其他调用者不被允许访问。 + + +### 网关实例级别开启 + +以下配置将对网关实例级别开启 Hamc Auth 认证，所有请求均需要经过认证后才能访问。 ```yaml +global_auth: true consumers: - key: appKey-example-1 secret: appSecret-example-1 @@ -80,18 +95,18 @@ consumers: ``` -# 签名机制说明 +## 签名机制说明 -## 配置准备 +### 配置准备 如上指引，在插件配置中配置生成和验证签名需要用的凭证配置 - key: 用于请求头 `x-ca-key` 中设置 - secret: 用于生成请求签名 -## 客户端签名生成方式 +### 客户端签名生成方式 -### 流程简介 +#### 流程简介 客户端生成签名一共分三步处理： @@ -104,7 +119,7 @@ consumers: 如下图所示： ![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/1745707061/p188113.png) -### 签名串提取流程 +#### 签名串提取流程 客户端需要从Http请求中提取出关键数据，组合成一个签名串，生成的签名串的格式如下： @@ -160,7 +175,7 @@ Path + "?" + Key1 + "=" + Value1 + "&" + Key2 + "=" + Value2 + ... "&" + KeyN + 4. Query和Form存在数组参数时（key相同，value不同的参数） ，取第一个Value参与签名计算 -### 签名串提取示例 +#### 签名串提取示例 初始的HTTP请求： ```text @@ -190,7 +205,7 @@ x-ca-timestamp:1525872629832 /http2test/test?param1=test&password=123456789&username=xiaoming ``` -### 签名计算流程 +#### 签名计算流程 客户端从HTTP请求中提取出关键数据组装成签名串后，需要对签名串进行加密及编码处理，形成最终的签名 @@ -206,7 +221,7 @@ String sign = Base64.encodeBase64String(result); 总结一下，就是将 `stringToSign` 使用UTF-8解码后得到Byte数组，然后使用加密算法对Byte数组进行加密，然后使用Base64算法进行编码，形成最终的签名。 -### 添加签名流程 +#### 添加签名流程 客户端需要将以下四个Header放在HTTP请求中传输给API网关，进行签名校验： @@ -238,9 +253,9 @@ content-length:33 username=xiaoming&password=123456789 ``` -## 服务端签名验证方式 +### 服务端签名验证方式 -### 流程简介 +#### 流程简介 服务器验证客户端签名一共分四步处理： @@ -256,7 +271,7 @@ username=xiaoming&password=123456789 ![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/1745707061/p188116.png) -## 签名排错方法 +### 签名排错方法 网关签名校验失败时，会将服务端的签名串（StringToSign）放到HTTP Response的Header中返回到客户端，Key为：X-Ca-Error-Message，用户只需要将本地计算的签名串（StringToSign）与服务端返回的签名串进行对比即可找到问题； @@ -269,7 +284,7 @@ X-Ca-Error-Message: Server StringToSign:`GET#application/json##application/json ``` -# 相关错误码 +## 相关错误码 | HTTP 状态码 | 出错信息 | 原因说明 | | ----------- | ---------------------- | -------------------------------------------------------------------------------- | @@ -281,5 +296,3 @@ X-Ca-Error-Message: Server StringToSign:`GET#application/json##application/json | 413 | Request Body Too Large | 请求 Body 超过限制大小：32 MB | | 413 | Payload Too Large | 请求 Body 超过全局配置 DownstreamConnectionBufferLimits | | 403 | Unauthorized Consumer | 请求的调用方无访问权限 | - - diff --git a/plugins/wasm-cpp/extensions/hmac_auth/README_EN.md b/plugins/wasm-cpp/extensions/hmac_auth/README_EN.md index d6966e9637..0e9f2c4b7a 100644 --- a/plugins/wasm-cpp/extensions/hmac_auth/README_EN.md +++ b/plugins/wasm-cpp/extensions/hmac_auth/README_EN.md @@ -1,76 +1,71 @@ -# Function Description -The `hmac-auth` plugin implements the generation of tamper-proof signatures for HTTP requests based on HMAC algorithm, and uses the signature for identity authentication and authorization. - -# Configuration Fields - -| Name | Data Type | Required | Default | Description | -| ------------- | --------------- | -------------| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `consumers` | array of object | Required | - | Configures the caller of the service to authenticate the request. | -| `date_offset` | number | Optional | - | Configures the maximum allowed time deviation of the client, in seconds. It is used to parse the client's UTC time from `the Date` header of the request, and can be used to prevent replay attacks. If not configured, no validation is performed. | -| `_rules_` | array of object | Optional | - | Configures the access control list for specific routes or domains, used for authorization of requests. | - -The configuration fields for each item in `consumers` are as follows : - -| Name | Data Type| Required | Default| Description | -| -------- | -------- | ------------ | ------ | ----------------------------------------------------------------------- | -| `key` | string | Required | - | Configures the key extracted from the `x-ca-key` header of the request. | -| `secret` | string | Required | - | Configures the secret used to generate the signature. | -| `name` | string | Required | - | Configures the name of the consumer. | - -The configuration fields for each item in `_rules_` are as follows: - -| Name | Data Type | Required | Default | Description | -| ---------------- | --------------- | ------------------------------------------------- | ---------------------------- | -------------------------------------------------- | -| `_match_route_` | array of string | Optional, either `_match_route_` or `_match_domain_` must be provided | - | Configures the name of the route to match. | -| `_match_domain_` | array of string | Optional, either `_match_route_` or `_match_domain_` must be provided | - | Configures the name of the domain to match. | -| `allow` | array of string | Required | - | Configures the name of the consumer to allow for requests that match the specified route or domain. | - -**Note：** -- If `_rules_` is not configured, authentication is enabled for all routes on the current gateway instance by default ； -- For requests that pass authentication and authorization, a `X-Mse-Consumer` header will be added to the request headers to identify the name of the consumer. - -# Configuration Example - -The following configuration enables Hmac Auth authentication and authorization for specific routes or domains on the gateway. Note that the `key` field should not be duplicated. - -## Enabling for specific routes or domains +--- +title: HMAC Authentication +keywords: [higress,hmac auth] +description: HMAC Authentication plugin configuration reference +--- +## Function Description +The `hmac-auth` plugin implements the generation of tamper-proof signatures for HTTP requests based on the HMAC algorithm, and performs authentication and authorization based on the signature. + +## Running Attributes +Plugin execution phase: `Authentication phase` +Plugin execution priority: `330` + +## Configuration Fields +**Note:** +- In a rule, authentication and authorization configurations cannot coexist. +- For requests that pass authentication and authorization, the request header will be added with an `X-Mse-Consumer` field to identify the caller's name. + +### Authentication Configuration +| Name | Data Type | Requirement | Default Value | Description | +| ------------- | ---------------- | ------------------- | ------------- | ----------------------------------------------------------------------------------------------------------------------- | +| `global_auth` | bool | Optional (**Instance level configuration only**) | - | Can only be configured at the instance level. If set to true, it acts globally; if false, only applies to configured domains and routes. If not configured, it will apply globally only when there are no domain and route configurations (to accommodate old user habits). | +| `consumers` | array of object | Mandatory | - | Configures the callers of the service for request authentication. | +| `date_offset` | number | Optional | - | Configures the maximum allowed client time offset, in seconds; parsed based on the request header `Date`; can be used to prevent request replay; no validation is performed if not configured. | + +The configuration fields for each item in `consumers` are as follows: +| Name | Data Type | Requirement | Default Value | Description | +| -------- | --------- | ----------- | ------------- | ------------------------------------------- | +| `key` | string | Mandatory | - | Configures the key extracted from the `x-ca-key` header of the request. | +| `secret` | string | Mandatory | - | Configures the secret used to generate the signature. | +| `name` | string | Mandatory | - | Configures the name of the consumer. | + +### Authorization Configuration (Optional) +| Name | Data Type | Requirement | Default Value | Description | +| ----------- | ---------------- | --------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `allow` | array of string | Optional (**Non-instance level configuration**) | - | Can only be configured on granular rules such as routes or domains. For requests that match the conditions, configure the allowed consumers to achieve fine-grained permission control. | + +## Configuration Example +### Global Configuration Authentication and Route Granular Authorization +Configure the following plugin settings at the instance level. Note that the `key` field cannot be duplicated: ```yaml -consumers: +global_auth: false +consumers: - key: appKey-example-1 secret: appSecret-example-1 name: consumer-1 - key: appKey-example-2 secret: appSecret-example-2 name: consumer-2 -# Configuring Fine-Grained Rules using _rules_ Field -_rules_: -# Rule 1: Matching by route name. -- _match_route_: - - route-a - - route-b - allow: - - consumer-1 -# Rule 2: Applies based on domain name matching. -- _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer-2 ``` -The `allow` field under each matching rule specifies the list of callers allowed to access under that matching condition; - -In this example, `route-a` and `route-b` specified in `_match_route_` are the route names filled in when creating the gateway route. When either of these routes is matched, it will allow access to the caller named `consumer-1`, while denying access to other callers； - -In` _match_domain_`, `*.example.com` and `test.com` are used to match the requested domain name. When a match is found, it will allow access to the caller named `consumer-2`, while denying access to other callers； - -Upon successful authentication, the `X-Mse-Consumer` field will be added to the request header with the value set to the caller's name, such as `consumer-1`.。 - -## Enable at the Gateway Instance Level +For route-a and route-b, configure the plugin as follows: +```yaml +allow: +- consumer1 +``` +For the two domains *.example.com and test.com, configure as follows: +```yaml +allow: +- consumer2 +``` +If configured in the console, the specified route names route-a and route-b correspond to the route names filled in when creating the gateway routes. When matched to these two routes, access will be allowed for the caller named consumer1, while other callers will not be allowed access. -The following configuration enables HMAC authentication at the gateway instance level. +The specified *.example.com and test.com are used to match the domains of the requests. When a domain match is found, access will be allowed for the caller named consumer2, while other callers will not be allowed access. +### Gateway Instance Level Activation +The following configuration will enable HMAC Auth authentication at the gateway instance level, requiring all requests to undergo authentication before access. ```yaml -consumers: +global_auth: true +consumers: - key: appKey-example-1 secret: appSecret-example-1 name: consumer-1 @@ -79,34 +74,24 @@ consumers: name: consumer-2 ``` +## Signature Mechanism Description +### Configuration Preparation +As mentioned above, configure the credentials required for generating and verifying signatures in the plugin settings. +- key: to be set in the request header `x-ca-key`. +- secret: used for generating request signatures. -# Description of Signing Mechanism - -## Configuration Preparation - -As mentioned in the guide above, configure the credential settings required for generating and validating signatures in the plugin configuration. - -- key: Used for setting in the request header `x-ca-key`. -- secret: Used for generating the request signature. - -## Client Signature Generation Method -### Overview of the Process - -The process for generating a signature on the client side consists of three steps: - -1. Extracting key data from the original request to obtain a string to be signed. +### Client Signature Generation Process +#### Overview +The client generates a signature through three main steps: +1. Extract key data from the original request to create a string for signing. +2. Encrypt the key data signing string using the algorithm and the configured `secret` to obtain the signature. +3. Include all relevant headers for the signature into the original HTTP request to form the final HTTP request. -2. Using encryption algorithms and the configured `secret` to encrypt the key data signing string and obtain a signature. - -3. Adding all headers related to the signature to the original HTTP request to obtain the final HTTP request. - -As shown below : +As shown in the figure below: ![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/1745707061/p188113.png) -### Process for Extracting Signing String - -To generate a signature, the client needs to extract key data from the HTTP request and combine it into a signing string. The format of the generated signing string is as follows: - +#### Signing String Extraction Process +The client needs to extract key data from the HTTP request, combine it into a signing string, which has the following format: ```text HTTPMethod Accept @@ -116,54 +101,38 @@ Date Headers PathAndParameters ``` - -The signing string consists of the above 7 fields separated by \n. If Headers is empty, no \n is needed. If other fields are empty, the \n should still be retained. The signature is case-sensitive. Below are the rules for extracting each field: - -- HTTPMethod: The HTTP method used in the request, in all capital letters, such as POST. - -- Accept: The value of the Accept header in the request, which can be empty. It is recommended to explicitly set the Accept header. When Accept is empty, some HTTP clients will set the default value of `*/*`, which may cause signature verification to fail. - -- Content-MD5: The value of the Content-MD5 header in the request, which can be empty. It is only calculated when there is a non-form body in the request. The following is a reference calculation method for Content-MD5 values in ： - - +The seven fields above constitute the entire signing string, separated by newline characters `\n`. If Headers is empty, no newline is needed; other fields should retain `\n` if empty. The signature is case-sensitive. Below are the extraction rules for each field: +- HTTPMethod: The HTTP method, all uppercase (e.g., POST). +- Accept: The value of the Accept header in the request, can be empty. It is recommended to explicitly set the Accept Header. When Accept is empty, some HTTP clients may set a default value of `*/*`, resulting in a signature verification failure. +- Content-MD5: The value of the Content-MD5 header in the request, can be empty. It is calculated only if there is a Body in the request and it is not in Form format. Here’s a reference calculation method for the Content-MD5 value in Java: ```java -String content-MD5 = Base64.encodeBase64(MD5(bodyStream.getbytes("UTF-8"))); +String content-MD5 = Base64.encodeBase64(MD5(bodyStream.getBytes("UTF-8"))); ``` - -- Content-Type: The value of the Content-Type header in the request, which can be empty. - -- Date: The value of the Date header in the request. When the` date_offset` configuration is not enabled, it can be empty. Otherwise, it will be used for time offset verification. - -- Headers: Users can select specific headers to participate in the signature. There are the following rules for concatenating the signature string with headers: - - The keys of the headers participating in the signature calculation are sorted in alphabetical order and concatenated as follows: +- Content-Type: The value of the Content-Type header in the request, can be empty. +- Date: The value of the Date header in the request. If the `date_offset` configuration is not turned on, it can be empty; otherwise, it will be used for time offset verification. +- Headers: Users can select specific headers to participate in the signature. The rules for concatenating the signing header string are as follows: + - The Keys of the headers participating in the signature calculation are concatenated after being sorted lexicographically, as follows: ```text - HeaderKey1 + ":" + HeaderValue1 + "\n"\+ - HeaderKey2 + ":" + HeaderValue2 + "\n"\+ + HeaderKey1 + ":" + HeaderValue1 + "\n" + + HeaderKey2 + ":" + HeaderValue2 + "\n" + ... HeaderKeyN + ":" + HeaderValueN + "\n" ``` - - If the value of a header is empty, it will participate in the signature with the `HeaderKey+":"+"\n"` only, and the key and english colon should be retained. - - The set of keys for all headers participating in the signature is separated by a comma and placed in the `X-Ca-Signature-Headers header`. + - If the Value of a certain header is empty, use HeaderKey + ":" + "\n" to participate in the signature, retaining the Key and the colon. + - The collection of all participating header Keys is placed in the Header with the key X-Ca-Signature-Headers, separated by commas. - The following headers are not included in the header signature calculation: X-Ca-Signature, X-Ca-Signature-Headers, Accept, Content-MD5, Content-Type, Date. - -- PathAndParameters: This field contains all parameters in the path, query, and form. The specific format is as follows: - +- PathAndParameters: This field includes Path, Query, and all parameters in Form, specifically organized as follows: ```text Path + "?" + Key1 + "=" + Value1 + "&" + Key2 + "=" + Value2 + ... "&" + KeyN + "=" + ValueN ``` - -Notes: -1. The keys of the query and form parameter pairs are sorted alphabetically, and the same format as above is used for concatenation. - -2. If there are no query and form parameters, use the path directly without adding `?` . - -3. If the value of a parameter is empty, only the key will be included in the signature. The equal sign should not be included in the signature. - -4. If there are array parameters in the query or form (parameters with the same key but different values), only the first value should be included in the signature calculation. - -### Example of Extracting Signing String - -The initial HTTP request : +Note: +1. The Key of Query and Form parameters should be sorted lexicographically before being concatenated as above. +2. If Query and Form parameters are empty, just use Path without adding `?`. +3. If the Value of parameters is empty, only the Key should be retained in the signature, the equal sign does not need to be added. +4. In the case of array parameters (parameters with the same key but different values), only the first Value should be used for signature calculation. + +#### Signing String Extraction Example +Initial HTTP request: ```text POST /http2test/test?param1=test HTTP/1.1 host:api.aliyun.com @@ -177,8 +146,7 @@ x-ca-nonce:c9f15cbf-f4ac-4a6c-b54d-f51abf4b5b44 content-length:33 username=xiaoming&password=123456789 ``` - -The correct generated signature string is : +The generated correct signing string is: ```text POST application/json; charset=utf-8 @@ -190,13 +158,10 @@ x-ca-signature-method:HmacSHA256 x-ca-timestamp:1525872629832 /http2test/test?param1=test&password=123456789&username=xiaoming ``` +#### Signature Calculation Process +After the client assembles the key data extracted from the HTTP request into a signing string, it needs to encrypt the signing string and encode it to form the final signature. -### Signature Calculation Process - -After extracting the key data from the HTTP request and assembling it into a signature string, the client needs to encrypt and encode the signature string to form the final signature. - -The specific encryption format is as follows, where `stringToSign` is the extracted signature string, `secret` is the one filled in the plugin configuration, and `sign` is the final generated signature: - +The specific encryption form is as follows, where `stringToSign` is the extracted signing string, `secret` is the one filled in the plugin configuration, and `sign` is the final generated signature: ```java Mac hmacSha256 = Mac.getInstance("HmacSHA256"); byte[] secretBytes = secret.getBytes("UTF-8"); @@ -204,23 +169,16 @@ hmacSha256.init(new SecretKeySpec(secretBytes, 0, secretBytes.length, "HmacSHA25 byte[] result = hmacSha256.doFinal(stringToSign.getBytes("UTF-8")); String sign = Base64.encodeBase64String(result); ``` +To summarize, the `stringToSign` is decoded using UTF-8 to obtain a Byte array, then the encryption algorithm is applied to the Byte array, and finally, the Base64 algorithm is used for encoding, forming the final signature. -In summary, the `stringToSign` is decoded using UTF-8 to obtain a Byte array. Then, an encryption algorithm is used to encrypt the Byte array, and finally, the Base64 algorithm is used to encode the encrypted data, resulting in the final signature. - -### The Process of Adding a Signature - -The client needs to include the following four headers in the HTTP request to be transmitted to the API gateway for signature verification: - -- x-ca-key: The value is the APP Key and is required. - -- x-ca-signature-method: The signature algorithm, the value can be HmacSHA256 or HmacSHA1, optional. The default value is HmacSHA256. - -- x-ca-signature-headers: The collection of keys for all signature headers, separated by commas. Optional. - -- x-ca-signature: The signature and it is required. - -Here is an example of a complete HTTP request with a signature : +#### Adding the Signature Process +The client needs to include the following four headers in the HTTP request to transmit to the API gateway for signature verification: +- x-ca-key: The APP Key, mandatory. +- x-ca-signature-method: The signature algorithm, can be HmacSHA256 or HmacSHA1, optional, default is HmacSHA256. +- x-ca-signature-headers: The collection of all signature header Keys, separated by commas, optional. +- x-ca-signature: The signature, mandatory. +Below is an example of the entire HTTP request carrying the signature: ```text POST /http2test/test?param1=test HTTP/1.1 host:api.aliyun.com @@ -239,48 +197,35 @@ content-length:33 username=xiaoming&password=123456789 ``` -## Server-side Signature Verification Method - -### Overview of the Process - -The server-side signature verification of the client's request involves four steps : - -1. Extract crucial data from the received request to obtain a string for signing. - -2. Retrieve the `key` from the received request and use it to query its corresponding `secret`. - -3. Encrypt the string for signing using the encryption algorithm and `secret`. - -4. Retrieve the client's signature from the received request, and compare the consistency of the server-side signature with the client's signature. +### Server-side Signature Verification Method +#### Overview +The server verifies the client signature through four main steps: +1. Extract key data from the received request to create a signing string. +2. Read the `key` from the received request and query the corresponding `secret`. +3. Encrypt the key data signing string using the algorithm and the `secret` to obtain the signature. +4. Read the client signature from the received request and compare the server-side signature with the client-side signature for consistency. -As shown below : +As shown in the figure below: ![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/1745707061/p188116.png) +### Signature Troubleshooting Method +When the gateway signature verification fails, the server's signing string (StringToSign) will be returned in the HTTP Response header to the client, with the key: X-Ca-Error-Message. The user only needs to compare the locally computed signing string (StringToSign) with the signing string returned by the server to find the issue. -## Troubleshooting Signature Errors - -When the gateway signature verification fails, the server-side signing string (StringToSign) will be returned to the client in the HTTP Response Header. The key is X-Ca-Error-Message. Users only need to compare the locally calculated signing string with the server-side signing string returned to locate the problem; - -If the StringToSign on the server side is consistent with that on the client side, please check whether the APP Secret used for signature calculation is correct； - -Because line breaks cannot be represented in HTTP headers, all line breaks in the StringToSign are replaced with #, as shown below: +If the StringToSign from the server and the client are consistent, please check whether the APP Secret used for signature calculation is correct. +Since HTTP headers cannot express line breaks, the line breaks in the StringToSign have been replaced with `#`, as shown below: ```text X-Ca-Error-Message: Server StringToSign:`GET#application/json##application/json##X-Ca-Key:200000#X-Ca-Timestamp:1589458000000#/app/v1/config/keys?keys=TEST` - ``` -# Related Error Codes - -| HTTP Status Code | Error Message | Reason | -| ----------- | ---------------------- | -------------------------------------------------------------------------------- | -| 401 | Invalid Key | The x-ca-key request header is not provided or is invalid. | -| 401 | Empty Signature | The x-ca-signature request header does not contain a signature. | -| 400 | Invalid Signature | The x-ca-signature request header contains a signature that does not match the server-calculated signature. | -| 400 | Invalid Content-MD5 | The content-md5 request header is incorrect. | -| 400 | Invalid Date | The time offset calculated based on the date request header exceeds the configured date_offset. | -| 413 | Request Body Too Large | The request body exceeds the size limit of 32 MB. | -| 413 | Payload Too Large | The request body exceeds the DownstreamConnectionBufferLimits global configuration. | -| 403 | Unauthorized Consumer | The requesting party does not have access permission. | - - +## Related Error Codes +| HTTP Status Code | Error Message | Reasoning | +| ---------------- | --------------------- | --------------------------------------------- | +| 401 | Invalid Key | The request header did not provide x-ca-key, or x-ca-key is invalid. | +| 401 | Empty Signature | The request header did not provide the x-ca-signature signing string. | +| 400 | Invalid Signature | The x-ca-signature signing string in the request header does not match the signature calculated by the server. | +| 400 | Invalid Content-MD5 | The Content-MD5 header in the request is incorrect. | +| 400 | Invalid Date | The time offset calculated based on the Date header in the request exceeds the configured date_offset. | +| 413 | Request Body Too Large| The request Body exceeds the maximum size of 32 MB. | +| 413 | Payload Too Large | The request Body exceeds the global configured DownstreamConnectionBufferLimits. | +| 403 | Unauthorized Consumer | The calling party does not have access permissions for the request. | diff --git a/plugins/wasm-cpp/extensions/jwt_auth/README.md b/plugins/wasm-cpp/extensions/jwt_auth/README.md index 6e98687094..6a5078bbf6 100644 --- a/plugins/wasm-cpp/extensions/jwt_auth/README.md +++ b/plugins/wasm-cpp/extensions/jwt_auth/README.md @@ -1,17 +1,216 @@ -# 功能说明 +--- +title: JWT 认证 +keywords: [higress,jwt auth] +description: JWT 认证插件配置参考 +--- + +## 功能说明 `jwt-auth`插件实现了基于JWT(JSON Web Tokens)进行认证鉴权的功能，支持从HTTP请求的URL参数、请求头、Cookie字段解析JWT，同时验证该Token是否有权限访问。 本插件和`安全能力->认证鉴权`中JWT认证的区别是，额外提供了调用方身份识别的能力，支持对不同调用方配置不同的JWT凭证。 -# 详细说明 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`340` + +## 配置字段 + +**注意：** + +- 在一个规则里，鉴权配置和认证配置不可同时存在 +- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 + +### 认证配置 + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `global_auth` | bool | 选填（**仅实例级别配置**） | - | 只能在实例级别配置，若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制，若不配置则仅当没有域名和路由配置时全局生效（兼容老用户使用习惯）。 | +| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | + +`consumers`中每一项的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------------------- | ----------------- | -------- | ------------------------------------------------- | ------------------------ | +| `name` | string | 必填 | - | 配置该consumer的名称 | +| `jwks` | string | 必填 | - | https://www.rfc-editor.org/rfc/rfc7517 指定的json格式字符串，是由验证JWT中签名的公钥（或对称密钥）组成的Json Web Key Set | +| `issuer` | string | 必填 | - | JWT的签发者，需要和payload中的iss字段保持一致 | +| `claims_to_headers` | array of object | 选填 | - | 抽取JWT的payload中指定字段，设置到指定的请求头中转发给后端 | +| `from_headers` | array of object | 选填 | {"name":"Authorization","value_prefix":"Bearer "} | 从指定的请求头中抽取JWT | +| `from_params` | array of string | 选填 | access_token | 从指定的URL参数中抽取JWT | +| `from_cookies` | array of string | 选填 | - | 从指定的cookie中抽取JWT | +| `clock_skew_seconds` | number | 选填 | 60 | 校验JWT的exp和iat字段时允许的时钟偏移量，单位为秒 | +| `keep_token` | bool | 选填 | ture | 转发给后端时是否保留JWT | + +**注意：** +- 只有当`from_headers`,`from_params`,`from_cookies`均未配置时，才会使用默认值 + +`from_headers` 中每一项的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求| 默认值 | 描述 | +| ---------------- | --------------- | ------- | ------ | --------------------------------------------------------- | +| `name` | string | 必填 | - | 抽取JWT的请求header | +| `value_prefix` | string | 必填 | - | 对请求header的value去除此前缀，剩余部分作为JWT | + +`claims_to_headers` 中每一项的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求| 默认值 | 描述 | +| ---------------- | --------------- | ------- | ------ | --------------------------------------------------------- | +| `claim` | string | 必填 | - | JWT payload中的指定字段，要求必须是字符串或无符号整数类型 | +| `header` | string | 必填 | - | 从payload取出字段的值设置到这个请求头中，转发给后端 | +| `override` | bool | 选填 | true | true时，存在同名请求头会进行覆盖；false时，追加同名请求头 | + + +### 鉴权配置（非必需） + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `allow` | array of string | 选填(**非实例级别配置**) | - | 只能在路由或域名等细粒度规则上配置，对于符合匹配条件的请求，配置允许访问的 consumer，从而实现细粒度的权限控制 | + +## 配置示例 -## 1、基于token的认证 +### 全局配置认证和路由粒度进行鉴权 -### 1.1 简介 +注意如果一个JWT能匹配多个`jwks`，则按照配置顺序命中第一个匹配的`consumer` + +在实例级别做如下插件配置： + +```yaml +global_auth: false +consumers: +- name: consumer1 + issuer: abcd + jwks: | + { + "keys": [ + { + "kty": "oct", + "kid": "123", + "k": "hM0k3AbXBPpKOGg__Ql2Obcq7s60myWDpbHXzgKUQdYo7YCRp0gUqkCnbGSvZ2rGEl4YFkKqIqW7mTHdj-bcqXpNr-NOznEyMpVPOIlqG_NWVC3dydBgcsIZIdD-MR2AQceEaxriPA_VmiUCwfwL2Bhs6_i7eolXoY11EapLQtutz0BV6ZxQQ4dYUmct--7PLNb4BWJyQeWu0QfbIthnvhYllyl2dgeLTEJT58wzFz5HeNMNz8ohY5K0XaKAe5cepryqoXLhA-V-O1OjSG8lCNdKS09OY6O0fkyweKEtuDfien5tHHSsHXoAxYEHPFcSRL4bFPLZ0orTt1_4zpyfew", + "alg": "HS256" + } + ] + } +- name: consumer2 + issuer: abc + jwks: | + { + "keys": [ + { + "kty": "RSA", + "e": "AQAB", + "use": "sig", + "kid": "123", + "alg": "RS256", + "n": "i0B67f1jggT9QJlZ_8QL9QQ56LfurrqDhpuu8BxtVcfxrYmaXaCtqTn7OfCuca7cGHdrJIjq99rz890NmYFZuvhaZ-LMt2iyiSb9LZJAeJmHf7ecguXS_-4x3hvbsrgUDi9tlg7xxbqGYcrco3anmalAFxsbswtu2PAXLtTnUo6aYwZsWA6ksq4FL3-anPNL5oZUgIp3HGyhhLTLdlQcC83jzxbguOim-0OEz-N4fniTYRivK7MlibHKrJfO3xa_6whBS07HW4Ydc37ZN3Rx9Ov3ZyV0idFblU519nUdqp_inXj1eEpynlxH60Ys_aTU2POGZh_25KXGdF_ZC_MSRw" + } + ] + } +``` + +对 route-a 和 route-b 这两个路由做如下配置： + +```yaml +allow: +- consumer1 +``` + +对 *.example.com 和 test.com 在这两个域名做如下配置: + +```yaml +allow: +- consumer2 +``` + +**说明：** + +此例指定的route-a和route-b即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许name为consumer1的调用者访问，其他调用者不允许访问。 + +此例指定的*.example.com和test.com用于匹配请求的域名，当发现域名匹配时，将允许name为consumer2的调用者访问，其他调用者不被允许访问。 + +根据该配置，下列请求可以允许访问： + +假设以下请求会匹配到route-a这条路由 + +**将 JWT 设置在 url 参数中** +```bash +curl 'http://xxx.hello.com/test?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' +``` +**将 JWT 设置在 http 请求头中** +```bash +curl http://xxx.hello.com/test -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' +``` + +认证鉴权通过后，请求的header中会被添加一个`X-Mse-Consumer`字段，在此例中其值为`consumer1`，用以标识调用方的名称 + +下列请求将拒绝访问： + +**请求未提供JWT，返回401** +```bash +curl http://xxx.hello.com/test +``` + +**根据请求提供的JWT匹配到的调用者无访问权限，返回403** +```bash +# consumer1不在*.example.com的allow列表里 +curl 'http://xxx.example.com/test' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' +``` + +#### 网关实例级别开启 + +以下配置将对网关实例级别开启 JWT Auth 认证，所有请求均需要经过认证后才能访问。 + +```yaml +global_auth: true +consumers: +- name: consumer1 + issuer: abcd + jwks: | + { + "keys": [ + { + "kty": "oct", + "kid": "123", + "k": "hM0k3AbXBPpKOGg__Ql2Obcq7s60myWDpbHXzgKUQdYo7YCRp0gUqkCnbGSvZ2rGEl4YFkKqIqW7mTHdj-bcqXpNr-NOznEyMpVPOIlqG_NWVC3dydBgcsIZIdD-MR2AQceEaxriPA_VmiUCwfwL2Bhs6_i7eolXoY11EapLQtutz0BV6ZxQQ4dYUmct--7PLNb4BWJyQeWu0QfbIthnvhYllyl2dgeLTEJT58wzFz5HeNMNz8ohY5K0XaKAe5cepryqoXLhA-V-O1OjSG8lCNdKS09OY6O0fkyweKEtuDfien5tHHSsHXoAxYEHPFcSRL4bFPLZ0orTt1_4zpyfew", + "alg": "HS256" + } + ] + } +- name: consumer2 + issuer: abc + jwks: | + { + "keys": [ + { + "kty": "RSA", + "e": "AQAB", + "use": "sig", + "kid": "123", + "alg": "RS256", + "n": "i0B67f1jggT9QJlZ_8QL9QQ56LfurrqDhpuu8BxtVcfxrYmaXaCtqTn7OfCuca7cGHdrJIjq99rz890NmYFZuvhaZ-LMt2iyiSb9LZJAeJmHf7ecguXS_-4x3hvbsrgUDi9tlg7xxbqGYcrco3anmalAFxsbswtu2PAXLtTnUo6aYwZsWA6ksq4FL3-anPNL5oZUgIp3HGyhhLTLdlQcC83jzxbguOim-0OEz-N4fniTYRivK7MlibHKrJfO3xa_6whBS07HW4Ydc37ZN3Rx9Ov3ZyV0idFblU519nUdqp_inXj1eEpynlxH60Ys_aTU2POGZh_25KXGdF_ZC_MSRw" + } + ] + } +``` + +## 常见错误码说明 + +| HTTP 状态码 | 出错信息 | 原因说明 | +| ----------- | ---------------------- | -------------------------------------------------------------------------------- | +| 401 | Jwt missing | 请求头未提供JWT | +| 401 | Jwt expired | JWT已经过期 | +| 401 | Jwt verification fails | JWT payload校验失败，如iss不匹配 | +| 403 | Access Denied | 无权限访问当前路由 | + +## 详细说明 + +### 1、基于token的认证 + +#### 1.1 简介 很多对外开放的API需要识别请求者的身份，并据此判断所请求的资源是否可以返回给请求者。token就是一种用于身份验证的机制，基于这种机制，应用不需要在服务端保留用户的认证信息或者会话信息，可实现无状态、分布式的Web应用授权，为应用的扩展提供了便利。 -### 1.2 流程描述 +#### 1.2 流程描述 ![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/2336348951/p135822.png) @@ -35,13 +234,13 @@ 在这个整个过程中, 网关利用token认证机制，实现了用户使用自己的用户体系对自己API进行授权的能力。下面我们就要介绍网关实现token认证所使用的结构化令牌Json Web Token(JWT)。 -### 1.3 JWT +#### 1.3 JWT -#### 1.3.1 简介 +##### 1.3.1 简介 Json Web Toke（JWT），是为了在网络应用环境间传递声明而执行的一种基于JSON的开放标准RFC7519。JWT一般可以用作独立的身份验证令牌，可以包含用户标识、用户角色和权限等信息，以便于从资源服务器获取资源，也可以增加一些额外的其它业务逻辑所必须的声明信息，特别适用于分布式站点的登录场景。 -#### 1.3.2 JWT的构成 +##### 1.3.2 JWT的构成 `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ` @@ -119,11 +318,11 @@ var signature = HMACSHA256(encodedString, '$secret'); 将这三部分用 . 连接成一个完整的字符串，就构成了 1.3.2 节最开始的JWT示例。 -#### 1.3.3 时效 +##### 1.3.3 时效 网关会验证token中的exp字段，一旦这个字段过期了，网关会认为这个token无效而将请求直接打回。过期时间这个值必须设置。 -#### 1.3.4 JWT的几个特点 +##### 1.3.4 JWT的几个特点 1. JWT 默认是不加密，不能将秘密数据写入 JWT。 2. JWT 不仅可以用于认证，也可以用于交换信息。有效使用 JWT，可以降低服务器查询数据库的次数。 @@ -131,9 +330,9 @@ var signature = HMACSHA256(encodedString, '$secret'); 4. JWT 本身包含了认证信息，一旦泄露，任何人都可以获得该令牌的所有权限。为了减少盗用，JWT 的有效期应该设置得比较短。对于一些比较重要的权限，使用时应该再次对用户进行认证。 5. 为了减少盗用，JWT 不应该使用 HTTP 协议明码传输，要使用HTTPS 协议传输。 -## 2、用户系统如何应用JWT插件保护API +### 2、用户系统如何应用JWT插件保护API -### 2.1 生成一对JWK（JSON Web 密钥） +#### 2.1 生成一对JWK（JSON Web 密钥） **方法一、在线生成：** @@ -162,7 +361,7 @@ final String publicKeyString = rsaJsonWebKey.toJson(JsonWebKey.OutputControlLeve final String privateKeyString = rsaJsonWebKey.toJson(JsonWebKey.OutputControlLevel.INCLUDE_PRIVATE); ``` -### 2.2 使用JWK中的私钥实现颁发token 的认证服务 +#### 2.2 使用JWK中的私钥实现颁发token 的认证服务 需要使用2.1节中在线生成的 Keypair JSON字符串（三个方框内的第一个）或者本地生成的 privateKeyString JSON字符串作为私钥来颁发token，用于授权可信的用户访问受保护的API，具体实现可以参考下方示例。 向客户颁发token的形式由用户根据具体的业务场景决定，可以将颁发token的功能部署到生产环境，配置成普通API后由访问者通过用户名密码获得，也可以直接在本地环境生成token 后，直接拷贝给指定用户使用。 @@ -216,187 +415,3 @@ public class GenerateJwtDemo { } } ``` - -# 插件配置说明 - -## 配置字段 - -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | -| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | -| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表，用于对请求进行鉴权 | - -`consumers`中每一项的配置字段说明如下： - -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ----------------------- | ----------------- | -------- | ------------------------------------------------- | ------------------------ | -| `name` | string | 必填 | - | 配置该consumer的名称 | -| `jwks` | string | 必填 | - | https://www.rfc-editor.org/rfc/rfc7517 指定的json格式字符串，是由验证JWT中签名的公钥（或对称密钥）组成的Json Web Key Set | -| `issuer` | string | 必填 | - | JWT的签发者，需要和payload中的iss字段保持一致 | -| `claims_to_headers` | array of object | 选填 | - | 抽取JWT的payload中指定字段，设置到指定的请求头中转发给后端 | -| `from_headers` | array of object | 选填 | {"name":"Authorization","value_prefix":"Bearer "} | 从指定的请求头中抽取JWT | -| `from_params` | array of string | 选填 | access_token | 从指定的URL参数中抽取JWT | -| `from_cookies` | array of string | 选填 | - | 从指定的cookie中抽取JWT | -| `clock_skew_seconds` | number | 选填 | 60 | 校验JWT的exp和iat字段时允许的时钟偏移量，单位为秒 | -| `keep_token` | bool | 选填 | ture | 转发给后端时是否保留JWT | - -**注意：** -- 只有当`from_headers`,`from_params`,`from_cookies`均未配置时，才会使用默认值 - -`from_headers` 中每一项的配置字段说明如下： - -| 名称 | 数据类型 | 填写要求| 默认值 | 描述 | -| ---------------- | --------------- | ------- | ------ | --------------------------------------------------------- | -| `name` | string | 必填 | - | 抽取JWT的请求header | -| `value_prefix` | string | 必填 | - | 对请求header的value去除此前缀，剩余部分作为JWT | - -`claims_to_headers` 中每一项的配置字段说明如下： - -| 名称 | 数据类型 | 填写要求| 默认值 | 描述 | -| ---------------- | --------------- | ------- | ------ | --------------------------------------------------------- | -| `claim` | string | 必填 | - | JWT payload中的指定字段，要求必须是字符串或无符号整数类型 | -| `header` | string | 必填 | - | 从payload取出字段的值设置到这个请求头中，转发给后端 | -| `override` | bool | 选填 | true | true时，存在同名请求头会进行覆盖；false时，追加同名请求头 | - - -`_rules_` 中每一项的配置字段说明如下： - -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的路由名称 | -| `_match_domain_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的域名 | -| `allow` | array of string | 必填 | - | 对于符合匹配条件的请求，配置允许访问的consumer名称 | - -**注意：** -- 若不配置`_rules_`字段，则默认对当前网关实例的所有路由开启认证； -- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 - -## 配置示例 - -### 对特定路由或域名开启 - -以下配置将对网关特定路由或域名开启 Jwt Auth 认证和鉴权，注意如果一个JWT能匹配多个`jwks`，则按照配置顺序命中第一个匹配的`consumer` - -```yaml -consumers: -- name: consumer1 - issuer: abcd - jwks: | - { - "keys": [ - { - "kty": "oct", - "kid": "123", - "k": "hM0k3AbXBPpKOGg__Ql2Obcq7s60myWDpbHXzgKUQdYo7YCRp0gUqkCnbGSvZ2rGEl4YFkKqIqW7mTHdj-bcqXpNr-NOznEyMpVPOIlqG_NWVC3dydBgcsIZIdD-MR2AQceEaxriPA_VmiUCwfwL2Bhs6_i7eolXoY11EapLQtutz0BV6ZxQQ4dYUmct--7PLNb4BWJyQeWu0QfbIthnvhYllyl2dgeLTEJT58wzFz5HeNMNz8ohY5K0XaKAe5cepryqoXLhA-V-O1OjSG8lCNdKS09OY6O0fkyweKEtuDfien5tHHSsHXoAxYEHPFcSRL4bFPLZ0orTt1_4zpyfew", - "alg": "HS256" - } - ] - } -- name: consumer2 - issuer: abc - jwks: | - { - "keys": [ - { - "kty": "RSA", - "e": "AQAB", - "use": "sig", - "kid": "123", - "alg": "RS256", - "n": "i0B67f1jggT9QJlZ_8QL9QQ56LfurrqDhpuu8BxtVcfxrYmaXaCtqTn7OfCuca7cGHdrJIjq99rz890NmYFZuvhaZ-LMt2iyiSb9LZJAeJmHf7ecguXS_-4x3hvbsrgUDi9tlg7xxbqGYcrco3anmalAFxsbswtu2PAXLtTnUo6aYwZsWA6ksq4FL3-anPNL5oZUgIp3HGyhhLTLdlQcC83jzxbguOim-0OEz-N4fniTYRivK7MlibHKrJfO3xa_6whBS07HW4Ydc37ZN3Rx9Ov3ZyV0idFblU519nUdqp_inXj1eEpynlxH60Ys_aTU2POGZh_25KXGdF_ZC_MSRw" - } - ] - } -# 使用 _rules_ 字段进行细粒度规则配置 -_rules_: -# 规则一：按路由名称匹配生效 -- _match_route_: - - route-a - - route-b - allow: - - consumer1 -# 规则二：按域名匹配生效 -- _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer2 -``` - -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许`name`为`consumer1`的调用者访问，其他调用者不允许访问； - -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将允许`name`为`consumer2`的调用者访问，其他调用者不允许访问。 - -#### 根据该配置，下列请求可以允许访问： - -假设以下请求会匹配到route-a这条路由 - -**将 JWT 设置在 url 参数中** -```bash -curl 'http://xxx.hello.com/test?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' -``` -**将 JWT 设置在 http 请求头中** -```bash -curl http://xxx.hello.com/test -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' -``` - -认证鉴权通过后，请求的header中会被添加一个`X-Mse-Consumer`字段，在此例中其值为`consumer1`，用以标识调用方的名称 - -#### 下列请求将拒绝访问： - -**请求未提供JWT，返回401** -```bash -curl http://xxx.hello.com/test -``` - -**根据请求提供的JWT匹配到的调用者无访问权限，返回403** -```bash -# consumer1不在*.example.com的allow列表里 -curl 'http://xxx.example.com/test' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' -``` - -### 网关实例级别开启 - -以下配置未指定`_rules_`字段，因此将对网关实例级别开启 JWT Auth 认证 - -```yaml -consumers: -- name: consumer1 - issuer: abcd - jwks: | - { - "keys": [ - { - "kty": "oct", - "kid": "123", - "k": "hM0k3AbXBPpKOGg__Ql2Obcq7s60myWDpbHXzgKUQdYo7YCRp0gUqkCnbGSvZ2rGEl4YFkKqIqW7mTHdj-bcqXpNr-NOznEyMpVPOIlqG_NWVC3dydBgcsIZIdD-MR2AQceEaxriPA_VmiUCwfwL2Bhs6_i7eolXoY11EapLQtutz0BV6ZxQQ4dYUmct--7PLNb4BWJyQeWu0QfbIthnvhYllyl2dgeLTEJT58wzFz5HeNMNz8ohY5K0XaKAe5cepryqoXLhA-V-O1OjSG8lCNdKS09OY6O0fkyweKEtuDfien5tHHSsHXoAxYEHPFcSRL4bFPLZ0orTt1_4zpyfew", - "alg": "HS256" - } - ] - } -- name: consumer2 - issuer: abc - jwks: | - { - "keys": [ - { - "kty": "RSA", - "e": "AQAB", - "use": "sig", - "kid": "123", - "alg": "RS256", - "n": "i0B67f1jggT9QJlZ_8QL9QQ56LfurrqDhpuu8BxtVcfxrYmaXaCtqTn7OfCuca7cGHdrJIjq99rz890NmYFZuvhaZ-LMt2iyiSb9LZJAeJmHf7ecguXS_-4x3hvbsrgUDi9tlg7xxbqGYcrco3anmalAFxsbswtu2PAXLtTnUo6aYwZsWA6ksq4FL3-anPNL5oZUgIp3HGyhhLTLdlQcC83jzxbguOim-0OEz-N4fniTYRivK7MlibHKrJfO3xa_6whBS07HW4Ydc37ZN3Rx9Ov3ZyV0idFblU519nUdqp_inXj1eEpynlxH60Ys_aTU2POGZh_25KXGdF_ZC_MSRw" - } - ] - } -``` - -# 常见错误码说明 - -| HTTP 状态码 | 出错信息 | 原因说明 | -| ----------- | ---------------------- | -------------------------------------------------------------------------------- | -| 401 | Jwt missing | 请求头未提供JWT | -| 401 | Jwt expired | JWT已经过期 | -| 401 | Jwt verification fails | JWT payload校验失败，如iss不匹配 | -| 403 | Access Denied | 无权限访问当前路由 | - diff --git a/plugins/wasm-cpp/extensions/jwt_auth/README_EN.md b/plugins/wasm-cpp/extensions/jwt_auth/README_EN.md index 25b1e7940e..4c10186d4a 100644 --- a/plugins/wasm-cpp/extensions/jwt_auth/README_EN.md +++ b/plugins/wasm-cpp/extensions/jwt_auth/README_EN.md @@ -1,280 +1,69 @@ -# Description -The `jwt-auth` plugin implements authentication and authorization based on JWT (JSON Web Token), supports parsing JWTs from URL parameters, request headers, and Cookie fields from HTTP requests, and verifies whether the token has permission to access. +--- +title: JWT Authentication +keywords: [higress,jwt auth] +description: JWT Authentication plugin configuration reference +--- -The difference between this plugin and the JWT authentication in `Security Capabilities->Authentication and Authorization` is that it provides additional capabilities for identifying the caller's identity, supporting the configuration of different JWT credentials for different callers. +## Function Description +`jwt-auth` plugin implements authentication and authorization based on JWT (JSON Web Tokens). It supports extracting JWT from HTTP request URL parameters, request headers, and Cookie fields, while verifying whether the Token has the necessary permissions to access the resource. -# Detailed Description +The difference between this plugin and the JWT authentication in `Security Capability -> Authentication and Authorization` is that it additionally provides the capability of identifying the caller's identity, supporting different JWT credentials for different callers. -## 1. Token-based authentication - -### 1.1 Introduction -Many open APIs need to identify the identity of the caller and determine whether the requested resource can be returned to the caller based on this identity. Token is a mechanism used for identity verification. Based on this mechanism, the application does not need to retain the user's authentication information or session information on the server, which can realize stateless and distributed web application authorization and provide convenience for application extension. - -### 1.2 Process Description - -![](process.png) - -The above figure is the business process sequence diagram when the gateway uses JWT for authentication, and the following we will describe the steps marked in the figure in detail in words: - -1. The client initiates an authentication request to the API gateway, usually carrying the end user's username and password in the request; - -2. The gateway forwards the request directly to the backend service; - -3. The backend service reads the authentication information (such as the username and password) in the request for verification. After the verification is passed, it uses the private key to generate a standard token and returns it to the gateway; - -4. The gateway returns the response with the token to the client, and the client needs to cache this token locally; - -5. The client sends a business request to the API gateway, carrying the token in the request; - -6. The gateway uses the public key set by the user to verify the token in the request. After the verification is passed, the request is passed through to the backend service; - -7. The backend service processes the business and responds; - -8. The gateway returns the business response to the client. - -In this entire process, the gateway uses the token authentication mechanism to realize the ability of users to use their own user system to authorize their API. Next, we will introduce the structured token JSON Web Token (JWT) used by the gateway to implement token authentication. - -### 1.3 JWT - -#### 1.3.1 Introduction - -JSON Web Token (JWT) is an open standard RFC7519 based on JSON for executing a type of claim to pass between network applications. JWT can generally be used as an independent identity verification token, which can contain user identification, user roles, and permissions information, making it easier to obtain resources from the resource server, and can also add some other necessary declarative information for other business logic, especially suitable for the login scenario of distributed sites. - -#### 1.3.2 Composition of JWT - -`eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ` - -As shown in the example above, JWT is a string consisting of three parts: - -- Header -- Payload -- Signature - -**Header** - -The header of the JWT carries two pieces of information: - -- The type of the token, which is JWT -- The algorithm used for encryption - -The gateway supports the following encryption algorithms: - -```text -ES256, ES384, ES512, -HS256, HS384, HS512, -RS256, RS384, RS512, -PS256, PS384, PS512, -EdDSA -``` - -The complete header looks like the following JSON: - -```js -{ - 'typ': 'JWT', - 'alg': 'HS256' -} -``` - -The header is then Base64-encoded (this encoding can be symmetrically decoded), forming the first part: - -`eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9` - -**Payload** - -The payload is where the actual information is stored. The details are defined below: - -```text -iss: The issuer of the token. This indicates who created the token and is a string value. -sub: The subject identifier. This is the unique identifier for the end user provided by the issuer, and is no longer than 255 ASCII characters, and is case-sensitive within the issuer's scope. -aud: The audience(s) of the token, which is an array of case-sensitive strings. -exp: The expiration time of the token, after which the token will be invalidated, is an integer declaration representing the Unix timestamp in seconds. -iat: The issuance time of the token, is an integer declaration representing the Unix timestamp in seconds. -jti: The unique identifier of the token, and the value is unique for every token created by the issuer. It is usually a cryptographically random value to prevent conflicts. This component adds a random entropy that an attacker cannot obtain to the structured token, making it more difficult for the token to be guessed or replayed. -``` - -Custom fields for a user feature can also be added, such as the example below adding a "name" field for the user's nickname: - -```js -{ - "sub": "1234567890", - "name": "John Doe" -} -``` - -The payload is then Base64-encoded, forming the second part of the JWT: - -`JTdCJTBBJTIwJTIwJTIyc3ViJTIyJTNBJTIwJTIyMTIzNDU2Nzg5MCUyMiUyQyUwQSUyMCUyMCUyMm5hbWUlMjIlM0ElMjAlMjJKb2huJTIwRG9lJTIyJTBBJTdE` - -**Signature** - -This part is a string that consists of the Base64-encoded header and Base64-encoded payload concatenated with a period, followed by the encryption of the resulting string using the algorithm specified in the header (where $secret represents the user's private key). - -```js -var encodedString = base64UrlEncode(header) + '.' + base64UrlEncode(payload); -var signature = HMACSHA256(encodedString, '$secret'); -``` - -These three parts are then concatenated using periods to form the complete JWT string as shown in the example at the beginning of this section. - -#### 1.3.3 Time validity - -The gateway will verify the exp field in the token. Once this field has expired, the gateway will consider the token invalid and reject the request directly. The expiration time value must be set. - -#### 1.3.4 Several Characteristics of JWT - -1. By default, JWT is not encrypted and cannot write secret data into JWT. -2. JWT can not only be used for authentication but also for exchanging information. Using JWT effectively can reduce the number of times servers query the database. -3. The biggest drawback of JWT is that the server cannot invalidate a token during use, or change the token's permission, because the server does not keep the session state. That is, once JWT is issued, it will always be valid before it expires, unless the server deploys additional logic. -4. JWT contains authentication information itself. Once leaked, anyone can obtain all permissions of the token. To reduce theft, the expiration time of JWT should be set relatively short. For some more important permissions, users should be authenticated again when using them. -5. To reduce theft, JWT should not be transmitted in plaintext using the HTTP protocol, and the HTTPS protocol should be used for transmission. - -## 2. How to apply the JWT plugin to protect the API of the user system - -### 2.1 Generate a pair of JWK (JSON Web Key) - -**Method 1: Generate online** - -Users can generate the private and public keys used for token generation and verification on this website https://mkjwk.org. The private key is used for the authorization service to issue JWT, and the public key is configured into the JWT plugin for the gateway to verify the signature of the request. Note that the JWKs format configuration used by the gateway requires the public key in the figure below to be placed in the keys structure, such as: `{"keys":[{"kty":"RSA","e":"AQAB",...}]}` - - - -**Method 2: Generate locally** - -This article uses a Java example to illustrate, and users of other languages can also find relevant tools to generate key pairs. Create a new Maven project and add the following dependencies: - -```xml - - org.bitbucket.b_c - jose4j - 0.7.0 - -``` - -Use the following code to generate a pair of RSA keys: - -```java -RsaJsonWebKey rsaJsonWebKey = RsaJwkGenerator.generateJwk(2048); -final String publicKeyString = rsaJsonWebKey.toJson(JsonWebKey.OutputControlLevel.PUBLIC_ONLY); -final String privateKeyString = rsaJsonWebKey.toJson(JsonWebKey.OutputControlLevel.INCLUDE_PRIVATE); -``` - -### 2.2 Implement the token issuance authentication service using the private key in JWK - -The Keypair JSON string generated online in Section 2.1 (the first of the three boxes) or privateKeyString JSON string generated locally needs to be used as the private key to issue tokens for trusted users to access protected APIs. The specific implementation can refer to the following example. The form of issuing tokens to customers is determined by the user according to the specific business scenario. The function of issuing tokens can be deployed in the production environment and configured as a normal API for visitors to obtain through username and password, or tokens can be generated directly in the local environment and copied to designated users for use. - -```java -import java.security.PrivateKey; -import org.jose4j.json.JsonUtil; -import org.jose4j.jwk.RsaJsonWebKey; -import org.jose4j.jwk.RsaJwkGenerator; -import org.jose4j.jws.AlgorithmIdentifiers; -import org.jose4j.jws.JsonWebSignature; -import org.jose4j.jwt.JwtClaims; -import org.jose4j.jwt.NumericDate; -import org.jose4j.lang.JoseException; -public class GenerateJwtDemo { - public static void main(String[] args) throws JoseException { - String keyId = "uniq_key"; - //Use the Keypair generated in section 2.1 of this article - String privateKeyJson = "{\n" - + " \"kty\": \"RSA\",\n" - + " \"d\": " - + - "\"O9MJSOgcjjiVMNJ4jmBAh0mRHF_TlaVva70Imghtlgwxl8BLfcf1S8ueN1PD7xV6Cnq8YenSKsfiNOhC6yZ_fjW1syn5raWfj68eR7cjHWjLOvKjwVY33GBPNOvspNhVAFzeqfWneRTBbga53Agb6jjN0SUcZdJgnelzz5JNdOGaLzhacjH6YPJKpbuzCQYPkWtoZHDqWTzCSb4mJ3n0NRTsWy7Pm8LwG_Fd3pACl7JIY38IanPQDLoighFfo-Lriv5z3IdlhwbPnx0tk9sBwQBTRdZ8JkqqYkxUiB06phwr7mAnKEpQJ6HvhZBQ1cCnYZ_nIlrX9-I7qomrlE1UoQ\",\n" - + " \"e\": \"AQAB\",\n" - + " \"alg\": \"RS256\",\n" - + " \"n\": \"vCuB8MgwPZfziMSytEbBoOEwxsG7XI3MaVMoocziP4SjzU4IuWuE_DodbOHQwb_thUru57_Efe" - + - "--sfATHEa0Odv5ny3QbByqsvjyeHk6ZE4mSAV9BsHYa6GWAgEZtnDceeeDc0y76utXK2XHhC1Pysi2KG8KAzqDa099Yh7s31AyoueoMnrYTmWfEyDsQL_OAIiwgXakkS5U8QyXmWicCwXntDzkIMh8MjfPskesyli0XQD1AmCXVV3h2Opm1Amx0ggSOOiINUR5YRD6mKo49_cN-nrJWjtwSouqDdxHYP-4c7epuTcdS6kQHiQERBd1ejdpAxV4c0t0FHF7MOy9kw\"\n" - + "}"; - JwtClaims claims = new JwtClaims(); - claims.setGeneratedJwtId(); - claims.setIssuedAtToNow(); - //Expiration time must be set - NumericDate date = NumericDate.now(); - date.addSeconds(120*60); - claims.setExpirationTime(date); - claims.setNotBeforeMinutesInThePast(1); - claims.setSubject("YOUR_SUBJECT"); - claims.setAudience("YOUR_AUDIENCE"); - //Add custom parameters, use String type for all values - claims.setClaim("userId", "1213234"); - claims.setClaim("email", "userEmail@youapp.com"); - JsonWebSignature jws = new JsonWebSignature(); - jws.setAlgorithmHeaderValue(AlgorithmIdentifiers.RSA_USING_SHA256); - jws.setKeyIdHeaderValue(keyId); - jws.setPayload(claims.toJson()); - PrivateKey privateKey = new RsaJsonWebKey(JsonUtil.parseJson(privateKeyJson)).getPrivateKey(); - - jws.setKey(privateKey); - String jwtResult = jws.getCompactSerialization(); - System.out.println("Generate Json Web token , result is " + jwtResult); - } -} -``` - -# Plugin Configuration Guide +## Runtime Properties +Plugin execution phase: `Authentication Phase` +Plugin execution priority: `340` ## Configuration Fields - -| Name | Data Type | Required | Default | Description | -| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | -| `consumers` | array of object | Yes | - | Configures callers of the service for authenticating requests | -| `_rules_` | array of object | Optional | - | Configures access control lists for specific routes or domains for authorizing requests | - -The configuration field descriptions for each item in consumers are as follows: - -| Name | Data Type | Required | Default | Description| -| ----------------------- | ----------------- | -------- | ------------------------------------------------- | ------------------------ | -| `name` | string | Yes | - | Configures the name of this consumer | -| `jwks` | string | Yes | - | Specifies a JSON Web Key Set, as defined in https://www.rfc-editor.org/rfc/rfc7517, consisting of public keys (or symmetric keys) used to verify the signature of JWT | -| `issuer` | string | Yes | - | The issuer of the JWT, which should be consistent with the iss field in the payload | -| `claims_to_headers` | array of object | Optional | - | Extracts the specified fields from the JWT's payload and sets them to the specified request headers for forwarding to the backend | -| `from_headers` | array of object | Optional | {"name":"Authorization","value_prefix":"Bearer "} | Extracts the JWT from the specified request headers | -| `from_params` | array of string | Optional | access_token | Extracts the JWT from the specified URL parameters | -| `from_cookies` | array of string | Optional | - | Extracts the JWT from the specified cookie(s) | -| `clock_skew_seconds` | number | Optional | 60 | The amount of clock skew, in seconds, that is allowed when verifying the exp and iat fields of the JWT | -| `keep_token` | bool | Optional | ture | Whether to keep the JWT when forwarding it to the backend | - **Note:** - -- The default value is used only if neither `from_headers`, `from_params`, nor `from_cookies` are configured. - -The configuration field descriptions for each item in `from_headers` are as follows: - -| Name | Data Type | Required| Default | Description | -| ---------------- | --------------- | ------- | ------ | --------------------------------------------------------- | -| `name` | string | Yes | - | Specifies the request header to extract the JWT from | -| `value_prefix` | string | Yes | - | Removes the specified prefix from the request header's value, leaving the rest as the JWT | - -The configuration field descriptions for each item in `claims_to_headers` are as follows: - -| Name | Data Type | Required| Default | Description | -| ---------------- | --------------- | ------- | ------ | --------------------------------------------------------- | -| `claim` | string | Yes | - | The name of the field in the JWT payload, which must be a string or unsigned integer | -| `header` | string | Yes | - | Sets the value of the specified request header to the value of the specified field in the payload, for forwarding to the backend | -| `override` | bool | Optional | true | If true, overrides an existing header with the same name; if false, appends the header to the existing headers | - -The configuration field descriptions for each item in `_rules_` are as follows: - -| Name | Data Type | Required| Default | Description | -| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | Optional, select either `_match_route_` or `_match_domain_` | - | Configures the route names to match| -| `_match_domain_` | array of string | Optional, select either `_match_route_` or `_match_domain_` | - | Configures the domains to match | -| `allow` | array of string | Required | - | Configures the consumer names allowed to access the matched requests | +- In one rule, authentication configuration and authorization configuration cannot coexist. +- For requests authenticated through authentication and authorization, the request header will be augmented with an `X-Mse-Consumer` field to identify the caller's name. + +### Authentication Configuration +| Name | Data Type | Requirements | Default Value | Description | +| ----------- | --------------- | ---------------------------------------------- | ------------- | ----------------------------------------------------------- | +| `global_auth` | bool | Optional (**instance-level configuration only**) | - | Can only be configured at the instance level. If set to true, it will globally enable the authentication mechanism; if set to false, it will only apply to the configured domain names and routes. If not configured, it will only globally take effect when no domain names and routes are configured (to be compatible with old user habits). | +| `consumers` | array of object | Required | - | Configure service consumers for request authentication | + +The configuration fields for each item in `consumers` are as follows: +| Name | Data Type | Requirements | Default Value | Description | +| ----------------------- | ------------------ | ------------ | -------------------------------------------------- | ------------------------------- | +| `name` | string | Required | - | The name of the consumer | +| `jwks` | string | Required | - | JSON format string specified by https://www.rfc-editor.org/rfc/rfc7517, consisting of the public key (or symmetric key) used to verify the JWT signature. | +| `issuer` | string | Required | - | The issuer of the JWT, must match the `iss` field in the payload. | +| `claims_to_headers` | array of object | Optional | - | Extract the specified fields from the JWT payload and set them in the specified request headers to forward to the backend. | +| `from_headers` | array of object | Optional | {"name":"Authorization","value_prefix":"Bearer "} | Extract JWT from the specified request headers. | +| `from_params` | array of string | Optional | access_token | Extract JWT from the specified URL parameters. | +| `from_cookies` | array of string | Optional | - | Extract JWT from the specified cookies. | +| `clock_skew_seconds` | number | Optional | 60 | The allowed clock skew when validating the `exp` and `iat` fields of the JWT, measured in seconds. | +| `keep_token` | bool | Optional | true | Whether to retain the JWT when forwarding to the backend. | **Note:** -- If the `_rules_` field is not configured, authentication and authorization are enabled for all routes of the current gateway instance by default; -- For authenticated and authorized requests, a `X-Mse-Consumer` field is added to the request header to identify the caller's name. - -## Configuration Example - -### Enable for Specific Routes or Domains - -The following configuration enables Jwt Auth authentication and authorization for specific routes or domains of the gateway. If a JWT can match multiple `jwks`, the first matching `consumer` is hit according to the configuration order. - +- The default values will only be used when `from_headers`, `from_params`, and `from_cookies` are not all configured. +The configuration fields for each item in `from_headers` are as follows: +| Name | Data Type | Requirements | Default Value | Description | +| --------------- | ---------------- | ------------ | ------------- | ----------------------------------------------- | +| `name` | string | Required | - | Extract JWT from the request header. | +| `value_prefix` | string | Required | - | Remove the prefix from the request header value, with the remaining part serving as the JWT. | + +The configuration fields for each item in `claims_to_headers` are as follows: +| Name | Data Type | Requirements | Default Value | Description | +| --------------- | ---------------- | ------------ | ------------- | --------------------------------------------- | +| `claim` | string | Required | - | The specified field in the JWT payload, must be a string or unsigned integer type. | +| `header` | string | Required | - | The value of the field extracted from the payload is set to this request header and forwarded to the backend. | +| `override` | bool | Optional | true | If true, existing headers with the same name will be overridden; if false, they will be appended. | + +### Authorization Configuration (Optional) +| Name | Data Type | Requirements | Default Value | Description | +| ----------- | --------------- | ---------------------------------------------- | ------------- | ----------------------------------------------------------------------------------------------------- | +| `allow` | array of string | Optional (**not instance-level configuration**) | - | Can only be configured on fine-grained rules such as routes or domain names, allowing specified consumers to access matching requests for fine-grained permission control. | + +## Configuration Examples +### Global Configuration for Authentication and Route-Level Authorization +Note: If a JWT matches multiple `jwks`, the first matching consumer will be applied according to the configuration order. + +Configure the plugin at the instance level as follows: ```yaml +global_auth: false consumers: - name: consumer1 issuer: abcd @@ -304,60 +93,56 @@ consumers: } ] } -# Use the _rules_ field for fine-grained rule configuration -_rules_: -# Rule 1: Effective when matched by route name -- _match_route_: - - route-a - - route-b - allow: - - consumer1 -# Rule 2: Effective when matched by domain name -- _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer2 ``` -In this example, the `route-a` and `route-b` specified in `_match_route_` are the names of the routes filled in when creating the gateway route. When these two routes are matched, access will be allowed for the caller with the `name` of `consumer1`, and other callers will not be allowed to access. +Configure the following for routes `route-a` and `route-b`: +```yaml +allow: +- consumer1 +``` + +Configure the following for domain names `*.example.com` and `test.com`: +```yaml +allow: +- consumer2 +``` -The `*.example.com` and `test.com` specified in `_match_domain_` are used to match the domain names of the requests. When a domain name match is found, access will be allowed for the caller with the `name` of `consumer2`, and other callers will not be allowed to access. +**Explanation:** +The specified `route-a` and `route-b` refer to the routing names filled in when creating the gateway route. When these two routes are matched, the caller with the name consumer1 will be allowed access, while others will not be permitted. -#### According to this configuration, the following requests are allowed: +The specified `*.example.com` and `test.com` are used to match the request domain names. When a matching domain name is found, the caller with the name consumer2 will be allowed access, while others will not. -Assuming the following requests will match the route-a route: +Based on this configuration, the following requests will be allowed: -**JWT is set in URL parameter** +Suppose the following request matches the route-a. +**Setting the JWT in URL parameters** ```bash curl 'http://xxx.hello.com/test?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' ``` -**JWT is set in HTTP request header** +**Setting the JWT in HTTP request headers** ```bash curl http://xxx.hello.com/test -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' ``` -After authentication and authorization, a `X-Mse-Consumer` field will be added in the request header with a value of `consumer1` in this example, to identify the name of the caller. +After successful authentication and authorization, the request's header will include an `X-Mse-Consumer` field, in this example with the value `consumer1`, to identify the caller's name. -#### The following requests will be denied: - -**Request without JWT provided, returns 401** +The following requests will be denied: +**Request without JWT returns 401** ```bash curl http://xxx.hello.com/test ``` -**The consumer matched by the provided JWT in the request does not have access, returns 403** +**Caller matching from the provided JWT has no access permission, returning 403** ```bash -# consumer1 is not in the allow list of *.example.com +# consumer1 is not in the allow list for *.example.com curl 'http://xxx.example.com/test' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4' ``` -### Enabling at Gateway Instance Level - -The following configuration does not specify the `_rules_` field, so JWT authentication will be enabled at the gateway instance level: - +#### Enable at Gateway Instance Level +The following configuration will enable JWT Auth authentication at the instance level, requiring all requests to be authenticated before accessing. ```yaml +global_auth: true consumers: - name: consumer1 issuer: abcd @@ -389,11 +174,174 @@ consumers: } ``` -# Common Error Codes +## Common Error Codes +| HTTP Status Code | Error Message | Reason Description | +| ---------------- | ----------------------------- | ---------------------------------------------------------------------------- | +| 401 | Jwt missing | The request header did not provide a JWT | +| 401 | Jwt expired | The JWT has expired | +| 401 | Jwt verification fails | JWT payload verification failed, such as mismatched `iss` | +| 403 | Access Denied | No permission to access the current route | + +## Detailed Description +### 1. Token-based Authentication +#### 1.1 Introduction +Many open APIs need to identify the requester's identity and determine whether the requested resource can be returned. A token is a mechanism used for identity verification. With this mechanism, applications do not need to retain user authentication information or session information on the server, allowing for stateless, distributed web application authorization, facilitating application scaling. + +#### 1.2 Process Description +![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/2336348951/p135822.png) +The above image shows the entire business process sequence diagram for gateway authentication using JWT. Below, we will detail the steps indicated in the diagram: + +1. The client initiates an authentication request to the API gateway, generally carrying the terminal user's username and password. +2. The gateway forwards the request directly to the backend service. +3. The backend service reads the verification information in the request (such as username and password) for validation. Upon successful verification, it generates a standard token using a private key and returns it to the gateway. +4. The gateway returns a response containing the token to the client, who must cache this token locally. +5. The client sends a business request to the API gateway, carrying the token in the request. +6. The gateway validates the token using the user's set public key. Upon successful validation, it forwards the request to the backend service. +7. The backend service processes the business and responds. +8. The gateway returns the business response to the client. + +Throughout this process, the gateway utilizes the token authentication mechanism, enabling the user to authorize their API using their user system. Next, we will introduce the structured token used by the gateway for token authentication: JSON Web Token (JWT). + +#### 1.3 JWT +##### 1.3.1 Introduction +JSON Web Token (JWT) is an open standard (RFC7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. JWT can be used as a stand-alone authentication token, capable of containing user identity, user roles, permissions, and other information, aiding in resource retrieval from resource servers and adding any additional claims required by business logic, particularly suitable for login scenarios for distributed sites. + +##### 1.3.2 JWT Structure +`eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ` +As shown above, JWT is a string composed of three parts: +- Header +- Payload +- Signature + +**Header** +The header of the JWT carries two pieces of information: +- Token type, which is JWT +- Signing algorithm -| HTTP Status Code | Error Message | Reason Description| -|------------------| ---------------------- | -------------------------------------------------------------------------------- | -| 401 | JWT missing | The JWT is not provided in the request header. | -| 401 | JWT expired | The JWT has expired. | -| 401 | JWT verification fails | The JWT payload verification failed, such as the iss mismatch. | -| 403 | Access denied | Access to the current route is denied. | +The supported signing algorithms by the gateway are as follows: +```text +ES256, ES384, ES512, +HS256, HS384, HS512, +RS256, RS384, RS512, +PS256, PS384, PS512, +EdDSA +``` + +The complete header looks like the following JSON: +```js +{ + 'typ': 'JWT', + 'alg': 'HS256' +} +``` +Then the header is Base64 encoded (this encoding is symmetrically decodable), forming the first part. +`eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9` + +**Payload** +The payload is where valid information is stored. Its details are defined as follows: +```text +iss: Token issuer. Indicates who created the token, this claim is a string. +sub: Subject Identifier, an identifier provided by the issuer for its end users, unique within the issuer's scope, up to 255 ASCII characters, case sensitive. +aud: Audience(s), the audience of the token, an array of strings that are case-sensitive. +exp: Expiration time, a timestamp of the token's expiration. Tokens expired beyond this time will be invalid. This claim is an integer, representing the number of seconds since January 1, 1970. +iat: Issuance time of the token, this claim is an integer, representing the number of seconds since January 1, 1970. +jti: Unique identifier for the token, the value of this claim must be unique for each token created by the token issuer to prevent collisions. It is typically a cryptographically random value. This value adds a random entropy component to the structured token that is not accessible to an attacker, helping prevent token guess and replay attacks. +``` +Custom fields necessary for the user system can also be added, for example, the following example adds a nickname `name`: +```js +{ + "sub": "1234567890", + "name": "John Doe" +} +``` +Then encode it in Base64 to obtain the second part of the JWT: +`JTdCJTBBJTIwJTIwJTIyc3ViJTIyJTNBJTIwJTIyMTIzNDU2Nzg5MCUyMiUyQyUwQSUyMCUyMCUyMm5hbWUlMjIlM0ElMjAlMjJKb2huJTIwRG9lJTIyJTBBJTdE` + +**Signature** +This part requires the Base64-encoded Header and Base64-encoded Payload to be connected by a period, and then encrypted using the signing method declared in the Header (where `$secret` represents the user's private key) to form the third part of the JWT. +```js +var encodedString = base64UrlEncode(header) + '.' + base64UrlEncode(payload); +var signature = HMACSHA256(encodedString, '$secret'); +``` +Connecting these three parts with a period creates a complete string that forms the JWT example at the beginning of section 1.3.2. + +##### 1.3.3 Validity Period +The gateway will validate the `exp` field in the token. Once this field expires, the gateway will consider this token invalid and directly reject the request. The expiration time must be set. + +##### 1.3.4 Characteristics of JWT +1. JWT is not encrypted by default; do not write secret data into JWT. +2. JWT can be used for both authentication and information exchange. Effectively using JWT can reduce the number of queries to the database on the server. +3. The biggest drawback of JWT is that since the server does not maintain session state, it cannot revoke a token during use or change the permissions of said token. In other words, once a JWT is issued, it will remain valid until expiration, unless the server implements additional logic. +4. JWT itself contains authentication information, and once leaked, anyone can gain all permissions of that token. To minimize theft, the validity period of JWT should be set to be relatively short. For some critical permissions, users should be re-authenticated. +5. To reduce theft, JWT should not be transmitted in plain text over HTTP but should use HTTPS for transmission. + +### 2. How User Systems Apply the JWT Plugin to Protect APIs +#### 2.1 Generating a Pair of JWKs (JSON Web Keys) +**Method 1: Online Generation:** +Users can generate the private and public keys used for token generation and verification at this site https://mkjwk.org. The private key is used by the authorization service to issue JWTs, and the public key is configured into the JWT plugin for the gateway to verify requests. Pay attention to the jwks format configuration used by the gateway. In the image below, the Public Key should be placed into the keys structure, for example: `{"keys":[{"kty":"RSA","e":"AQAB",...}]}` + + +**Method 2: Local Generation:** +This article demonstrates using Java; users of other languages can find related tools to generate key pairs. Create a new Maven project and include the following dependency: +```xml + + org.bitbucket.b_c + jose4j + 0.7.0 + +``` +Use the following code to generate a pair of RSA keys: +```java +RsaJsonWebKey rsaJsonWebKey = RsaJwkGenerator.generateJwk(2048); +final String publicKeyString = rsaJsonWebKey.toJson(JsonWebKey.OutputControlLevel.PUBLIC_ONLY); +final String privateKeyString = rsaJsonWebKey.toJson(JsonWebKey.OutputControlLevel.INCLUDE_PRIVATE); +``` + +#### 2.2 Using the Private Key in JWK to Implement the Token Issuance Authentication Service +You will need to use the Keypair JSON string (the first inside the three boxes) generated online in section 2.1 or the locally generated privateKeyString JSON string as the private key to issue tokens for authorizing trusted users to access protected APIs. The specific implementation can refer to the example below. The form of issuing tokens to customers can be determined by the user based on the specific business scenario; it can be deployed in the production environment, configured to be a normal API that visitors can access through username and password, or tokens can be generated locally and directly copied for specified users to use. +```java +import java.security.PrivateKey; +import org.jose4j.json.JsonUtil; +import org.jose4j.jwk.RsaJsonWebKey; +import org.jose4j.jwk.RsaJwkGenerator; +import org.jose4j.jws.AlgorithmIdentifiers; +import org.jose4j.jws.JsonWebSignature; +import org.jose4j.jwt.JwtClaims; +import org.jose4j.jwt.NumericDate; +import org.jose4j.lang.JoseException; + +public class GenerateJwtDemo { + public static void main(String[] args) throws JoseException { + String keyId = "uniq_key"; + // Use the Keypair generated in section 2.1 + String privateKeyJson = "{\n" + + " \"kty\": \"RSA\",\n" + + " \"d\": \"O9MJSOgcjjiVMNJ4jmBAh0mRHF_TlaVva70Imghtlgwxl8BLfcf1S8ueN1PD7xV6Cnq8YenSKsfiNOhC6yZ_fjW1syn5raWfj68eR7cjHWjLOvKjwVY33GBPNOvspNhVAFzeqfWneRTBbga53Agb6jjN0SUcZdJgnelzz5JNdOGaLzhacjH6YPJKpbuzCQYPkWtoZHDqWTzCSb4mJ3n0NRTsWy7Pm8LwG_Fd3pACl7JIY38IanPQDLoighFfo-Lriv5z3IdlhwbPnx0tk9sBwQBTRdZ8JkqqYkxUiB06phwr7mAnKEpQJ6HvhZBQ1cCnYZ_nIlrX9-I7qomrlE1UoQ\",\n" + + " \"e\": \"AQAB\",\n" + + " \"alg\": \"RS256\",\n" + + " \"n\": \"vCuB8MgwPZfziMSytEbBoOEwxsG7XI3MaVMoocziP4SjzU4IuWuE_DodbOHQwb_thUru57_Efe--sfATHEa0Odv5ny3QbByqsvjyeHk6ZE4mSAV9BsHYa6GWAgEZtnDceeeDc0y76utXK2XHhC1Pysi2KG8KAzqDa099Yh7s31AyoueoMnrYTmWfEyDsQL_OAIiwgXakkS5U8QyXmWicCwXntDzkIMh8MjfPskesyli0XQD1AmCXVV3h2Opm1Amx0ggSOOiINUR5YRD6mKo49_cN-nrJWjtwSouqDdxHYP-4c7epuTcdS6kQHiQERBd1ejdpAxV4c0t0FHF7MOy9kw\"\n" + + "}"; + JwtClaims claims = new JwtClaims(); + claims.setGeneratedJwtId(); + claims.setIssuedAtToNow(); + // Expiration time must be set + NumericDate date = NumericDate.now(); + date.addSeconds(120*60); + claims.setExpirationTime(date); + claims.setNotBeforeMinutesInThePast(1); + claims.setSubject("YOUR_SUBJECT"); + claims.setAudience("YOUR_AUDIENCE"); + // Add custom parameters, all values should be String type + claims.setClaim("userId", "1213234"); + claims.setClaim("email", "userEmail@youapp.com"); + JsonWebSignature jws = new JsonWebSignature(); + jws.setAlgorithmHeaderValue(AlgorithmIdentifiers.RSA_USING_SHA256); + jws.setKeyIdHeaderValue(keyId); + jws.setPayload(claims.toJson()); + PrivateKey privateKey = new RsaJsonWebKey(JsonUtil.parseJson(privateKeyJson)).getPrivateKey(); + jws.setKey(privateKey); + String jwtResult = jws.getCompactSerialization(); + System.out.println("Generate Json Web token , result is " + jwtResult); + } +} +``` diff --git a/plugins/wasm-cpp/extensions/key_auth/README.md b/plugins/wasm-cpp/extensions/key_auth/README.md index bc42a213f2..786124e033 100644 --- a/plugins/wasm-cpp/extensions/key_auth/README.md +++ b/plugins/wasm-cpp/extensions/key_auth/README.md @@ -1,15 +1,32 @@ -# 功能说明 +--- +title: Key 认证 +keywords: [higress,key auth] +description: Key 认证插件配置参考 +--- + +## 功能说明 `key-auth`插件实现了基于 API Key 进行认证鉴权的功能，支持从 HTTP 请求的 URL 参数或者请求头解析 API Key，同时验证该 API Key 是否有权限访问。 -# 配置字段 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`310` + +## 配置字段 + +**注意：** + +- 在一个规则里，鉴权配置和认证配置不可同时存在 +- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | -| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | -| `keys` | array of string | 必填 | - | API Key 的来源字段名称，可以是 URL 参数或者 HTTP 请求头名称 | -| `in_query` | bool | `in_query` 和 `in_header` 至少有一个为 true | true | 配置 true 时，网关会尝试从 URL 参数中解析 API Key | -| `in_header` | bool | `in_query` 和 `in_header` 至少有一个为 true | true | 配置 true 时，网关会尝试从 HTTP 请求头中解析 API Key | -| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表，用于对请求进行鉴权 | +### 认证配置 +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `global_auth` | bool | 选填（**仅实例级别配置**） | - | 只能在实例级别配置，若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制，若不配置则仅当没有域名和路由配置时全局生效（兼容老用户使用习惯）。 | +| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | +| `keys` | array of string | 必填 | - | API Key 的来源字段名称，可以是 URL 参数或者 HTTP 请求头名称 | +| `in_query` | bool | `in_query` 和 `in_header` 至少有一个为 true | true | 配置 true 时，网关会尝试从 URL 参数中解析 API Key | +| `in_header` | bool | `in_query` 和 `in_header` 至少有一个为 true | true | 配置 true 时，网关会尝试从 HTTP 请求头中解析 API Key | `consumers`中每一项的配置字段说明如下： @@ -18,25 +35,22 @@ | `credential` | string | 必填 | - | 配置该consumer的访问凭证 | | `name` | string | 必填 | - | 配置该consumer的名称 | -`_rules_` 中每一项的配置字段说明如下： +### 鉴权配置（非必需） -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的路由名称 | -| `_match_domain_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的域名 | -| `allow` | array of string | 必填 | - | 对于符合匹配条件的请求，配置允许访问的consumer名称 | +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `allow` | array of string | 选填(**非实例级别配置**) | - | 只能在路由或域名等细粒度规则上配置，对于符合匹配条件的请求，配置允许访问的 consumer，从而实现细粒度的权限控制 | -**注意：** -- 若不配置`_rules_`字段，则默认对当前网关实例的所有路由开启认证； -- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 +## 配置示例 -# 配置示例 +### 全局配置认证和路由粒度进行鉴权 -## 对特定路由或域名开启 +以下配置将对网关特定路由或域名开启Key Auth认证和鉴权。credential字段不能重复。 -以下配置将对网关特定路由或域名开启 Key Auth 认证和鉴权，注意`credential`字段不能重复 +在实例级别做如下插件配置： ```yaml +global_auth: false consumers: - credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 name: consumer1 @@ -44,31 +58,33 @@ consumers: name: consumer2 keys: - apikey -in_query: true -# 使用 _rules_ 字段进行细粒度规则配置 -_rules_: -# 规则一：按路由名称匹配生效 -- _match_route_: - - route-a - - route-b - allow: - - consumer1 -# 规则二：按域名匹配生效 -- _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer2 +- x-api-key +``` + +对 route-a 和 route-b 这两个路由做如下配置： + +```yaml +allow: +- consumer1 +``` + +对 *.example.com 和 test.com 在这两个域名做如下配置: + +```yaml +allow: +- consumer2 ``` -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许`name`为`consumer1`的调用者访问，其他调用者不允许访问； +**说明：** -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将允许`name`为`consumer2`的调用者访问，其他调用者不允许访问。 +此例指定的route-a和route-b即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许name为consumer1的调用者访问，其他调用者不允许访问。 -### 根据该配置，下列请求可以允许访问： +此例指定的*.example.com和test.com用于匹配请求的域名，当发现域名匹配时，将允许name为consumer2的调用者访问，其他调用者不被允许访问。 -假设以下请求会匹配到route-a这条路由 +根据该配置，下列请求可以允许访问： +假设以下请求会匹配到route-a这条路由 +n **将 API Key 设置在 url 参数中** ```bash curl http://xxx.hello.com/test?apikey=2bda943c-ba2b-11ec-ba07-00163e1250b5 @@ -80,7 +96,7 @@ curl http://xxx.hello.com/test -H 'x-api-key: 2bda943c-ba2b-11ec-ba07-00163e125 认证鉴权通过后，请求的header中会被添加一个`X-Mse-Consumer`字段，在此例中其值为`consumer1`，用以标识调用方的名称 -### 下列请求将拒绝访问： +下列请求将拒绝访问： **请求未提供 API Key，返回401** ```bash @@ -97,11 +113,12 @@ curl http://xxx.hello.com/test?apikey=926d90ac-ba2e-11ec-ab68-00163e1250b5 curl http://xxx.hello.com/test?apikey=c8c8e9ca-558e-4a2d-bb62-e700dcc40e35 ``` -## 网关实例级别开启 +### 网关实例级别开启 -以下配置未指定`_rules_`字段，因此将对网关实例级别开启 Key Auth 认证 +以下配置将对网关实例级别开启 Basic Auth 认证，所有请求均需要经过认证后才能访问。 ```yaml +global_auth: true consumers: - credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 name: consumer1 @@ -109,13 +126,15 @@ consumers: name: consumer2 keys: - apikey -in_query: true +- x-api-key ``` -# 相关错误码 + +## 相关错误码 | HTTP 状态码 | 出错信息 | 原因说明 | | ----------- | --------------------------------------------------------- | ----------------------- | -| 401 | No API key found in request | 请求未提供 API Key | +| 401 | Request denied by Key Auth check. Muti API key found in request | 请求提供多个 API Key | +| 401 | Request denied by Key Auth check. No API key found in request | 请求未提供 API Key | | 401 | Request denied by Key Auth check. Invalid API key | 不允许当前 API Key 访问 | -| 403 | Request denied by Basic Auth check. Unauthorized consumer | 请求的调用方无访问权限 | +| 403 | Request denied by Key Auth check. Unauthorized consumer | 请求的调用方无访问权限 | diff --git a/plugins/wasm-cpp/extensions/key_auth/README_EN.md b/plugins/wasm-cpp/extensions/key_auth/README_EN.md index 45a4926456..3abccc4f66 100644 --- a/plugins/wasm-cpp/extensions/key_auth/README_EN.md +++ b/plugins/wasm-cpp/extensions/key_auth/README_EN.md @@ -1,43 +1,47 @@ -# Features -The `key-auth` plug-in implements the authentication function based on the API Key, supports parsing the API Key from the URL parameter or request header of the HTTP request, and verifies whether the API Key has permission to access. - -# Configuration field - -| Name | Data Type | Parameter requirements | Default| Description | -| ----------- | --------------- | -------------------------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------- | -| `consumers` | array of object | Required | - | Configure the caller of the service to authenticate the request. | -| `keys` | array of string | Required | - | The name of the source field of the API Key, which can be a URL parameter or an HTTP request header name. | -| `in_query` | bool | At least one of `in_query` and `in_header` must be true. | true | When configured true, the gateway will try to parse the API Key from the URL parameters. | -| `in_header` | bool | The same as above. | true | The same as above. | -| `_rules_` | array of object | Optional | - | Configure the access list of a specific route or domain name for authenticating requests. | - - -The configuration fields of each item in `consumers` are described as follows: - -| Name | Data Type | Parameter requirements | Default | Description | -| ------------ | --------- | -----------------------| ------ | ------------------------------------------- | -| `credential` | string | Required | - | Configure the consumer's access credentials. | -| `name` | string | Required | - | Configure the name of the consumer. | - -The configuration fields of each item in `_rules_` are described as follows: - -| Name | Data Type | Parameter requirements | Default| Description | -| ---------------- | --------------- | --------------------------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | Optional，Optionally fill in one of `_match_route_`, `_match_domain_`. | - | Configure the route name to match. | -| `_match_domain_` | array of string | Optional，Optionally fill in one of `_match_route_`, `_match_domain_`. | - | Configure the domain name to match. | -| `allow` | array of string | Required | - | For requests that meet the matching conditions, configure the name of the consumer that is allowed to access. | - -**Warning：** -- If the `_rules_` field is not configured, authentication will be enabled for all routes of the current gateway instance by default; -- For a request that passes authentication, an `X-Mse-Consumer` field will be added to the request header to identify the name of the caller. - -# Example configuration - -## Enabled for specific routes or domains - -The following configuration will enable Key Auth authentication and authentication for gateway-specific routes or domain names. Note that the `credential` field can not be repeated. - +--- +title: Key Authentication +keywords: [higress,key auth] +description: Key Authentication Plugin Configuration Reference +--- +## Function Description +The `key-auth` plugin implements authentication based on API Key, supporting the parsing of the API Key from HTTP request URL parameters or request headers, while also verifying whether the API Key has permission to access the resource. + +## Runtime Properties +Plugin Execution Phase: `Authentication Phase` +Plugin Execution Priority: `310` + +## Configuration Fields +**Note:** +- Authentication and authorization configurations cannot coexist within a single rule. +- For requests that are authenticated, a header field `X-Mse-Consumer` will be added to identify the caller's name. + +### Authentication Configuration +| Name | Data Type | Requirements | Default Value | Description | +| ------------- | ---------------- | ----------------------------------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `global_auth` | bool | Optional (**Instance-Level Configuration Only**) | - | Can only be configured at the instance level; if set to true, the authentication mechanism takes effect globally; if set to false, it only applies to the configured hostnames and routes. If not configured, it will only take effect globally when no hostname and route configurations are present (to maintain compatibility with older user habits). | +| `consumers` | array of object | Required | - | Configures the service callers for request authentication. | +| `keys` | array of string | Required | - | Source field names for the API Key, which can be URL parameters or HTTP request header names. | +| `in_query` | bool | At least one of `in_query` and `in_header` must be true | true | When configured as true, the gateway will attempt to parse the API Key from URL parameters. | +| `in_header` | bool | At least one of `in_query` and `in_header` must be true | true | When configured as true, the gateway will attempt to parse the API Key from HTTP request headers. | + +The configuration field descriptions for each item in `consumers` are as follows: +| Name | Data Type | Requirements | Default Value | Description | +| ------------ | --------- | ------------ | ------------- | ------------------------------ | +| `credential` | string | Required | - | Configures the access credential for this consumer. | +| `name` | string | Required | - | Configures the name for this consumer. | + +### Authorization Configuration (Optional) +| Name | Data Type | Requirements | Default Value | Description | +| ----------- | ---------------- | ----------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `allow` | array of string | Optional (**Non-Instance Level Configuration**) | - | Can only be configured on fine-grained rules such as routes or hostnames; specifies the allowed consumers for matching requests, allowing for fine-grained permission control. | + +## Configuration Example +### Global Configuration for Authentication and Granular Route Authorization +The following configuration will enable Key Auth authentication and authorization for specific routes or hostnames in the gateway. The `credential` field must not repeat. + +At the instance level, do the following plugin configuration: ```yaml +global_auth: false consumers: - credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 name: consumer1 @@ -45,64 +49,63 @@ consumers: name: consumer2 keys: - apikey -in_query: true -# Use the _rules_ field for fine-grained rule configuration -_rules_: -# Rule 1: Match by route name to take effect -- _match_route_: - - route-a - - route-b - allow: - - consumer1 -# Rule 2: Take effect by domain name matching -- _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer2 +- x-api-key ``` -The `route-a` and `route-b` specified in `_match_route_` in this example are the route names filled in when creating the gateway route. When these two routes are matched, calls whose `name` is `consumer1` will be allowed Access by callers, other callers are not allowed to access; +For routes route-a and route-b, do the following configuration: +```yaml +allow: +- consumer1 +``` + +For the hostnames *.example.com and test.com, do the following configuration: +```yaml +allow: +- consumer2 +``` -`*.example.com` and `test.com` specified in `_match_domain_` in this example are used to match the domain name of the request. When the domain name matches, the caller whose `name` is `consumer2` will be allowed to access, and other calls access is not allowed. +**Note:** +The routes route-a and route-b specified in this example refer to the route names filled in when creating the gateway routes. When matched with these two routes, requests from the caller named consumer1 will be allowed while others will be denied. -### Depending on this configuration, the following requests would allow access： +The specified hostnames *.example.com and test.com are used to match the request's domain name. When a domain name is matched, callers named consumer2 will be allowed while others will be denied. -Assume that the following request will match the route-a route: +Based on this configuration, the following requests will be allowed: -**Set the API Key in the url parameter** +Assuming the following request matches route-a: +**Setting API Key in URL Parameters** ```bash curl http://xxx.hello.com/test?apikey=2bda943c-ba2b-11ec-ba07-00163e1250b5 ``` -**Set the API Key in the http request header** + +**Setting API Key in HTTP Request Headers** ```bash curl http://xxx.hello.com/test -H 'x-api-key: 2bda943c-ba2b-11ec-ba07-00163e1250b5' ``` -After the authentication is passed, an `X-Mse-Consumer` field will be added to the header of the request. In this example, its value is `consumer1`, which is used to identify the name of the caller. +After successful authentication and authorization, the request's header will have an added `X-Mse-Consumer` field with the value `consumer1`, to identify the name of the caller. -### The following requests will deny access： - -**The request does not provide an API Key, return 401** +The following requests will be denied access: +**Request without an API Key returns 401** ```bash curl http://xxx.hello.com/test ``` -**The API Key provided by the request is not authorized to access, return 401** + +**Request with an invalid API Key returns 401** ```bash curl http://xxx.hello.com/test?apikey=926d90ac-ba2e-11ec-ab68-00163e1250b5 ``` -**The caller matched according to the API Key provided in the request has no access rights, return 403** +**Caller matched with provided API Key has no access rights, returns 403** ```bash # consumer2 is not in the allow list of route-a curl http://xxx.hello.com/test?apikey=c8c8e9ca-558e-4a2d-bb62-e700dcc40e35 ``` -## Gateway instance level enabled - -The following configuration does not specify the `_rules_` field, so Key Auth authentication will be enabled at the gateway instance level. +### Enabling at the Instance Level +The following configuration will enable Basic Auth authentication at the instance level for the gateway, requiring all requests to pass authentication before accessing. ```yaml +global_auth: true consumers: - credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 name: consumer1 @@ -110,14 +113,13 @@ consumers: name: consumer2 keys: - apikey -in_query: true +- x-api-key ``` -# Error code - -| HTTP status code | Error information | Reason | -| ---------------- | --------------------------------------------------------- | -------------------------------------------- | -| 401 | No API key found in request. | API not provided by request Key. | -| 401 | Request denied by Key Auth check. Invalid API key. | Current API Key access is not allowed. | -| 403 | Request denied by Basic Auth check. Unauthorized consumer. | The requested caller does not have access. | - +## Related Error Codes +| HTTP Status Code | Error Message | Reason Explanation | +| ---------------- | ---------------------------------------------------------- | --------------------------------- | +| 401 | Request denied by Key Auth check. Multiple API keys found in request | Multiple API Keys provided in the request. | +| 401 | Request denied by Key Auth check. No API key found in request | API Key not provided in the request. | +| 401 | Request denied by Key Auth check. Invalid API key | The current API Key is not authorized for access. | +| 403 | Request denied by Key Auth check. Unauthorized consumer | The caller does not have access permissions. | diff --git a/plugins/wasm-cpp/extensions/key_rate_limit/README.md b/plugins/wasm-cpp/extensions/key_rate_limit/README.md index 97af36ff8f..8edc00bfa9 100644 --- a/plugins/wasm-cpp/extensions/key_rate_limit/README.md +++ b/plugins/wasm-cpp/extensions/key_rate_limit/README.md @@ -1,11 +1,18 @@ -

- English | 中文 -

+--- +title: 基于 Key 的本地限流 +keywords: [higress,key rate limit] +description: Key 本地限流插件配置参考 +--- -# 功能说明 +## 功能说明 `key-rate-limit`插件实现了基于特定键值实现限流，键值来源可以是 URL 参数、HTTP 请求头 -# 配置字段 +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`10` + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- | -------- | -------- | @@ -23,9 +30,9 @@ | query_per_hour | number | 选填，`query_per_second`,`query_per_minute`,`query_per_hour`,`query_per_day` 中选填一项 | - | 允许每小时请求次数 | | query_per_day | number | 选填，`query_per_second`,`query_per_minute`,`query_per_hour`,`query_per_day` 中选填一项 | - | 允许每天请求次数 | -# 配置示例 +## 配置示例 -## 识别请求参数 apikey，进行区别限流 +### 识别请求参数 apikey，进行区别限流 ```yaml limit_by_param: apikey limit_keys: @@ -35,7 +42,7 @@ limit_keys: query_per_minute: 100 ``` -## 识别请求头 x-ca-key，进行区别限流 +### 识别请求头 x-ca-key，进行区别限流 ```yaml limit_by_header: x-ca-key limit_keys: @@ -45,29 +52,3 @@ limit_keys: query_per_hour: 10 ``` - -## 对特定路由或域名开启 -```yaml -# 使用 _rules_ 字段进行细粒度规则配置 -_rules_: -# 规则一：按路由名称匹配生效 -- _match_route_: - - route-a - - route-b - limit_by_header: x-ca-key - limit_keys: - - key: 102234 - query_per_second: 10 -# 规则二：按域名匹配生效 -- _match_domain_: - - "*.example.com" - - test.com - limit_by_header: x-ca-key - limit_keys: - - key: 102234 - query_per_second: 100 - -``` -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将使用此段配置； -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将使用此段配置； -配置的匹配生效顺序，将按照 `_rules_` 下规则的排列顺序，匹配第一个规则后生效对应配置，后续规则将被忽略。 diff --git a/plugins/wasm-cpp/extensions/key_rate_limit/README_EN.md b/plugins/wasm-cpp/extensions/key_rate_limit/README_EN.md index 7f40c0938a..714dd13984 100644 --- a/plugins/wasm-cpp/extensions/key_rate_limit/README_EN.md +++ b/plugins/wasm-cpp/extensions/key_rate_limit/README_EN.md @@ -1,31 +1,36 @@ -

- English | 中文 -

- -# Description -`key-rate-limit` plugin implements a rate-limiting function based on specific key-values. The key-values may come from URL parameters or HTTP headers. - -# Configuration Fields - -| Name | Type | Requirement | Default Value | Description | -| -------- | -------- | -------- | -------- | -------- | -| limit_by_header | string | Optional. Choose one from following: `limit_by_header`, `limit_by_param`. | - | The name of HTTP header used to obtain key-value used in rate-limiting. | -| limit_by_param | string | Optional. Choose one from following: `limit_by_header`, `limit_by_param`. | - | The name of URL parameter used to obtain key-value used in rate-limiting. | -| limit_keys | array of object | Required | - | Rate-limiting thresholds when matching specific key-values | - -Field descriptions of `limit_keys` items: - -| Name | Type | Requirement | Default Value | Description | -| -------- | -------- | -------- | -------- | -------- | -| key | string | Required | - | Value to match of the specific key | -| query_per_second | number | Optional. Choose one from following: `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day`. | - | Number of requests allowed per second | -| query_per_minute | number | Optional. Choose one from following: `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day`. | - | Number of requests allowed per minute | -| query_per_hour | number | Optional. Choose one from following: `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day`. | - | Number of requests allowed per hour | -| query_per_day | number | Optional. Choose one from following: `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day`. | - | Number of requests allowed per day | - -# Configuration Samples - -## Use query parameter `apikey` for rate-limiting +--- +title: Key-based Local Rate Limiting +keywords: [higress,key rate limit] +description: Configuration reference for Key local rate limiting plugin +--- + +## Functional Description +The `key-rate-limit` plugin implements rate limiting based on specific key values, which can originate from URL parameters or HTTP request headers. + +## Running Properties +Plugin execution phase: `default phase` +Plugin execution priority: `10` + +## Configuration Fields + +| Name | Data Type | Required | Default Value | Description | +|-----------------|-----------------|---------------------------------------------------------------|---------------|----------------------------------------------------------------------------------------| +| limit_by_header | string | Optional, choose one from `limit_by_header`, `limit_by_param` | - | Configuration for the source of the rate limiting key value (HTTP request header name) | +| limit_by_param | string | Optional, choose one from `limit_by_header`, `limit_by_param` | - | Configuration for the source of the rate limiting key value (URL parameter name) | +| limit_keys | array of object | Required | - | Configuration for the rate limiting frequency based on matched key values | + +Explanation of each configuration field in `limit_keys` + +| Name | Data Type | Required | Default Value | Description | +|------------------|-----------|-----------------------------------------------------------------------------------------------------|---------------|---------------------------------------| +| key | string | Required | - | Matched key value | +| query_per_second | number | Optional, choose one from `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day` | - | Allowed number of requests per second | +| query_per_minute | number | Optional, choose one from `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day` | - | Allowed number of requests per minute | +| query_per_hour | number | Optional, choose one from `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day` | - | Allowed number of requests per hour | +| query_per_day | number | Optional, choose one from `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day` | - | Allowed number of requests per day | + +## Configuration Examples +### Identify request parameter apikey for differentiated rate limiting ```yaml limit_by_param: apikey limit_keys: @@ -35,7 +40,7 @@ limit_keys: query_per_minute: 100 ``` -## Use HTTP header parameter `x-ca-key` for rate-limiting +### Identify request header x-ca-key for differentiated rate limiting ```yaml limit_by_header: x-ca-key limit_keys: @@ -44,28 +49,3 @@ limit_keys: - key: 308239 query_per_hour: 10 ``` - -## Enable rate-limiting for specific routes or domains -```yaml -# Use _rules_ field for fine-grained rule configurations -_rules_: -# Rule 1: Match by route name -- _match_route_: - - route-a - - route-b - limit_by_header: x-ca-key - limit_keys: - - key: 102234 - query_per_second: 10 -# Rule 2: Match by domain -- _match_domain_: - - "*.example.com" - - test.com - limit_by_header: x-ca-key - limit_keys: - - key: 102234 - query_per_second: 100 -``` -In the rule sample of `_match_route_`, `route-a` and `route-b` are the route names provided when creating a new gateway route. When the current route names matches the configuration, the rule following shall be applied. -In the rule sample of `_match_domain_`, `*.example.com` and `test.com` are the domain names used for request matching. When the current domain name matches the configuration, the rule following shall be applied. -All rules shall be checked following the order of items in the `_rules_` field, The first matched rule will be applied. All remained will be ignored. \ No newline at end of file diff --git a/plugins/wasm-cpp/extensions/oauth/README.md b/plugins/wasm-cpp/extensions/oauth/README.md index 3a14b794af..0e54260e33 100644 --- a/plugins/wasm-cpp/extensions/oauth/README.md +++ b/plugins/wasm-cpp/extensions/oauth/README.md @@ -1,21 +1,32 @@ -# 功能说明 +--- +title: OAuth2 认证 +keywords: [higress,oauth2] +description: OAuth2 认证插件配置参考 +--- + +## 功能说明 `OAuth2`插件实现了基于JWT(JSON Web Tokens)进行OAuth2 Access Token签发的能力, 遵循[RFC9068](https://datatracker.ietf.org/doc/html/rfc9068)规范 -# 插件配置说明 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`350` ## 配置字段 -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | -| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | -| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表，用于对请求进行鉴权 | -| `issuer` | string | 选填 | Higress-Gateway | 用于填充JWT中的issuer | -| `auth_path` | string | 选填 | /oauth2/token | 指定路径后缀用于签发Token，路由级配置时，要确保首先能匹配对应的路由 | -| `global_credentials` | bool | 选填 | ture | 是否开启全局凭证，即允许路由A下的auth_path签发的Token可以用于访问路由B | -| `auth_header_name` | string | 选填 | Authorization | 用于指定从哪个请求头获取JWT | -| `token_ttl` | number | 选填 | 7200 | token从签发后多久内有效，单位为秒 | -| `clock_skew_seconds` | number | 选填 | 60 | 校验JWT的exp和iat字段时允许的时钟偏移量，单位为秒 | -| `keep_token` | bool | 选填 | ture | 转发给后端时是否保留JWT | +### 认证配置 + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | +| `issuer` | string | 选填 | Higress-Gateway | 用于填充JWT中的issuer | +| `auth_path` | string | 选填 | /oauth2/token | 指定路径后缀用于签发Token，路由级配置时，要确保首先能匹配对应的路由, 使用 API 管理时，需要创建相同路径的接口 | +| `global_credentials` | bool | 选填 | ture | 在通过 consumer 认证的前提下，允许任意路由签发的凭证访问 | +| `auth_header_name` | string | 选填 | Authorization | 用于指定从哪个请求头获取JWT | +| `token_ttl` | number | 选填 | 7200 | token从签发后多久内有效，单位为秒 | +| `clock_skew_seconds` | number | 选填 | 60 | 校验JWT的exp和iat字段时允许的时钟偏移量，单位为秒 | +| `keep_token` | bool | 选填 | ture | 转发给后端时是否保留JWT | +| `global_auth` | array of string | 选填(**仅实例级别配置**) | - | 只能在实例级别配置，若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制; 若不配置则仅当没有域名和路由配置时全局 生效（兼容老用户使用习惯） | `consumers`中每一项的配置字段说明如下： @@ -25,27 +36,53 @@ | `client_id` | string | 必填 | - | OAuth2 client id | | `client_secret` | string | 必填 | - | OAuth2 client secret | -`_rules_` 中每一项的配置字段说明如下： - -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的路由名称 | -| `_match_domain_` | array of string | 选填，`_match_route_`，`_match_domain_`中选填一项 | - | 配置要匹配的域名 | -| `allow` | array of string | 必填 | - | 对于符合匹配条件的请求，配置允许访问的consumer名称 | - **注意：** -- 对于开启该配置的路由，如果路径后缀和`auth_path`匹配，则该路由到原目标服务，而是用于生成Token +- 对于开启该配置的路由，如果路径后缀和`auth_path`匹配，则该路由不会到原目标服务，而是用于生成Token - 如果关闭`global_credentials`,请确保启用此插件的路由不是精确匹配路由，此时若存在另一条前缀匹配路由，则可能导致预期外行为 -- 若不配置`_rules_`字段，则默认对当前网关实例的所有路由开启认证； - 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 +### 鉴权配置（非必需） + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `allow` | array of string | 选填(**非实例级别配置**) | - | 只能在路由或域名等细粒度规则上配置，对于符合匹配条件的请求，配置允许访问的 consumer，从而实现细粒度的权限控制 | + +**注意：** +- 在一个规则里，鉴权配置和认证配置不可同时存在 + ## 配置示例 -### 对特定路由或域名开启 +### 路由粒度配置认证 + +在`route-a`和`route-b`两个路由做如下插件配置： + +```yaml +consumers: +- name: consumer1 + client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx +``` + +此时虽然使用同一份配置，但`route-a` 下签发的凭证无法用于访问 `route-b`，反之亦然。 + +如果希望同一份配置共享凭证访问权限，可以做如下配置: + +```yaml +global_credentials: true +consumers: +- name: consumer1 + client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx +``` + +### 全局配置认证，路由粒度进行鉴权 以下配置将对网关特定路由或域名开启 Jwt Auth 认证和鉴权，注意如果一个JWT能匹配多个`jwks`，则按照配置顺序命中第一个匹配的`consumer` +在实例级别做如下插件配置： + ```yaml +global_auth: false consumers: - name: consumer1 client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx @@ -53,37 +90,54 @@ consumers: - name: consumer2 client_id: 87654321-xxxx-xxxx-xxxx-xxxxxxxxxxxx client_secret: hgfedcba-xxxx-xxxx-xxxx-xxxxxxxxxxxx -# 使用 _rules_ 字段进行细粒度规则配置 -_rules_: -# 规则一：按路由名称匹配生效 -- _match_route_: - - route-a - - route-b - allow: - - consumer1 -# 规则二：按域名匹配生效 -- _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer2 ``` -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许`name`为`consumer1`的调用者访问，其他调用者不允许访问； +在`route-a`和`route-b`两个路由做如下插件配置： + +```yaml +allow: +- consumer1 +``` -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将允许`name`为`consumer2`的调用者访问，其他调用者不允许访问。 +在`*.exmaple.com`和`test.com`两个域名做如下插件配置： -#### 使用 Client Credential 授权模式 +```yaml +allow: +- consumer2 +``` -**获取 AccessToken** +此例指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许`name`为`consumer1`的调用者访问，其他调用者不允许访问； + +此例指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将允许`name`为`consumer2`的调用者访问，其他调用者不允许访问。 + +### 网关实例级别开启 + +以下配置将对网关实例级别开启 OAuth2 认证，所有请求均需要经过认证后才能访问 + +```yaml +global_auth: true +consumers: +- name: consumer1 + client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx +- name: consumer2 + client_id: 87654321-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: hgfedcba-xxxx-xxxx-xxxx-xxxxxxxxxxxx +``` + +# 请求示例 + +## 使用 Client Credential 授权模式 + +### 获取 AccessToken ```bash -# 通过 GET 方法获取 +# 通过 GET 方法获取（推荐） curl 'http://test.com/oauth2/token?grant_type=client_credentials&client_id=12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx&client_secret=abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx' -# 通过 POST 方法获取 (需要先匹配到有真实目标服务的路由) +# 通过 POST 方法获取（需要先匹配到有真实目标服务的路由，否则网关不会读取请求 Body） curl 'http://test.com/oauth2/token' -H 'content-type: application/x-www-form-urlencoded' -d 'grant_type=client_credentials&client_id=12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx&client_secret=abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx' @@ -96,29 +150,13 @@ curl 'http://test.com/oauth2/token' -H 'content-type: application/x-www-form-url ``` -**使用 AccessToken 请求** +### 使用 AccessToken 请求 ```bash curl 'http://test.com' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6ImFwcGxpY2F0aW9uXC9hdCtqd3QifQ.eyJhdWQiOiJkZWZhdWx0IiwiY2xpZW50X2lkIjoiMTIzNDU2NzgteHh4eC14eHh4LXh4eHgteHh4eHh4eHh4eHh4IiwiZXhwIjoxNjg3OTUxNDYzLCJpYXQiOjE2ODc5NDQyNjMsImlzcyI6IkhpZ3Jlc3MtR2F0ZXdheSIsImp0aSI6IjEwOTU5ZDFiLThkNjEtNGRlYy1iZWE3LTk0ODEwMzc1YjYzYyIsInN1YiI6ImNvbnN1bWVyMSJ9.NkT_rG3DcV9543vBQgneVqoGfIhVeOuUBwLJJ4Wycb0' ``` -因为 test.com 仅授权了 consumer2，但这个 Access Token 是基于 consumer1 的 `client_id`，`client_secret` 获取的，因此将返回 `403 Access Denied` - - -### 网关实例级别开启 - -以下配置未指定`_rules_`字段，因此将对网关实例级别开启 OAuth2 认证 - -```yaml -consumers: -- name: consumer1 - client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx - client_secret: abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx -- name: consumer2 - client_id: 87654321-xxxx-xxxx-xxxx-xxxxxxxxxxxx - client_secret: hgfedcba-xxxx-xxxx-xxxx-xxxxxxxxxxxx -``` # 常见错误码说明 @@ -126,4 +164,3 @@ consumers: | ----------- | ---------------------- | -------------------------------------------------------------------------------- | | 401 | Invalid Jwt token | 请求头未提供JWT, 或者JWT格式错误，或过期等原因 | | 403 | Access Denied | 无权限访问当前路由 | - diff --git a/plugins/wasm-cpp/extensions/oauth/README_EN.md b/plugins/wasm-cpp/extensions/oauth/README_EN.md new file mode 100644 index 0000000000..2755bef2fa --- /dev/null +++ b/plugins/wasm-cpp/extensions/oauth/README_EN.md @@ -0,0 +1,138 @@ +--- +title: OAuth2 Authentication +keywords: [higress,oauth2] +description: OAuth2 authentication plugin configuration reference +--- +## Function Description +`OAuth2` plugin implements the capability of issuing OAuth2 Access Tokens based on JWT (JSON Web Tokens), complying with the [RFC9068](https://datatracker.ietf.org/doc/html/rfc9068) specification. + +## Runtime Properties +Plugin execution phase: `Authentication Phase` +Plugin execution priority: `350` + +## Configuration Fields +### Authentication Configuration +| Name | Data Type | Requirement | Default Value | Description | +| -------------------- | ---------------- | ------------------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `consumers` | array of object | Required | - | Configures the callers of the service for request authentication | +| `issuer` | string | Optional | Higress-Gateway | Used to fill the issuer in the JWT | +| `auth_path` | string | Optional | /oauth2/token | Specifies the path suffix for issuing Tokens. When configured at the routing level, ensure it matches the corresponding route first. When using API management, create an interface with the same path. | +| `global_credentials` | bool | Optional | true | Allows any route to issue credentials for access under the condition of authentication through the consumer. | +| `auth_header_name` | string | Optional | Authorization | Specifies which request header to retrieve the JWT from | +| `token_ttl` | number | Optional | 7200 | The time duration in seconds for which the token is valid after issuance. | +| `clock_skew_seconds` | number | Optional | 60 | Allowed clock skew when verifying the exp and iat fields of the JWT, in seconds. | +| `keep_token` | bool | Optional | true | Indicates whether to keep the JWT when forwarding to the backend. | +| `global_auth` | array of string | Optional (**Instance-level configuration only**) | - | Can only be configured at the instance level. If set to true, the global authentication mechanism takes effect; if false, the authentication mechanism only takes effect for configured domains and routes; if not configured, global effect occurs only when there are no domain and route configurations (compatible with legacy user habits). | + +The configuration fields for each item in `consumers` are as follows: +| Name | Data Type | Requirement | Default Value | Description | +| ----------------------- | ------------------| ----------- | ------------------------------------------------- | ---------------------------------- | +| `name` | string | Required | - | Configures the name of the consumer. | +| `client_id` | string | Required | - | OAuth2 client id | +| `client_secret` | string | Required | - | OAuth2 client secret | + +**Note:** +- For routes with this configuration enabled, if the path suffix matches `auth_path`, the route will not forward to the original target service but will be used to generate a Token. +- If `global_credentials` is disabled, please ensure that the routes enabling this plugin do not precisely match routes. If there is another prefix-matching route, it may lead to unexpected behavior. +- For requests authenticated and authorized, the request header will have an `X-Mse-Consumer` field added to identify the caller's name. + +### Authorization Configuration (Optional) +| Name | Data Type | Requirement | Default Value | Description | +| ----------- | ---------------- | ---------------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `allow` | array of string | Optional (**Non-instance-level configuration**) | - | Can only be configured on fine-grained rules such as routes or domains, allowing specified consumers to access requests that meet the matching conditions for fine-grained permission control. | + +**Note:** +- Authentication and authorization configurations cannot coexist in one rule. + +## Configuration Example +### Route Granularity Configuration Authentication +For the two routes `route-a` and `route-b`, do the following plugin configuration: +```yaml +consumers: +- name: consumer1 + client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx +``` +At this time, although using the same configuration, the credentials issued under `route-a` cannot be used to access `route-b`, and vice versa. + +If you want the same configuration to share credential access permissions, you can configure as follows: +```yaml +global_credentials: true +consumers: +- name: consumer1 + client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx +``` + +### Global Configuration Authentication, Route Granularity Authorization +The following configuration will enable Jwt Auth for specific routes or domains on the gateway. Note that if a JWT matches multiple `jwks`, it will hit the first matching `consumer` in the order of configuration. + +At the instance level, do the following plugin configuration: +```yaml +global_auth: false +consumers: +- name: consumer1 + client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx +- name: consumer2 + client_id: 87654321-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: hgfedcba-xxxx-xxxx-xxxx-xxxxxxxxxxxx +``` + +For the routes `route-a` and `route-b`, do the following plugin configuration: +```yaml +allow: +- consumer1 +``` + +For the domains `*.example.com` and `test.com`, do the following plugin configuration: +```yaml +allow: +- consumer2 +``` + +In this example, route names `route-a` and `route-b` refer to the route names filled in when creating the gateway route. When these two routes are matched, it will allow access for the caller with `name` as `consumer1`, and other callers will not be allowed to access. + +In this example, the domains `*.example.com` and `test.com` are used to match request domains. When a matching domain is found, it will allow access for the caller with `name` as `consumer2`, while other callers will not be allowed to access. + +### Enable at Gateway Instance Level +The following configuration will enable OAuth2 authentication at the gateway instance level, requiring all requests to be authenticated before access. +```yaml +global_auth: true +consumers: +- name: consumer1 + client_id: 12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx +- name: consumer2 + client_id: 87654321-xxxx-xxxx-xxxx-xxxxxxxxxxxx + client_secret: hgfedcba-xxxx-xxxx-xxxx-xxxxxxxxxxxx +``` + +# Request Example +## Using Client Credential Authorization Mode +### Get AccessToken +```bash +# Get via GET method (recommended) +curl 'http://test.com/oauth2/token?grant_type=client_credentials&client_id=12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx&client_secret=abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx' + +# Get via POST method (requires matching a route with a real target service first, or the gateway will not read the request Body) +curl 'http://test.com/oauth2/token' -H 'content-type: application/x-www-form-urlencoded' -d 'grant_type=client_credentials&client_id=12345678-xxxx-xxxx-xxxx-xxxxxxxxxxxx&client_secret=abcdefgh-xxxx-xxxx-xxxx-xxxxxxxxxxxx' + +# Simply get the access_token field from the response: +{ + "token_type": "bearer", + "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6ImFwcGxpY2F0aW9uXC9hdCtqd3QifQ.eyJhdWQiOiJkZWZhdWx0IiwiY2xpZW50X2lkIjoiMTIzNDU2NzgteHh4eC14eHh4LXh4eHgteHh4eHh4eHh4eHh4IiwiZXhwIjoxNjg3OTUxNDYzLCJpYXQiOjE2ODc5NDQyNjMsImlzcyI6IkhpZ3Jlc3MtR2F0ZXdheSIsImp0aSI6IjEwOTU5ZDFiLThkNjEtNGRlYy1iZWE3LTk0ODEwMzc1YjYzYyIsInN1YiI6ImNvbnN1bWVyMSJ9.NkT_rG3DcV9543vBQgneVqoGfIhVeOuUBwLJJ4Wycb0", + "expires_in": 7200 +} +``` + +### AccessToken Request +```bash +curl 'http://test.com' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6ImFwcGxpY2F0aW9uXC9hdCtqd3QifQ.eyJhdWQiOiJkZWZhdWx0IiwiY2xpZW50X2lkIjoiMTIzNDU2NzgteHh4eC14eHh4LXh4eHgteHh4eHh4eHh4eHh4IiwiZXhwIjoxNjg3OTUxNDYzLCJpYXQiOjE2ODc5NDQyNjMsImlzcyI6IkhpZ3Jlc3MtR2F0ZXdheSIsImp0aSI6IjEwOTU5ZDFiLThkNjEtNGRlYy1iZWE3LTk0ODEwMzc1YjYzYyIsInN1YiI6ImNvbnN1bWVyMSJ9.NkT_rG3DcV9543vBQgneVqoGfIhVeOuUBwLJJ4Wycb0' +``` + +# Common Error Code Description +| HTTP Status Code | Error Message | Explanation | +| ---------------- | ----------------------| --------------------------------------------------------------------------- | +| 401 | Invalid Jwt token | JWT not provided in request header, or JWT format is incorrect, or expired, etc. | +| 403 | Access Denied | No permission to access the current route. | diff --git a/plugins/wasm-cpp/extensions/request_block/README.md b/plugins/wasm-cpp/extensions/request_block/README.md index 167ffc8907..95c3cdecb7 100644 --- a/plugins/wasm-cpp/extensions/request_block/README.md +++ b/plugins/wasm-cpp/extensions/request_block/README.md @@ -1,11 +1,18 @@ -

- English | 中文 -

+--- +title: 请求屏蔽 +keywords: [higress,request block] +description: 请求屏蔽插件配置参考 +--- -# 功能说明 +## 功能说明 `request-block`插件实现了基于 URL、请求头等特征屏蔽 HTTP 请求，可以用于防护部分站点资源不对外部暴露 -# 配置字段 +## 运行属性 + +插件执行阶段：`鉴权阶段` +插件执行优先级：`320` + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- | -------- | -------- | @@ -16,9 +23,9 @@ | blocked_message | string | 选填 | - | 配置请求被屏蔽时返回的 HTTP 应答 Body | | case_sensitive | bool | 选填 | true | 配置匹配时是否区分大小写，默认区分 | -# 配置示例 +## 配置示例 -## 屏蔽请求 url 路径 +### 屏蔽请求 url 路径 ```yaml block_urls: - swagger.html @@ -33,7 +40,7 @@ curl http://example.com?foo=Bar curl http://exmaple.com/Swagger.html ``` -## 屏蔽请求 header +### 屏蔽请求 header ```yaml block_headers: - example-key @@ -47,7 +54,7 @@ curl http://example.com -H 'example-key: 123' curl http://exmaple.com -H 'my-header: example-value' ``` -## 屏蔽请求 body +### 屏蔽请求 body ```yaml block_bodies: - "hello world" @@ -61,30 +68,8 @@ curl http://example.com -d 'Hello World' curl http://exmaple.com -d 'hello world' ``` -## 对特定路由或域名开启 -```yaml -# 使用 _rules_ 字段进行细粒度规则配置 -_rules_: -# 规则一：按路由名称匹配生效 -- _match_route_: - - route-a - - route-b - block_bodies: - - "hello world" -# 规则二：按域名匹配生效 -- _match_domain_: - - "*.example.com" - - test.com - block_urls: - - "swagger.html" - block_bodies: - - "hello world" -``` -此例 `_match_route_` 中指定的 `route-a` 和 `route-b` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将使用此段配置； -此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将使用此段配置； -配置的匹配生效顺序，将按照 `_rules_` 下规则的排列顺序，匹配第一个规则后生效对应配置，后续规则将被忽略。 -# 请求 Body 大小限制 +## 请求 Body 大小限制 当配置了 `block_bodies` 时，仅支持小于 32 MB 的请求 Body 进行匹配。若请求 Body 大于此限制，并且不存在匹配到的 `block_urls` 和 `block_headers` 项时，不会对该请求执行屏蔽操作 当配置了 `block_bodies` 时，若请求 Body 超过全局配置 DownstreamConnectionBufferLimits，将返回 `413 Payload Too Large` diff --git a/plugins/wasm-cpp/extensions/request_block/README_EN.md b/plugins/wasm-cpp/extensions/request_block/README_EN.md index 01df5afe8a..97a0b46bfb 100644 --- a/plugins/wasm-cpp/extensions/request_block/README_EN.md +++ b/plugins/wasm-cpp/extensions/request_block/README_EN.md @@ -1,24 +1,28 @@ -

- English | 中文 -

- -# Description -`request-block` plugin implements a request blocking function based on request characteristics such as URL and request header. It can be used to protect internal resources from unauthorized access. - -# Configuration Fields - -| Name | Type | Requirement | Default Value | Description | -| -------- | -------- | -------- | -------- | -------- | -| block_urls | array of string | Optional. Choose one from following: `block_urls`, `block_headers`, `block_bodies` | - | HTTP URLs to be blocked. | -| block_headers | array of string | Optional. Choose one from following: `block_urls`, `block_headers`, `block_bodies` | - | HTTP request headers to be blocked. | -| block_bodies | array of string | Optional. Choose one from following: `block_urls` ,`block_headers`, `block_bodies` | - | HTTP request bodies to be blocked. | -| blocked_code | number | Optional | 403 | HTTP response status code to be sent when corresponding request is blocked. | -| blocked_message | string | Optional | - | HTTP response body to be sent when corresponding request is blocked. | -| case_sensitive | bool | Optional | true | Whether to use case-senstive comparison when matching. Enabled by default. | - -# Configuration Samples - -## Block Specific Request URLs +--- +title: Request Blocking +keywords: [higress,request block] +description: Request blocking plugin configuration reference +--- +## Function Description +The `request-block` plugin implements HTTP request blocking based on features such as URL, request headers, etc. It can be used to protect certain site resources from being exposed to the outside. + +## Running Attributes +Plugin Execution Stage: `Authentication Stage` + +Plugin Execution Priority: `320` + +## Configuration Fields +| Name | Data Type | Fill Requirement | Default Value | Description | +|--------------------|--------------------|---------------------------------------------------------|---------------|------------------------------------------------------------| +| block_urls | array of string | Optional, at least one of `block_urls`, `block_headers`, `block_bodies` must be filled | - | Configure strings for matching URLs that need to be blocked | +| block_headers | array of string | Optional, at least one of `block_urls`, `block_headers`, `block_bodies` must be filled | - | Configure strings for matching request headers that need to be blocked | +| block_bodies | array of string | Optional, at least one of `block_urls`, `block_headers`, `block_bodies` must be filled | - | Configure strings for matching request bodies that need to be blocked | +| blocked_code | number | Optional | 403 | Configure the HTTP status code returned when a request is blocked | +| blocked_message | string | Optional | - | Configure the HTTP response body returned when a request is blocked | +| case_sensitive | bool | Optional | true | Configure whether matching is case-sensitive, default is case-sensitive | + +## Configuration Example +### Blocking Request URL Paths ```yaml block_urls: - swagger.html @@ -26,65 +30,39 @@ block_urls: case_sensitive: false ``` -According to the configuration above, following requests will be blocked: - +Based on this configuration, the following requests will be denied access: ```bash curl http://example.com?foo=Bar curl http://exmaple.com/Swagger.html ``` -## Block Specific Request Headers +### Blocking Request Headers ```yaml block_headers: - example-key - example-value ``` -According to the configuration above, following requests will be blocked: - +Based on this configuration, the following requests will be denied access: ```bash curl http://example.com -H 'example-key: 123' curl http://exmaple.com -H 'my-header: example-value' ``` -## Block Specific Request Bodies +### Blocking Request Bodies ```yaml block_bodies: - "hello world" case_sensitive: false ``` -According to the configuration above, following requests will be blocked: - +Based on this configuration, the following requests will be denied access: ```bash curl http://example.com -d 'Hello World' curl http://exmaple.com -d 'hello world' ``` -## Only Enable for Specific Routes or Domains -```yaml -# Use _rules_ field for fine-grained rule configurations -_rules_: -# Rule 1: Match by route name -- _match_route_: - - route-a - - route-b - block_bodies: - - "hello world" -# Rule 2: Match by domain -- _match_domain_: - - "*.example.com" - - test.com - block_urls: - - "swagger.html" - block_bodies: - - "hello world" -``` -In the rule sample of `_match_route_`, `route-a` and `route-b` are the route names provided when creating a new gateway route. When the current route names matches the configuration, the rule following shall be applied. -In the rule sample of `_match_domain_`, `*.example.com` and `test.com` are the domain names used for request matching. When the current domain name matches the configuration, the rule following shall be applied. -All rules shall be checked following the order of items in the `_rules_` field, The first matched rule will be applied. All remained will be ignored. - -# Maximum Request Body Size Limitation +## Request Body Size Limit +When `block_bodies` is configured, only request bodies smaller than 32 MB are supported for matching. If the request body exceeds this limit and there are no matching `block_urls` or `block_headers`, the blocking operation will not be executed for that request. -When `block_bodies` is configured, body matching shall only be performed when its size is smaller than 32MB. If not, and no `block_urls` or `block_headers` configuration is matched, the request won't be blocked. -When `block_bodies` is configured, if the size of request body exceeds the global configuration of DownstreamConnectionBufferLimits, a ``413 Payload Too Large`` response will be returned. \ No newline at end of file +When `block_bodies` is configured and the request body exceeds the global configuration DownstreamConnectionBufferLimits, it will return `413 Payload Too Large`. diff --git a/plugins/wasm-cpp/extensions/sni_misdirect/README.md b/plugins/wasm-cpp/extensions/sni_misdirect/README.md index eae1f5f53e..523ae78fb0 100644 --- a/plugins/wasm-cpp/extensions/sni_misdirect/README.md +++ b/plugins/wasm-cpp/extensions/sni_misdirect/README.md @@ -1,11 +1,14 @@ -# 功能说明 +## 功能说明 + +**此插件已经废弃，从1.4版本开始，Higress 从根本上解决了这一问题，不再需要此插件** + `http2-misdirect`插件用于解决网关开启 HTTP2 时，因为浏览器复用连接导致访问出现 404 等问题。 -# 插件原理 +## 插件原理 HTTP2 协议允许两个不同域名的请求，在域名解析到相同 IP，并且使用了相同证书的情况下，复用同一条连接。这在一些特殊场景会导致复用连接的请求发送给了错误的 Virtual Host 进行处理，从而导致出现 404 等问题。 本插件基于`HTTP/2 RFC 7540`的`9.1.1`和`9.1.2`章节描述，在发现请求 SNI 与当前 Virtual Host 不匹配时，发送 HTTP 421 状态码，强制浏览器新建连接，并根据当前请求域名生成匹配的 SNI，从而让网关能正确处理路由。 -# 浏览器兼容性 +## 浏览器兼容性 `Safari` 浏览器 `15.1` 版本以下不支持 HTTP 421 状态码，若有此类客户端访问场景，建议对相应域名关闭 HTTP2 的 ALPN diff --git a/plugins/wasm-go/extensions/ai-agent/README.md b/plugins/wasm-go/extensions/ai-agent/README.md index 3419fa7ecf..38915a106b 100644 --- a/plugins/wasm-go/extensions/ai-agent/README.md +++ b/plugins/wasm-go/extensions/ai-agent/README.md @@ -9,6 +9,10 @@ description: AI Agent插件配置参考 agent流程图如下： ![ai-agent](https://github.com/user-attachments/assets/b0761a0c-1afa-496c-a98e-bb9f38b340f8) +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`20` ## 配置字段 @@ -347,4 +351,4 @@ curl 'http://<这里换成网关公网IP>/api/openai/v1/chat/completions' \ ```json {"id":"65dcf12c-61ff-9e68-bffa-44fc9e6070d5","choices":[{"index":0,"message":{"role":"assistant","content":" “九头蛇万岁!”的德语翻译为“Hoch lebe Hydra!”。"},"finish_reason":"stop"}],"created":1724043865,"model":"qwen-max-0403","object":"chat.completion","usage":{"prompt_tokens":908,"completion_tokens":52,"total_tokens":960}} -``` \ No newline at end of file +``` diff --git a/plugins/wasm-go/extensions/ai-agent/README_EN.md b/plugins/wasm-go/extensions/ai-agent/README_EN.md new file mode 100644 index 0000000000..4d818f9a6f --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/README_EN.md @@ -0,0 +1,328 @@ +--- +title: AI Agent +keywords: [ AI Gateway, AI Agent ] +description: AI Agent plugin configuration reference +--- +## Functional Description +A customizable API AI Agent that supports configuring HTTP method types as GET and POST APIs. Currently, it only supports non-streaming mode. +The agent flow chart is as follows: +![ai-agent](https://github.com/user-attachments/assets/b0761a0c-1afa-496c-a98e-bb9f38b340f8) + +## Runtime Properties +Plugin execution phase: `Default Phase` +Plugin execution priority: `20` + +## Configuration Fields + +### Basic Configuration +| Name | Data Type | Requirement | Default Value | Description | +|------------------|-----------|-------------|---------------|----------------------------------| +| `llm` | object | Required | - | Configuration information for AI service provider | +| `apis` | object | Required | - | Configuration information for external API service provider | +| `promptTemplate` | object | Optional | - | Configuration information for Agent ReAct template | + +The configuration fields for `llm` are as follows: +| Name | Data Type | Requirement | Default Value | Description | +|--------------------|-----------|-------------|---------------|-------------------------------------| +| `apiKey` | string | Required | - | Token for authentication when accessing large model services. | +| `serviceName` | string | Required | - | Name of the large model service | +| `servicePort` | int | Required | - | Port of the large model service | +| `domain` | string | Required | - | Domain for accessing the large model service | +| `path` | string | Required | - | Path for accessing the large model service | +| `model` | string | Required | - | Model name for accessing the large model service | +| `maxIterations` | int | Required | 15 | Maximum steps before ending the execution loop | +| `maxExecutionTime` | int | Required | 50000 | Timeout for each request to the large model, in milliseconds | +| `maxTokens` | int | Required | 1000 | Token limit for each request to the large model | + +The configuration fields for `apis` are as follows: +| Name | Data Type | Requirement | Default Value | Description | +|-----------------|-----------|-------------|---------------|-------------------------------------| +| `apiProvider` | object | Required | - | Information about the external API service | +| `api` | string | Required | - | OpenAPI documentation of the tool | + +The configuration fields for `apiProvider` are as follows: +| Name | Data Type | Requirement | Default Value | Description | +|-----------------|-----------|-------------|---------------|--------------------------------------------------| +| `apiKey` | object | Optional | - | Token for authentication when accessing external API services. | +| `serviceName` | string | Required | - | Name of the external API service | +| `servicePort` | int | Required | - | Port of the external API service | +| `domain` | string | Required | - | Domain for accessing the external API | + +The configuration fields for `apiKey` are as follows: +| Name | Data Type | Requirement | Default Value | Description | +|-------------------|-----------|-------------|---------------|-------------------------------------------------------------------------------------| +| `in` | string | Optional | header | Whether the authentication token for accessing the external API service is in the header or in the query; default is header. | +| `name` | string | Optional | - | The name of the token for authentication when accessing the external API service. | +| `value` | string | Optional | - | The value of the token for authentication when accessing the external API service. | + +The configuration fields for `promptTemplate` are as follows: +| Name | Data Type | Requirement | Default Value | Description | +|-----------------|-----------|-------------|---------------|----------------------------------------------------| +| `language` | string | Optional | EN | Language type of the Agent ReAct template, including CH and EN. | +| `chTemplate` | object | Optional | - | Agent ReAct Chinese template | +| `enTemplate` | object | Optional | - | Agent ReAct English template | + +The configuration fields for `chTemplate` and `enTemplate` are as follows: +| Name | Data Type | Requirement | Default Value | Description | +|-----------------|-----------|-------------|---------------|---------------------------------------------------| +| `question` | string | Optional | - | The question part of the Agent ReAct template | +| `thought1` | string | Optional | - | The thought1 part of the Agent ReAct template | +| `actionInput` | string | Optional | - | The actionInput part of the Agent ReAct template | +| `observation` | string | Optional | - | The observation part of the Agent ReAct template | +| `thought2` | string | Optional | - | The thought2 part of the Agent ReAct template | +| `finalAnswer` | string | Optional | - | The finalAnswer part of the Agent ReAct template | +| `begin` | string | Optional | - | The begin part of the Agent ReAct template | + +## Usage Example +**Configuration Information** +```yaml +llm: + apiKey: xxxxxxxxxxxxxxxxxx + domain: dashscope.aliyuncs.com + serviceName: dashscope.dns + servicePort: 443 + path: /compatible-mode/v1/chat/completions + model: qwen-max-0403 + maxIterations: 2 +promptTemplate: + language: CH +apis: +- apiProvider: + domain: restapi.amap.com + serviceName: geo.dns + servicePort: 80 + apiKey: + in: query + name: key + value: xxxxxxxxxxxxxxx + api: | + openapi: 3.1.0 + info: + title: Amap + description: Get related information of POI + version: v1.0.0 + servers: + - url: https://restapi.amap.com + paths: + /v5/place/text: + get: + description: Get latitude and longitude coordinates based on POI name + operationId: get_location_coordinate + parameters: + - name: keywords + in: query + description: POI name, must be in Chinese + required: true + schema: + type: string + - name: region + in: query + description: The name of the region where the POI is located, must be in Chinese + required: true + schema: + type: string + deprecated: false + /v5/place/around: + get: + description: Search for POI near the given coordinates + operationId: search_nearby_pois + parameters: + - name: keywords + in: query + description: Keywords for the target POI + required: true + schema: + type: string + - name: location + in: query + description: Latitude and longitude of the center point, separated by a comma + required: true + schema: + type: string + deprecated: false + components: + schemas: {} +- apiProvider: + domain: api.seniverse.com + serviceName: seniverse.dns + servicePort: 80 + apiKey: + in: query + name: key + value: xxxxxxxxxxxxxxx + api: | + openapi: 3.1.0 + info: + title: XZWeather + description: Get weather related information + version: v1.0.0 + servers: + - url: https://api.seniverse.com + paths: + /v3/weather/now.json: + get: + description: Get weather conditions for a specified city + operationId: get_weather_now + parameters: + - name: location + in: query + description: The city to query + required: true + schema: + type: string + - name: language + in: query + description: Language used for the weather query results + required: true + schema: + type: string + default: zh-Hans + enum: + - zh-Hans + - en + - ja + - name: unit + in: query + description: Units of temperature, available in Celsius and Fahrenheit + required: true + schema: + type: string + default: c + enum: + - c + - f + deprecated: false + components: + schemas: {} +- apiProvider: + apiKey: + in: "header" + name: "DeepL-Auth-Key" + value: "73xxxxxxxxxxxxxxx:fx" + domain: "api-free.deepl.com" + serviceName: "deepl.dns" + servicePort: 443 + api: | + openapi: 3.1.0 + info: + title: DeepL API Documentation + description: The DeepL API provides programmatic access to DeepL’s machine translation technology. + version: v1.0.0 + servers: + - url: https://api-free.deepl.com/v2 + paths: + /translate: + post: + summary: Request Translation + operationId: translateText + requestBody: + required: true + content: + application/json: + schema: + type: object + required: + - text + - target_lang + properties: + text: + description: | + Text to be translated. Only UTF-8-encoded plain text is supported. + The parameter may be specified up to 50 times in a single request. + Translations are returned in the same order as they are requested. + type: array + maxItems: 50 + items: + type: string + example: Hello, World! + target_lang: + description: The language into which the text should be translated. + type: string + enum: + - BG + - CS + - DA + - DE + - EL + - EN-GB + - EN-US + - ES + - ET + - FI + - FR + - HU + - ID + - IT + - JA + - KO + - LT + - LV + - NB + - NL + - PL + - PT-BR + - PT-PT + - RO + - RU + - SK + - SL + - SV + - TR + - UK + - ZH + - ZH-HANS + example: DE + components: + schemas: {} +``` +This example configures three services demonstrating both GET and POST types of tools. The GET type tools include Amap and XZWeather, while the POST type tool is the DeepL translation. All three services need to be properly configured in the Higress service with DNS domain names and should be healthy. +Amap provides two tools, one for obtaining the coordinates of a specified location and the other for searching for points of interest near the coordinates. Document: https://lbs.amap.com/api/webservice/guide/api-advanced/newpoisearch +XZWeather provides one tool to get real-time weather conditions for a specified city, supporting results in Chinese, English, and Japanese, as well as representations in Celsius and Fahrenheit. Document: https://seniverse.yuque.com/hyper_data/api_v3/nyiu3t +DeepL provides one tool for translating given sentences, supporting multiple languages. Document: https://developers.deepl.com/docs/v/zh/api-reference/translate?fallback=true + +Below are test cases. For stability, it is recommended to maintain a stable version of the large model. The example used here is qwen-max-0403: +**Request Example** +```shell +curl 'http:///api/openai/v1/chat/completions' \ +-H 'Accept: application/json, text/event-stream' \ +-H 'Content-Type: application/json' \ +--data-raw '{"model":"qwen","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[{"role":"user","content":"I want to have coffee near the Xinshi Building in Jinan, please recommend a few."}],"presence_penalty":0,"temperature":0,"top_p":0}' +``` +**Response Example** +```json +{"id":"139487e7-96a0-9b13-91b4-290fb79ac992","choices":[{"index":0,"message":{"role":"assistant","content":" Near the Xinshi Building in Jinan, you can choose from the following coffee shops:\n1. luckin coffee 瑞幸咖啡(鑫盛大厦店), located in the lobby of Xinshi Building, No. 1299 Xinluo Avenue;\n2. 三庆齐盛广场挪瓦咖啡(三庆·齐盛广场店), located 60 meters southwest of the intersection of Xinluo Avenue and Yingxiu Road;\n3. luckin coffee 瑞幸咖啡(三庆·齐盛广场店), located at No. 1267 Yingxiu Road;\n4. 库迪咖啡(齐鲁软件园店), located in the commercial space of Building 4, Sanqing Qisheng Plaza, Xinluo Avenue;\n5. 库迪咖啡(美莲广场店), located at L117, Meilian Plaza, No. 1166 Xinluo Avenue, High-tech Zone; and a few other options. I hope these suggestions help!"},"finish_reason":"stop"}],"created":1723172296,"model":"qwen-max-0403","object":"chat.completion","usage":{"prompt_tokens":886,"completion_tokens":50,"total_tokens":936}} +``` +**Request Example** +```shell +curl 'http:///api/openai/v1/chat/completions' \ +-H 'Accept: application/json, text/event-stream' \ +-H 'Content-Type: application/json' \ +--data-raw '{"model":"qwen","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[{"role":"user","content":"What is the current weather in Jinan?"}],"presence_penalty":0,"temperature":0,"top_p":0}' +``` +**Response Example** +```json +{"id":"ebd6ea91-8e38-9e14-9a5b-90178d2edea4","choices":[{"index":0,"message":{"role":"assistant","content":" The current weather condition in Jinan is overcast, with a temperature of 31°C. This information was last updated on August 9, 2024, at 15:12 (Beijing time)."},"finish_reason":"stop"}],"created":1723187991,"model":"qwen-max-0403","object":"chat.completion","usage":{"prompt_tokens":890,"completion_tokens":56,"total_tokens":946}} +``` +**Request Example** +```shell +curl 'http:///api/openai/v1/chat/completions' \ +-H 'Accept: application/json, text/event-stream' \ +-H 'Content-Type: application/json' \ +--data-raw '{"model":"qwen","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[{"role":"user","content":"What is the current weather in Jinan? Please indicate in Fahrenheit and respond in Japanese."}],"presence_penalty":0,"temperature":0,"top_p":0}' +``` +**Response Example** +```json +{"id":"ebd6ea91-8e38-9e14-9a5b-90178d2edea4","choices":[{"index":0,"message":{"role":"assistant","content":" 現在の济南の天気は曇りで、気温は88°Fです。この情報は2024年8月9日15時12分（東京時間）に更新されました。"},"finish_reason":"stop"}],"created":1723187991,"model":"qwen-max-0403","object":"chat.completion","usage":{"prompt_tokens":890,"completion_tokens":56,"total_tokens":946}} +``` +**Request Example** +```shell +curl 'http:///api/openai/v1/chat/completions' \ +-H 'Accept: application/json, text/event-stream' \ +-H 'Content-Type: application/json' \ +--data-raw '{"model":"qwen","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[{"role":"user","content":"Help me translate the following sentence into German: \"Hail Hydra!\""}],"presence_penalty":0,"temperature":0,"top_p":0}' +``` +**Response Example** +```json +{"id":"65dcf12c-61ff-9e68-bffa-44fc9e6070d5","choices":[{"index":0,"message":{"role":"assistant","content":" The German translation of \"Hail Hydra!\" is \"Hoch lebe Hydra!\"."},"finish_reason":"stop"}],"created":1724043865,"model":"qwen-max-0403","object":"chat.completion","usage":{"prompt_tokens":908,"completion_tokens":52,"total_tokens":960}} +``` + diff --git a/plugins/wasm-go/extensions/ai-cache/README.md b/plugins/wasm-go/extensions/ai-cache/README.md index 4e70c4e050..97728f5177 100644 --- a/plugins/wasm-go/extensions/ai-cache/README.md +++ b/plugins/wasm-go/extensions/ai-cache/README.md @@ -1,13 +1,19 @@ -## 简介 +--- +title: AI 缓存 +keywords: [higress,ai cache] +description: AI 缓存插件配置参考 +--- -**Note** -> 需要数据面的proxy wasm版本大于等于0.2.100 - -> 编译时，需要带上版本的tag，例如：`tinygo build -o main.wasm -scheduler=none -target=wasi -gc=custom -tags="custommalloc nottinygc_finalizer proxy_wasm_version_0_2_100" ./` +## 功能说明 LLM 结果缓存插件，默认配置方式可以直接用于 openai 协议的结果缓存，同时支持流式和非流式响应的缓存。 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`10` + ## 配置说明 | Name | Type | Requirement | Default | Description | diff --git a/plugins/wasm-go/extensions/ai-cache/README_EN.md b/plugins/wasm-go/extensions/ai-cache/README_EN.md new file mode 100644 index 0000000000..81099e509c --- /dev/null +++ b/plugins/wasm-go/extensions/ai-cache/README_EN.md @@ -0,0 +1,41 @@ +--- +title: AI Cache +keywords: [higress,ai cache] +description: AI Cache Plugin Configuration Reference +--- +## Function Description +LLM result caching plugin, the default configuration can be directly used for result caching under the OpenAI protocol, and it supports caching of both streaming and non-streaming responses. + +## Runtime Properties +Plugin Execution Phase: `Authentication Phase` +Plugin Execution Priority: `10` + +## Configuration Description +| Name | Type | Requirement | Default | Description | +| -------- | -------- | -------- | -------- | -------- | +| cacheKeyFrom.requestBody | string | optional | "messages.@reverse.0.content" | Extracts a string from the request Body based on [GJSON PATH](https://github.com/tidwall/gjson/blob/master/SYNTAX.md) syntax | +| cacheValueFrom.responseBody | string | optional | "choices.0.message.content" | Extracts a string from the response Body based on [GJSON PATH](https://github.com/tidwall/gjson/blob/master/SYNTAX.md) syntax | +| cacheStreamValueFrom.responseBody | string | optional | "choices.0.delta.content" | Extracts a string from the streaming response Body based on [GJSON PATH](https://github.com/tidwall/gjson/blob/master/SYNTAX.md) syntax | +| cacheKeyPrefix | string | optional | "higress-ai-cache:" | Prefix for the Redis cache key | +| cacheTTL | integer | optional | 0 | Cache expiration time in seconds, default value is 0, which means never expire | +| redis.serviceName | string | required | - | The complete FQDN name of the Redis service, including the service type, e.g., my-redis.dns, redis.my-ns.svc.cluster.local | +| redis.servicePort | integer | optional | 6379 | Redis service port | +| redis.timeout | integer | optional | 1000 | Timeout for requests to Redis, in milliseconds | +| redis.username | string | optional | - | Username for logging into Redis | +| redis.password | string | optional | - | Password for logging into Redis | +| returnResponseTemplate | string | optional | `{"id":"from-cache","choices":[%s],"model":"gpt-4o","object":"chat.completion","usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}` | Template for returning HTTP response, with %s marking the part to be replaced by cache value | +| returnStreamResponseTemplate | string | optional | `data:{"id":"from-cache","choices":[{"index":0,"delta":{"role":"assistant","content":"%s"},"finish_reason":"stop"}],"model":"gpt-4o","object":"chat.completion","usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}\n\ndata:[DONE]\n\n` | Template for returning streaming HTTP response, with %s marking the part to be replaced by cache value | + +## Configuration Example +```yaml +redis: + serviceName: my-redis.dns + timeout: 2000 +``` + +## Advanced Usage +The current default cache key is based on the GJSON PATH expression: `messages.@reverse.0.content`, meaning to get the content of the first item after reversing the messages array; +GJSON PATH supports conditional syntax, for instance, if you want to take the content of the last role as user as the key, it can be written as: `messages.@reverse.#(role=="user").content`; +If you want to concatenate all the content with role as user into an array as the key, it can be written as: `messages.@reverse.#(role=="user")#.content`; +It also supports pipeline syntax, for example, if you want to take the second role as user as the key, it can be written as: `messages.@reverse.#(role=="user")#.content|1`. +For more usage, you can refer to the [official documentation](https://github.com/tidwall/gjson/blob/master/SYNTAX.md) and use the [GJSON Playground](https://gjson.dev/) for syntax testing. diff --git a/plugins/wasm-go/extensions/ai-history/README.md b/plugins/wasm-go/extensions/ai-history/README.md index 8d1e6a3a1d..d4684d292d 100644 --- a/plugins/wasm-go/extensions/ai-history/README.md +++ b/plugins/wasm-go/extensions/ai-history/README.md @@ -7,14 +7,16 @@ description: AI 历史对话插件配置参考 ## 功能说明 `AI 历史对话` 基于请求头实现用户身份识别，并自动缓存对应用户的历史对话,且在后续对话中自动填充到上下文。同时支持用户主动查询历史对话。 + **Note** -> 需要数据面的proxy wasm版本大于等于0.2.100 +> 路径后缀匹配 `ai-history/query` 时，会返回历史对话 + +## 运行属性 -> 编译时，需要带上版本的tag，例如： -`tinygo build -o main.wasm -scheduler=none -target=wasi -gc=custom -tags="custommalloc nottinygc_finalizer proxy_wasm_version_0_2_100" ./` +插件执行阶段：`默认阶段` +插件执行优先级：`650` -> 路径后缀匹配 `ai-history/query` 时，会返回历史对话 ## 配置字段 @@ -169,4 +171,4 @@ curl 'http://example.com/api/openai/v1/chat/completions/ai-history/query?cnt=3' ] ``` -返回三个历史对话,如果未传入 cnt 默认返回所有缓存历史对话。 \ No newline at end of file +返回三个历史对话,如果未传入 cnt 默认返回所有缓存历史对话。 diff --git a/plugins/wasm-go/extensions/ai-history/README_EN.md b/plugins/wasm-go/extensions/ai-history/README_EN.md new file mode 100644 index 0000000000..1fc6144d40 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-history/README_EN.md @@ -0,0 +1,158 @@ +--- +title: AI History Dialogue +keywords: [ AI Gateway, AI History Dialogue ] +description: AI History Dialogue Plugin Configuration Reference +--- +## Functional Description +`AI History Dialogue` implements user identity recognition based on request headers and automatically caches the historical dialogues of corresponding users, which are then automatically filled into the context in subsequent dialogues. It also supports users to actively query historical dialogues. + +**Note** + +> When the path suffix matches `ai-history/query`, it will return the historical dialogues. + +## Runtime Properties +Plugin Execution Phase: `Default Phase` +Plugin Execution Priority: `650` + +## Configuration Fields +| Name | Data Type | Required | Default Value | Description | +|-------------------|---------|----------|-----------------------|---------------------------------------------------------------------------| +| identityHeader | string | optional | "Authorization" | The request header for identity resolution, can be Authorization, X-Mse-Consumer, etc. | +| fillHistoryCnt | integer | optional | 3 | Default number of historical dialogues to be filled. | +| cacheKeyPrefix | string | optional | "higress-ai-history:" | Prefix for Redis cache key. | +| cacheTTL | integer | optional | 0 | Cache expiration time in seconds, default value is 0, meaning it never expires. | +| redis.serviceName | string | required | - | Redis service name, full FQDN name with service type, e.g., my-redis.dns, redis.my-ns.svc.cluster.local | +| redis.servicePort | integer | optional | 6379 | Redis service port. | +| redis.timeout | integer | optional | 1000 | Timeout for requests to Redis, in milliseconds. | +| redis.username | string | optional | - | Username for logging into Redis. | +| redis.password | string | optional | - | Password for logging into Redis. | + +## Usage Example +### Configuration Information +```yaml +redis: + serviceName: my-redis.dns + timeout: 2000 +``` + +### Request Example +**Auto-fill Request Example:** + +First Round Request: +``` + curl 'http://example.com/api/openai/v1/chat/completions?fill_history_cnt=3' \ + -H 'Accept: application/json, text/event-stream' \ + -H 'Content-Type: application/json' \ + -H 'Authorization: Bearer sk-Nzf7RtkdS4s0zFyn5575124129254d9bAf9473A5D7D06dD3' + --data-raw '{"model":"qwen-long","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[ + { + "role": "user", + "content": "Can Higress replace Nginx?" + } + ],"presence_penalty":0,"temperature":0.7,"top_p":0.95}' +``` +After Request Fill: +> First round request, no fill. Consistent with the original request. + +First Round Response: +```json +{ + "id": "02f4c621-820e-97d4-a905-1e3d0d8f59c6", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "While both Higress and Nginx have gateway functionalities, their design philosophies and application scenarios differ. Nginx is better known as a high-performance HTTP and reverse proxy server, while Higress is a cloud-native gateway that integrates many cloud-native features such as service mesh, observability, and security management in addition to basic routing capabilities.\n\nTherefore, if you want to deploy applications in a cloud-native environment and wish to obtain advanced features required for modern applications, such as service governance, gray release, circuit breaker and rate limiting, and security authentication, then Higress can be a good alternative to Nginx. However, if it's a relatively simple static website or only requires basic reverse proxy functionality, traditional Nginx configurations may be simpler and more direct." + }, + "finish_reason": "stop" + } + ], + "created": 1724077770, + "model": "qwen-long", + "object": "chat.completion", + "usage": { + "prompt_tokens": 7316, + "completion_tokens": 164, + "total_tokens": 7480 + } +} +``` + +Second Round Request: +``` + curl 'http://example.com/api/openai/v1/chat/completions?fill_history_cnt=3' \ + -H 'Accept: application/json, text/event-stream' \ + -H 'Content-Type: application/json' \ + -H 'Authorization: Bearer sk-Nzf7RtkdS4s0zFyn5575124129254d9bAf9473A5D7D06dD3' + --data-raw '{"model":"qwen-long","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[ + { + "role": "user", + "content": "What about Spring Cloud GateWay?" + } + ],"presence_penalty":0,"temperature":0.7,"top_p":0.95}' +``` +After Request Fill: +> Second round request, automatically filled with the historical dialogue from the previous round. +``` + curl 'http://example.com/api/openai/v1/chat/completions?fill_history_cnt=3' \ + -H 'Accept: application/json, text/event-stream' \ + -H 'Content-Type: application/json' \ + -H 'Authorization: Bearer sk-Nzf7RtkdS4s0zFyn5575124129254d9bAf9473A5D7D06dD3' + --data-raw '{"model":"qwen-long","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[ + { + "role": "user", + "content": "Can Higress replace Nginx?" + }, + { + "role": "assistant", + "content": "While both Higress and Nginx have gateway functionalities, their design philosophies and application scenarios differ. Nginx is better known as a high-performance HTTP and reverse proxy server, while Higress is a cloud-native gateway that integrates many cloud-native features such as service mesh, observability, and security management in addition to basic routing capabilities.\n\nTherefore, if you want to deploy applications in a cloud-native environment and wish to obtain advanced features required for modern applications, such as service governance, gray release, circuit breaker and rate limiting, and security authentication, then Higress can be a good alternative to Nginx. However, if it's a relatively simple static website or only requires basic reverse proxy functionality, traditional Nginx configurations may be simpler and more direct." + }, + { + "role": "user", + "content": "What about Spring Cloud GateWay?" + } + ],"presence_penalty":0,"temperature":0.7,"top_p":0.95}' +``` + +Each round of requests only needs to include the current question and the number of historical dialogues to fill in, enabling automatic filling of historical dialogues. + +**Example of Retrieving Historical Data:** +``` +curl 'http://example.com/api/openai/v1/chat/completions/ai-history/query?cnt=3' \ + -H 'Accept: application/json, text/event-stream' \ + -H 'Content-Type: application/json' \ + -H 'Authorization: Bearer sk-Nzf7RtkdS4s0zFyn5575124129254d9bAf9473A5D7D06dD3' +``` + +Response Example: +```json +[ + { + "role": "user", + "content": "Can Higress replace Nginx?" + }, + { + "role": "assistant", + "content": "While both Higress and Nginx have gateway functionalities, their design philosophies and application scenarios differ. Nginx is better known as a high-performance HTTP and reverse proxy server, while Higress is a cloud-native gateway that integrates many cloud-native features such as service mesh, observability, and security management in addition to basic routing capabilities.\\n\\nTherefore, if you want to deploy applications in a cloud-native environment and wish to obtain advanced features required for modern applications, such as service governance, gray release, circuit breaker and rate limiting, and security authentication, then Higress can be a good alternative to Nginx. However, if it's a relatively simple static website or only requires basic reverse proxy functionality, traditional Nginx configurations may be simpler and more direct." + }, + { + "role": "user", + "content": "What about Spring Cloud GateWay?" + }, + { + "role": "assistant", + "content": "Compared to Spring Cloud Gateway, Higress is also an API gateway, but there are some key differences between them:\\n\\n- **Design Philosophy**: Spring Cloud Gateway mainly targets service-to-service communication and routing in microservice architectures. As part of the Spring Cloud ecosystem, it is more focused on scenarios for Java developers in microservices. In contrast, Higress, as a cloud-native gateway, not only focuses on service communication but also offers a range of cloud-native features such as service mesh, observability, security management, etc.\\n- **Deployment Method**: Spring Cloud Gateway typically runs as part of microservice applications on application servers, whereas Higress usually deploys as an independent microservice or containerized service in Kubernetes environments, suitable for modern cloud-native deployment models.\\n- **Scalability and Integration**: Higress provides wider integration and support, for example, deep integration with ecosystems like Istio, Kubernetes, etc., making it better suited for complex cloud-native environments.\\n\\nTherefore, if your application is built on Spring Cloud and you want a lightweight, easy-to-integrate service gateway, then Spring Cloud Gateway might be a suitable choice. However, if you are building or refactoring cloud-native applications and require more powerful routing rules, service governance, observability, etc., then Higress is likely a better choice." + }, + { + "role": "user", + "content": "Can Higress replace Nginx?" + }, + { + "role": "assistant", + "content": "While both Higress and Nginx have gateway functionalities, their design philosophies and application scenarios differ. Nginx is better known as a high-performance HTTP and reverse proxy server, while Higress is a cloud-native gateway that integrates many cloud-native features such as service mesh, observability, and security management in addition to basic routing capabilities.\\n\\nTherefore, if you want to deploy applications in a cloud-native environment and wish to obtain advanced features required for modern applications, such as service governance, gray release, circuit breaker and rate limiting, and security authentication, then Higress can be a good alternative to Nginx. However, if it's a relatively simple static website or only requires basic reverse proxy functionality, traditional Nginx configurations may be simpler and more direct." + } +] +``` + +Returns three historical dialogues. If the `cnt` parameter is not provided, it will default to returning all cached historical dialogues. diff --git a/plugins/wasm-go/extensions/ai-intent/README.md b/plugins/wasm-go/extensions/ai-intent/README.md index 93854c3629..5d448d0344 100644 --- a/plugins/wasm-go/extensions/ai-intent/README.md +++ b/plugins/wasm-go/extensions/ai-intent/README.md @@ -1,15 +1,20 @@ -## 简介 +--- +title: AI 意图识别 +keywords: [ AI网关, AI意图识别 ] +description: AI 意图识别插件配置参考 +--- -**Note** +## 功能说明 -> 需要数据面的proxy wasm版本大于等于0.2.100 +LLM 意图识别插件，能够智能判断用户请求与某个领域或agent的功能契合度，从而提升不同模型的应用效果和用户体验 -> 编译时，需要带上版本的tag，例如：`tinygo build -o main.wasm -scheduler=none -target=wasi -gc=custom -tags="custommalloc nottinygc_finalizer proxy_wasm_version_0_2_100" ./` +## 运行属性 -LLM 意图识别插件，能够智能判断用户请求与某个领域或agent的功能契合度，从而提升不同模型的应用效果和用户体验 +插件执行阶段：`默认阶段` +插件执行优先级：`700` ## 配置说明 -> 1.该插件的优先级要高于ai-cache、ai-proxy等后续使用意图的插件，后续插件可以通过proxywasm.GetProperty([]string{"intent_category"})方法获取到意图主题，按照意图主题去做不同缓存库或者大模型的选择 +> 1.该插件的优先级高于ai-proxy等后续使用意图的插件，后续插件可以通过proxywasm.GetProperty([]string{"intent_category"})方法获取到意图主题，按照意图主题去做不同缓存库或者大模型的选择 > 2.需新建一条higress的大模型路由，供该插件访问大模型,如：路由以 /intent 作为前缀，服务选择大模型服务，为该路由开启ai-proxy插件 @@ -19,7 +24,7 @@ LLM 意图识别插件，能够智能判断用户请求与某个领域或agent | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------------- | --------------- | -------- | ------ | ------------------------------------------------------------ | -| `scene.category` | string | 必填 | - | 预设场景类别，以"|"分割，如："金融|电商|法律|Higress"| +| `scene.category` | string | 必填 | - | 预设场景类别，以`|`分割，如：`金融|电商|法律|Higress`| | `scene.prompt` | string | 非必填 | 你是一个智能类别识别助手，负责根据用户提出的问题和预设的类别，确定问题属于哪个预设的类别，并给出相应的类别。用户提出的问题为:%s,预设的类别为%s，直接返回一种具体类别，如果没有找到就返回'NotFound'。 | llm请求prompt模板 | | `llm.proxyServiceName` | string | 必填 | - | 新建的higress服务，指向大模型 (取higress中的 FQDN 值)| | `llm.proxyUrl` | string | 必填 | - | 大模型路由请求地址全路径，可以是网关自身的地址，也可以是其他大模型的地址（openai协议），例如：http://127.0.0.1:80/intent/compatible-mode/v1/chat/completions | diff --git a/plugins/wasm-go/extensions/ai-intent/README_EN.md b/plugins/wasm-go/extensions/ai-intent/README_EN.md new file mode 100644 index 0000000000..e2ac7e283f --- /dev/null +++ b/plugins/wasm-go/extensions/ai-intent/README_EN.md @@ -0,0 +1,45 @@ +--- +title: AI Intent Recognition +keywords: [ AI Gateway, AI Intent Recognition ] +description: AI Intent Recognition Plugin Configuration Reference +--- +## Function Description +LLM Intent Recognition plugin can intelligently determine the alignment between user requests and the functionalities of a certain domain or agent, thereby enhancing the application effectiveness of different models and user experience. + +## Execution Attributes +Plugin execution phase: `Default Phase` + +Plugin execution priority: `700` + +## Configuration Instructions +> 1. This plugin's priority is higher than that of plugins such as ai-proxy which follow up and use intent. Subsequent plugins can retrieve the intent category using the proxywasm.GetProperty([]string{"intent_category"}) method and make selections for different cache libraries or large models based on the intent category. +> 2. A new Higress large model route needs to be created to allow this plugin to access the large model. For example: the route should use `/intent` as a prefix, the service should select the large model service, and the ai-proxy plugin should be enabled for this route. +> 3. A fixed-address service needs to be created (for example, intent-service), which points to 127.0.0.1:80 (i.e., the gateway instance and port). The ai-intent plugin requires this service for calling to access the newly added route. The service name corresponds to llm.proxyServiceName (a DNS type service can also be created to allow the plugin to access other large models). +> 4. If using a fixed-address service to call the gateway itself, 127.0.0.1 must be added to the gateway's access whitelist. + +| Name | Data Type | Requirement | Default Value | Description | +| -------------- | --------------- | ----------- | ------------- | --------------------------------------------------------------- | +| `scene.category` | string | Required | - | Preset scene categories, separated by "|", e.g.: "Finance|E-commerce|Law|Higress" | +| `scene.prompt` | string | Optional | You are a smart category recognition assistant responsible for determining which preset category a user’s question belongs to based on the question posed by the user and the preset categories, and returning the corresponding category. The user's question is: %s, the preset categories are %s, directly return a specific category; if not found, return 'NotFound'. | llm request prompt template | +| `llm.proxyServiceName` | string | Required | - | Newly created Higress service pointing to the large model (use the FQDN value from Higress) | +| `llm.proxyUrl` | string | Required | - | The full path to the large model route request address, which can be the gateway’s own address or the address of another large model (OpenAI protocol), for example: http://127.0.0.1:80/intent/compatible-mode/v1/chat/completions | +| `llm.proxyDomain` | string | Optional | Retrieved from proxyUrl | Domain of the large model service | +| `llm.proxyPort` | string | Optional | Retrieved from proxyUrl | Port number of the large model service | +| `llm.proxyApiKey` | string | Optional | - | API_KEY corresponding to the external large model service when using it | +| `llm.proxyModel` | string | Optional | qwen-long | Type of the large model | +| `llm.proxyTimeout` | number | Optional | 10000 | Timeout for calling the large model, unit ms, default: 10000ms | + +## Configuration Example +```yaml +scene: + category: "Finance|E-commerce|Law|Higress" + prompt: "You are a smart category recognition assistant responsible for determining which preset category a user's question belongs to based on the question posed by the user and the preset categories, and returning the corresponding category. The user's question is: '%s', the preset categories are '%s', directly return a specific category; if not found, return 'NotFound'." +llm: + proxyServiceName: "intent-service.static" + proxyUrl: "http://127.0.0.1:80/intent/compatible-mode/v1/chat/completions" + proxyDomain: "127.0.0.1" + proxyPort: "80" + proxyModel: "qwen-long" + proxyApiKey: "" + proxyTimeout: "10000" +``` diff --git a/plugins/wasm-go/extensions/ai-json-resp/README.md b/plugins/wasm-go/extensions/ai-json-resp/README.md index 9fc5a5eee5..7fd25b0063 100644 --- a/plugins/wasm-go/extensions/ai-json-resp/README.md +++ b/plugins/wasm-go/extensions/ai-json-resp/README.md @@ -1,15 +1,17 @@ -## 简介 - -**Note** - -> 需要数据面的proxy wasm版本大于等于0.2.100 -> - -> 编译时，需要带上版本的tag，例如：tinygo build -o main.wasm -scheduler=none -target=wasi -gc=custom -tags="custommalloc nottinygc_finalizer proxy_wasm_version_0_2_100" ./ +--- +title: AI JSON 格式化 +keywords: [ AI网关, AI JSON 格式化 ] +description: AI JSON 格式化插件配置参考 +--- +## 功能说明 LLM响应结构化插件，用于根据默认或用户配置的Json Schema对AI的响应进行结构化，以便后续插件处理。注意目前只支持 `非流式响应`。 +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`150` ### 配置说明 @@ -109,7 +111,6 @@ curl -X POST "http://localhost:8001/v1/chat/completions" \ ### 支持openai接口的AI服务 以qwen为例，基本配置如下： -Yaml格式配置如下 ```yaml serviceName: qwen serviceDomain: dashscope.aliyuncs.com @@ -133,70 +134,15 @@ jsonSchema: additionalProperties: false ``` -JSON 格式配置 -```json -{ - "serviceName": "qwen", - "serviceUrl": "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions", - "apiKey": "[Your API Key]", - "jsonSchema": { - "title": "ActionItemsSchema", - "type": "object", - "properties": { - "action_items": { - "type": "array", - "items": { - "type": "object", - "properties": { - "description": { - "type": "string", - "description": "Description of the action item." - }, - "due_date": { - "type": ["string", "null"], - "description": "Due date for the action item, can be null if not specified." - }, - "owner": { - "type": ["string", "null"], - "description": "Owner responsible for the action item, can be null if not specified." - } - }, - "required": ["description", "due_date", "owner"], - "additionalProperties": false - }, - "description": "List of action items from the meeting." - } - }, - "required": ["action_items"], - "additionalProperties": false - } -} -``` - ### 本地网关服务 -为了能复用已经配置好的服务，本插件也支持配置本地网关服务。例如，若网关已经配置好了[AI-proxy服务](../ai-proxy/README.md)，则可以直接配置如下： -1. 创建一个固定IP为127.0.0.1的服务，例如localservice.static -```yaml -- name: outbound|10000||localservice.static - connect_timeout: 30s - type: LOGICAL_DNS - dns_lookup_family: V4_ONLY - lb_policy: ROUND_ROBIN - load_assignment: - cluster_name: outbound|8001||localservice.static - endpoints: - - lb_endpoints: - - endpoint: - address: - socket_address: - address: 127.0.0.1 - port_value: 10000 -``` +为了能复用已经配置好的服务，本插件也支持配置本地网关服务。例如，若网关已经配置好了AI-proxy服务，则可以直接配置如下： +1. 创建一个固定IP地址为127.0.0.1:80的服务，例如localservice.static + 2. 配置文件中添加localservice.static的服务配置 ```yaml serviceName: localservice serviceDomain: 127.0.0.1 -servicePort: 10000 +servicePort: 80 ``` 3. 自动提取请求的Path，Header等信息 插件会自动提取请求的Path，Header等信息，从而避免对AI服务的重复配置。 diff --git a/plugins/wasm-go/extensions/ai-json-resp/README_EN.md b/plugins/wasm-go/extensions/ai-json-resp/README_EN.md new file mode 100644 index 0000000000..1fdb740d07 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-json-resp/README_EN.md @@ -0,0 +1,140 @@ +--- +title: AI JSON Formatting +keywords: [ AI Gateway, AI JSON Formatting ] +description: AI JSON Formatting plugin configuration reference +--- +## Function Description +LLM structured response plugin, used to structure AI responses according to the default or user-configured Json Schema for subsequent plugin processing. Note that only `non-streaming responses` are currently supported. + +## Running Attributes +Plugin execution phase: `default phase` +Plugin execution priority: `150` + +### Configuration Description +| Name | Type | Requirement | Default | **Description** | +| --- | --- | --- | --- | --- | +| serviceName | str | required | - | AI service or gateway service name that supports AI-Proxy | +| serviceDomain | str | optional | - | AI service or gateway service domain/IP address that supports AI-Proxy | +| servicePath | str | optional | '/v1/chat/completions' | AI service or gateway service base path that supports AI-Proxy | +| serviceUrl | str | optional | - | AI service or gateway service URL that supports AI-Proxy; the plugin will automatically extract domain and path to fill in the unconfigured serviceDomain or servicePath | +| servicePort | int | optional | 443 | Gateway service port | +| serviceTimeout | int | optional | 50000 | Default request timeout | +| maxRetry | int | optional | 3 | Number of retry attempts when the answer cannot be correctly extracted and formatted | +| contentPath | str | optional | "choices.0.message.content” | gpath path to extract the response result from the LLM answer | +| jsonSchema | str (json) | optional | - | The jsonSchema against which the request is validated; if empty, only valid Json format responses are returned | +| enableSwagger | bool | optional | false | Whether to enable the Swagger protocol for validation | +| enableOas3 | bool | optional | true | Whether to enable the Oas3 protocol for validation | +| enableContentDisposition | bool | optional | true | Whether to enable the Content-Disposition header; if enabled, the response header will include `Content-Disposition: attachment; filename="response.json"` | + +> For performance reasons, the maximum supported Json Schema depth is 6 by default. Json Schemas exceeding this depth will not be used to validate responses; the plugin will only check if the returned response is a valid Json format. + +### Request and Return Parameter Description +- **Request Parameters**: The request format for this plugin is the OpenAI request format, including the `model` and `messages` fields, where `model` is the AI model name and `messages` is a list of conversation messages, each containing `role` and `content` fields, with `role` being the message role and `content` being the message content. + ```json + { + "model": "gpt-4", + "messages": [ + {"role": "user", "content": "give me a api doc for add the variable x to x+5"} + ] + } + ``` + Other request parameters should refer to the corresponding documentation of the configured AI service or gateway service. + +- **Return Parameters**: + - Returns a `Json format response` that satisfies the constraints of the defined Json Schema. + - If no Json Schema is defined, returns a valid `Json format response`. + - If an internal error occurs, returns `{ "Code": 10XX, "Msg": "Error message" }`. + +## Request Example +```bash +curl -X POST "http://localhost:8001/v1/chat/completions" \ +-H "Content-Type: application/json" \ +-d '{ + "model": "gpt-4", + "messages": [ + {"role": "user", "content": "give me a api doc for add the variable x to x+5"} + ] +}' +``` + +## Return Example +### Normal Return +Under normal circumstances, the system should return JSON data validated by the JSON Schema. If no JSON Schema is configured, the system will return legally valid JSON data that complies with JSON standards. +```json +{ + "apiVersion": "1.0", + "request": { + "endpoint": "/add_to_five", + "method": "POST", + "port": 8080, + "headers": { + "Content-Type": "application/json" + }, + "body": { + "x": 7 + } + } +} +``` + +### Exception Return +In case of an error, the return status code is `500`, and the return content is a JSON format error message. It contains two fields: error code `Code` and error message `Msg`. +```json +{ + "Code": 1006, + "Msg": "retry count exceed max retry count" +} +``` + +### Error Code Description +| Error Code | Description | +| --- | --- | +| 1001 | The configured Json Schema is not in a valid Json format | +| 1002 | The configured Json Schema compilation failed; it is not a valid Json Schema format or depth exceeds jsonSchemaMaxDepth while rejectOnDepthExceeded is true | +| 1003 | Unable to extract valid Json from the response | +| 1004 | Response is an empty string | +| 1005 | Response does not conform to the Json Schema definition | +| 1006 | Retry count exceeds the maximum limit | +| 1007 | Unable to retrieve the response content; may be due to upstream service configuration errors or incorrect ContentPath path to get the content | +| 1008 | serviceDomain is empty; please note that either serviceDomain or serviceUrl cannot be empty at the same time | + +## Service Configuration Description +This plugin requires configuration of upstream services to support automatic retry mechanisms in case of exceptions. Supported configurations mainly include `AI services supporting OpenAI interfaces` or `local gateway services`. + +### AI Services Supporting OpenAI Interfaces +Taking Qwen as an example, the basic configuration is as follows: +```yaml +serviceName: qwen +serviceDomain: dashscope.aliyuncs.com +apiKey: [Your API Key] +servicePath: /compatible-mode/v1/chat/completions +jsonSchema: + title: ReasoningSchema + type: object + properties: + reasoning_steps: + type: array + items: + type: string + description: The reasoning steps leading to the final conclusion. + answer: + type: string + description: The final answer, taking into account the reasoning steps. + required: + - reasoning_steps + - answer + additionalProperties: false +``` + +### Local Gateway Services +To reuse already configured services, this plugin also supports configuring local gateway services. For example, if the gateway has already configured the AI-proxy service, it can be directly configured as follows: + +1. Create a service with a fixed IP address of 127.0.0.1:80, for example, localservice.static. +2. Add the service configuration for localservice.static in the configuration file. +```yaml +serviceName: localservice +serviceDomain: 127.0.0.1 +servicePort: 80 +``` +3. Automatically extract request Path, Header, and other information. +The plugin will automatically extract request Path, Header, and other information to avoid repetitive configuration for the AI service. diff --git a/plugins/wasm-go/extensions/ai-prompt-decorator/README.md b/plugins/wasm-go/extensions/ai-prompt-decorator/README.md index 4feb712521..17220f4ba2 100644 --- a/plugins/wasm-go/extensions/ai-prompt-decorator/README.md +++ b/plugins/wasm-go/extensions/ai-prompt-decorator/README.md @@ -1,7 +1,19 @@ -# 简介 -AI提示词装饰器插件，支持在LLM的请求前后插入prompt。 +--- +title: AI 提示词 +keywords: [ AI网关, AI提示词 ] +description: AI 提示词插件配置参考 +--- -# 配置说明 +## 功能说明 + +AI提示词插件，支持在LLM的请求前后插入prompt。 + +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`450` + +## 配置说明 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | |----------------|-----------------|------|-----|----------------------------------| @@ -15,7 +27,7 @@ message object 配置说明： | `role` | string | 必填 | - | 角色 | | `content` | string | 必填 | - | 消息 | -# 示例 +## 示例 配置示例如下： @@ -69,7 +81,7 @@ curl http://localhost/test \ ``` -# 基于geo-ip插件的能力，扩展AI提示词装饰器插件携带用户地理位置信息 +## 基于geo-ip插件的能力，扩展AI提示词装饰器插件携带用户地理位置信息 如果需要在LLM的请求前后加入用户地理位置信息，请确保同时开启geo-ip插件和AI提示词装饰器插件。并且在相同的请求处理阶段里，geo-ip插件的优先级必须高于AI提示词装饰器插件。首先geo-ip插件会根据用户ip计算出用户的地理位置信息，然后通过请求属性传递给后续插件。比如在默认阶段里，geo-ip插件的priority配置1000，ai-prompt-decorator插件的priority配置500。 geo-ip插件配置示例： diff --git a/plugins/wasm-go/extensions/ai-prompt-decorator/README_EN.md b/plugins/wasm-go/extensions/ai-prompt-decorator/README_EN.md new file mode 100644 index 0000000000..211209144a --- /dev/null +++ b/plugins/wasm-go/extensions/ai-prompt-decorator/README_EN.md @@ -0,0 +1,130 @@ +--- +title: AI Prompts +keywords: [ AI Gateway, AI Prompts ] +description: AI Prompts plugin configuration reference +--- +## Function Description +The AI Prompts plugin allows inserting prompts before and after requests in LLM. + +## Execution Properties +Plugin execution phase: `Default Phase` +Plugin execution priority: `450` + +## Configuration Description +| Name | Data Type | Requirement | Default Value | Description | +|---------------|----------------------|-------------|---------------|--------------------------------------| +| `prepend` | array of message object | optional | - | Statements inserted before the initial input | +| `append` | array of message object | optional | - | Statements inserted after the initial input | + +Message object configuration description: +| Name | Data Type | Requirement | Default Value | Description | +|-----------|-------------|-------------|---------------|-------------| +| `role` | string | required | - | Role | +| `content` | string | required | - | Message | + +## Example +An example configuration is as follows: +```yaml +prepend: +- role: system + content: "Please answer the questions in English." +append: +- role: user + content: "After answering each question, try to ask a follow-up question." +``` + +Using the above configuration to initiate a request: +```bash +curl http://localhost/test \ +-H "content-type: application/json" \ +-d '{ + "model": "gpt-3.5-turbo", + "messages": [ + { + "role": "user", + "content": "Who are you?" + } + ] +} +``` + +After processing through the plugin, the actual request will be: +```bash +curl http://localhost/test \ +-H "content-type: application/json" \ +-d '{ + "model": "gpt-3.5-turbo", + "messages": [ + { + "role": "system", + "content": "Please answer the questions in English." + }, + { + "role": "user", + "content": "Who are you?" + }, + { + "role": "user", + "content": "After answering each question, try to ask a follow-up question." + } + ] +} +``` + +## Based on the geo-ip plugin's capabilities, extend AI Prompt Decorator plugin to carry user geographic location information. +If you need to include user geographic location information before and after the LLM's requests, please ensure both the geo-ip plugin and the AI Prompt Decorator plugin are enabled. Moreover, in the same request processing phase, the geo-ip plugin's priority must be higher than that of the AI Prompt Decorator plugin. First, the geo-ip plugin will calculate the user's geographic location information based on the user's IP, and then pass it to subsequent plugins via request attributes. For instance, in the default phase, the geo-ip plugin's priority configuration is 1000, while the ai-prompt-decorator plugin's priority configuration is 500. + +Example configuration for the geo-ip plugin: +```yaml +ipProtocal: "ipv4" +``` + +An example configuration for the AI Prompt Decorator plugin is as follows: +```yaml +prepend: +- role: system + content: "The user's current geographic location is, country: ${geo-country}, province: ${geo-province}, city: ${geo-city}." +append: +- role: user + content: "After answering each question, try to ask a follow-up question." +``` + +Using the above configuration to initiate a request: +```bash +curl http://localhost/test \ +-H "content-type: application/json" \ +-H "x-forwarded-for: 87.254.207.100,4.5.6.7" \ +-d '{ + "model": "gpt-3.5-turbo", + "messages": [ + { + "role": "user", + "content": "How is the weather today?" + } + ] +}' +``` + +After processing through the plugin, the actual request will be: +```bash +curl http://localhost/test \ +-H "content-type: application/json" \ +-H "x-forwarded-for: 87.254.207.100,4.5.6.7" \ +-d '{ + "model": "gpt-3.5-turbo", + "messages": [ + { + "role": "system", + "content": "The user's current geographic location is, country: China, province: Beijing, city: Beijing." + }, + { + "role": "user", + "content": "How is the weather today?" + }, + { + "role": "user", + "content": "After answering each question, try to ask a follow-up question." + } + ] +}' +``` diff --git a/plugins/wasm-go/extensions/ai-prompt-template/README.md b/plugins/wasm-go/extensions/ai-prompt-template/README.md index 6fe6437e80..ab52f68814 100644 --- a/plugins/wasm-go/extensions/ai-prompt-template/README.md +++ b/plugins/wasm-go/extensions/ai-prompt-template/README.md @@ -1,7 +1,19 @@ -# 简介 +--- +title: AI 提示词模版 +keywords: [ AI网关, AI提示词模版 ] +description: AI 提示词模版配置参考 +--- + +## 功能说明 + AI提示词模板，用于快速构建同类型的AI请求。 -# 配置说明 +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`500` + +## 配置说明 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | |----------------|-----------------|------|-----|----------------------------------| | `templates` | array of object | 必填 | - | 模板设置 | @@ -45,4 +57,4 @@ templates: "language": "python" } } -``` \ No newline at end of file +``` diff --git a/plugins/wasm-go/extensions/ai-prompt-template/README_EN.md b/plugins/wasm-go/extensions/ai-prompt-template/README_EN.md new file mode 100644 index 0000000000..7df1041809 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-prompt-template/README_EN.md @@ -0,0 +1,53 @@ +--- +title: AI Prompt Template +keywords: [ AI Gateway, AI Prompt Template ] +description: AI Prompt Template Configuration Reference +--- +## Function Description +AI prompt templates are used to quickly build similar types of AI requests. + +## Execution Properties +Plugin Execution Phase: `Default Phase` +Plugin Execution Priority: `500` + +## Configuration Description +| Name | Data Type | Required | Default Value | Description | +|-----------------|-------------------|----------|---------------|-----------------------------------| +| `templates` | array of object | Required | - | Template settings | + +Template object configuration description: +| Name | Data Type | Required | Default Value | Description | +|-----------------------|-------------------|----------|---------------|-----------------------------------| +| `name` | string | Required | - | Template name | +| `template.model` | string | Required | - | Model name | +| `template.messages` | array of object | Required | - | Input for large model | + +Message object configuration description: +| Name | Data Type | Required | Default Value | Description | +|----------------|-------------------|----------|---------------|-----------------------------------| +| `role` | string | Required | - | Role | +| `content` | string | Required | - | Message | + +Configuration example: +```yaml +templates: +- name: "developer-chat" + template: + model: gpt-3.5-turbo + messages: + - role: system + content: "You are a {{program}} expert, in {{language}} programming language." + - role: user + content: "Write me a {{program}} program." +``` + +Example request body using the above configuration: +```json +{ + "template": "developer-chat", + "properties": { + "program": "quick sort", + "language": "python" + } +} +``` diff --git a/plugins/wasm-go/extensions/ai-proxy/README.md b/plugins/wasm-go/extensions/ai-proxy/README.md index 830e7db1d4..c2dc62c95e 100644 --- a/plugins/wasm-go/extensions/ai-proxy/README.md +++ b/plugins/wasm-go/extensions/ai-proxy/README.md @@ -15,6 +15,11 @@ description: AI 代理插件配置参考 > 请求路径后缀匹配 `/v1/embeddings` 时，对应文本向量场景，会用 OpenAI 的文本向量协议解析请求 Body，再转换为对应 LLM 厂商的文本向量协议 +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`100` + ## 配置字段 diff --git a/plugins/wasm-go/extensions/ai-proxy/README_EN.md b/plugins/wasm-go/extensions/ai-proxy/README_EN.md new file mode 100644 index 0000000000..a2b756a99a --- /dev/null +++ b/plugins/wasm-go/extensions/ai-proxy/README_EN.md @@ -0,0 +1,1487 @@ +--- +title: AI Proxy +keywords: [AI Gateway, AI Proxy] +description: Reference for configuring the AI Proxy plugin +--- + +## Function Description + +The `AI Proxy` plugin implements AI proxy functionality based on the OpenAI API contract. It currently supports AI service providers such as OpenAI, Azure OpenAI, Moonshot, and Qwen. + +> **Note:** + +> When the request path suffix matches `/v1/chat/completions`, it corresponds to text-to-text scenarios. The request body will be parsed using OpenAI's text-to-text protocol and then converted to the corresponding LLM vendor's text-to-text protocol. + +> When the request path suffix matches `/v1/embeddings`, it corresponds to text vector scenarios. The request body will be parsed using OpenAI's text vector protocol and then converted to the corresponding LLM vendor's text vector protocol. + +## Execution Properties +Plugin execution phase: `Default Phase` +Plugin execution priority: `100` + + +## Configuration Fields + +### Basic Configuration + +| Name | Data Type | Requirement | Default | Description | +|------------|--------|------|-----|------------------| +| `provider` | object | Required | - | Configures information for the target AI service provider | + +**Details for the `provider` configuration fields:** + +| Name | Data Type | Requirement | Default | Description | +| -------------- | --------------- | -------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `type` | string | Required | - | Name of the AI service provider | +| `apiTokens` | array of string | Optional | - | Tokens used for authentication when accessing AI services. If multiple tokens are configured, the plugin randomly selects one for each request. Some service providers only support configuring a single token. | +| `timeout` | number | Optional | - | Timeout for accessing AI services, in milliseconds. The default value is 120000, which equals 2 minutes. | +| `modelMapping` | map of string | Optional | - | Mapping table for AI models, used to map model names in requests to names supported by the service provider.
1. Supports prefix matching. For example, "gpt-3-*" matches all model names starting with “gpt-3-”;
2. Supports using "*" as a key for a general fallback mapping;
3. If the mapped target name is an empty string "", the original model name is preserved. | +| `protocol` | string | Optional | - | API contract provided by the plugin. Currently supports the following values: openai (default, uses OpenAI's interface contract), original (uses the raw interface contract of the target service provider) | +| `context` | object | Optional | - | Configuration for AI conversation context information | +| `customSettings` | array of customSetting | Optional | - | Specifies overrides or fills parameters for AI requests | + +**Details for the `context` configuration fields:** + +| Name | Data Type | Requirement | Default | Description | +|---------------|--------|------|-----|----------------------------------| +| `fileUrl` | string | Required | - | File URL to save AI conversation context. Only supports file content of plain text type | +| `serviceName` | string | Required | - | Full name of the Higress backend service corresponding to the URL | +| `servicePort` | number | Required | - | Port for accessing the Higress backend service corresponding to the URL | + +**Details for the `customSettings` configuration fields:** + +| Name | Data Type | Requirement | Default | Description | +| ----------- | --------------------- | -------- | ------ | ---------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | Required | - | Name of the parameter to set, e.g., `max_tokens` | +| `value` | string/int/float/bool | Required | - | Value of the parameter to set, e.g., 0 | +| `mode` | string | Optional | "auto" | Mode for setting the parameter, can be set to "auto" or "raw"; if "auto", the parameter name will be automatically rewritten based on the protocol; if "raw", no rewriting or restriction checks will be applied | +| `overwrite` | bool | Optional | true | If false, the parameter is only filled if the user has not set it; otherwise, it directly overrides the user's existing parameter settings | + +The `custom-setting` adheres to the following table, replacing the corresponding field based on `name` and protocol. Users need to fill in values from the `settingName` column that exists in the table. For instance, if a user sets `name` to `max_tokens`, in the openai protocol, it replaces `max_tokens`; for gemini, it replaces `maxOutputTokens`. `"none"` indicates that the protocol does not support this parameter. If `name` is not in this table or the corresponding protocol does not support the parameter, and "raw" mode is not set, the configuration will not take effect. + +| settingName | openai | baidu | spark | qwen | gemini | hunyuan | claude | minimax | +| ----------- | ----------- | ----------------- | ----------- | ----------- | --------------- | ----------- | ----------- | ------------------ | +| max_tokens | max_tokens | max_output_tokens | max_tokens | max_tokens | maxOutputTokens | none | max_tokens | tokens_to_generate | +| temperature | temperature | temperature | temperature | temperature | temperature | Temperature | temperature | temperature | +| top_p | top_p | top_p | none | top_p | topP | TopP | top_p | top_p | +| top_k | none | none | top_k | none | topK | none | top_k | none | +| seed | seed | none | none | seed | none | none | none | none | + +If raw mode is enabled, `custom-setting` will directly alter the JSON content using the input `name` and `value`, without any restrictions or modifications to the parameter names. +For most protocols, `custom-setting` modifies or fills parameters at the root path of the JSON content. For the `qwen` protocol, ai-proxy configures under the `parameters` subpath. For the `gemini` protocol, it configures under the `generation_config` subpath. + +### Provider-Specific Configurations + +#### OpenAI + +For OpenAI, the corresponding `type` is `openai`. Its unique configuration fields include: + +| Name | Data Type | Requirement | Default | Description | +|-------------------|----------|----------|--------|-------------------------------------------------------------------------------| +| `openaiCustomUrl` | string | Optional | - | Custom backend URL based on the OpenAI protocol, e.g., www.example.com/myai/v1/chat/completions | +| `responseJsonSchema` | object | Optional | - | Predefined Json Schema that OpenAI responses must adhere to; note that currently only a few specific models support this usage| + +#### Azure OpenAI + +For Azure OpenAI, the corresponding `type` is `azure`. Its unique configuration field is: + +| Name | Data Type | Filling Requirements | Default Value | Description | +|---------------------|-------------|----------------------|---------------|---------------------------------------------------------------------------------------------------------------| +| `azureServiceUrl` | string | Required | - | The URL of the Azure OpenAI service, must include the `api-version` query parameter. | + +**Note:** Azure OpenAI only supports configuring one API Token. + +#### Moonshot + +For Moonshot, the corresponding `type` is `moonshot`. Its unique configuration field is: + +| Name | Data Type | Filling Requirements | Default Value | Description | +|-------------------|-------------|----------------------|---------------|-----------------------------------------------------------------------------------------------------------------| +| `moonshotFileId` | string | Optional | - | The file ID uploaded via the file interface to Moonshot, whose content will be used as context for AI conversations. Cannot be configured with the `context` field. | + +#### Qwen (Tongyi Qwen) + +For Qwen (Tongyi Qwen), the corresponding `type` is `qwen`. Its unique configuration fields are: + +| Name | Data Type | Filling Requirements | Default Value | Description | +|--------------------|-----------------|----------------------|---------------|------------------------------------------------------------------------------------------------------------------------| +| `qwenEnableSearch` | boolean | Optional | - | Whether to enable the built-in Internet search function provided by Qwen. | +| `qwenFileIds` | array of string | Optional | - | The file IDs uploaded via the Dashscope file interface, whose content will be used as context for AI conversations. Cannot be configured with the `context` field. | + +#### Baichuan AI + +For Baichuan AI, the corresponding `type` is `baichuan`. It has no unique configuration fields. + +#### Yi (Zero One Universe) + +For Yi (Zero One Universe), the corresponding `type` is `yi`. It has no unique configuration fields. + +#### Zhipu AI + +For Zhipu AI, the corresponding `type` is `zhipuai`. It has no unique configuration fields. + +#### DeepSeek + +For DeepSeek, the corresponding `type` is `deepseek`. It has no unique configuration fields. + +#### Groq + +For Groq, the corresponding `type` is `groq`. It has no unique configuration fields. + +#### ERNIE Bot + +For ERNIE Bot, the corresponding `type` is `baidu`. It has no unique configuration fields. + +### 360 Brain + +For 360 Brain, the corresponding `type` is `ai360`. It has no unique configuration fields. + +### Mistral + +For Mistral, the corresponding `type` is `mistral`. It has no unique configuration fields. + +#### Minimax + +For Minimax, the corresponding `type` is `minimax`. Its unique configuration field is: + +| Name | Data Type | Filling Requirements | Default Value | Description | +| ---------------- | -------- | --------------------- |---------------|------------------------------------------------------------------------------------------------------------| +| `minimaxGroupId` | string | Required when using models `abab6.5-chat`, `abab6.5s-chat`, `abab5.5s-chat`, `abab5.5-chat` | - | When using models `abab6.5-chat`, `abab6.5s-chat`, `abab5.5s-chat`, `abab5.5-chat`, Minimax uses ChatCompletion Pro and requires setting the groupID. | + +#### Anthropic Claude + +For Anthropic Claude, the corresponding `type` is `claude`. Its unique configuration field is: + +| Name | Data Type | Filling Requirements | Default Value | Description | +|------------|-------------|----------------------|---------------|---------------------------------------------------------------------------------------------------------------| +| `claudeVersion` | string | Optional | - | The version of the Claude service's API, default is 2023-06-01. | + +#### Ollama + +For Ollama, the corresponding `type` is `ollama`. Its unique configuration field is: + +| Name | Data Type | Filling Requirements | Default Value | Description | +|-------------------|-------------|----------------------|---------------|---------------------------------------------------------------------------------------------------------| +| `ollamaServerHost` | string | Required | - | The host address of the Ollama server. | +| `ollamaServerPort` | number | Required | - | The port number of the Ollama server, defaults to 11434. | + +#### Hunyuan + +For Hunyuan, the corresponding `type` is `hunyuan`. Its unique configuration fields are: + +| Name | Data Type | Filling Requirements | Default Value | Description | +|-------------------|-------------|----------------------|---------------|---------------------------------------------------------------------------------------------------------| +| `hunyuanAuthId` | string | Required | - | Hunyuan authentication ID for version 3 authentication. | +| `hunyuanAuthKey` | string | Required | - | Hunyuan authentication key for version 3 authentication. | + +#### Stepfun + +For Stepfun, the corresponding `type` is `stepfun`. It has no unique configuration fields. + +#### Cloudflare Workers AI + +For Cloudflare Workers AI, the corresponding `type` is `cloudflare`. Its unique configuration field is: + +| Name | Data Type | Filling Requirements | Default Value | Description | +|-------------------|-------------|----------------------|---------------|---------------------------------------------------------------------------------------------------------| +| `cloudflareAccountId` | string | Required | - | [Cloudflare Account ID](https://developers.cloudflare.com/workers-ai/get-started/rest-api/#1-get-api-token-and-account-id). | + +#### Spark + +For Spark, the corresponding `type` is `spark`. It has no unique configuration fields. + +The `apiTokens` field value for Xunfei Spark (Xunfei Star) is `APIKey:APISecret`. That is, enter your own APIKey and APISecret, separated by `:`. + +#### Gemini + +For Gemini, the corresponding `type` is `gemini`. Its unique configuration field is: + +| Name | Data Type | Filling Requirements | Default Value | Description | +|---------------------|----------|----------------------|---------------|---------------------------------------------------------------------------------------------------------| +| `geminiSafetySetting` | map of string | Optional | - | Gemini AI content filtering and safety level settings. Refer to [Safety settings](https://ai.google.dev/gemini-api/docs/safety-settings). | + +### DeepL + +For DeepL, the corresponding `type` is `deepl`. Its unique configuration field is: + +| Name | Data Type | Requirement | Default | Description | +| ------------ | --------- | ----------- | ------- | ------------------------------------ | +| `targetLang` | string | Required | - | The target language required by the DeepL translation service | + +## Usage Examples + +### Using OpenAI Protocol Proxy for Azure OpenAI Service + +Using the basic Azure OpenAI service without configuring any context. + +**Configuration Information** + +```yaml +provider: + type: azure + apiTokens: + - "YOUR_AZURE_OPENAI_API_TOKEN" + azureServiceUrl: "https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2024-02-15-preview", +``` + +**Request Example** + +```json +{ + "model": "gpt-3", + "messages": [ + { + "role": "user", + "content": "Hello, who are you?" + } + ], + "temperature": 0.3 +} +``` + +**Response Example** + +```json +{ + "choices": [ + { + "content_filter_results": { + "hate": { + "filtered": false, + "severity": "safe" + }, + "self_harm": { + "filtered": false, + "severity": "safe" + }, + "sexual": { + "filtered": false, + "severity": "safe" + }, + "violence": { + "filtered": false, + "severity": "safe" + } + }, + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "Hello! I am an AI assistant, here to answer your questions and provide assistance. Is there anything I can help you with?", + "role": "assistant" + } + } + ], + "created": 1714807624, + "id": "chatcmpl-abcdefg1234567890", + "model": "gpt-35-turbo-16k", + "object": "chat.completion", + "prompt_filter_results": [ + { + "prompt_index": 0, + "content_filter_results": { + "hate": { + "filtered": false, + "severity": "safe" + }, + "self_harm": { + "filtered": false, + "severity": "safe" + }, + "sexual": { + "filtered": false, + "severity": "safe" + }, + "violence": { + "filtered": false, + "severity": "safe" + } + } + } + ], + "system_fingerprint": null, + "usage": { + "completion_tokens": 40, + "prompt_tokens": 15, + "total_tokens": 55 + } +} +``` + +### Using OpenAI Protocol Proxy for Qwen Service + +Using Qwen service and configuring the mapping relationship between OpenAI large models and Qwen models. + +**Configuration Information** + +```yaml +provider: + type: qwen + apiTokens: + - "YOUR_QWEN_API_TOKEN" + modelMapping: + 'gpt-3': "qwen-turbo" + 'gpt-35-turbo': "qwen-plus" + 'gpt-4-turbo': "qwen-max" + 'gpt-4-*': "qwen-max" + 'gpt-4o': "qwen-vl-plus" + 'text-embedding-v1': 'text-embedding-v1' + '*': "qwen-turbo" +``` + +**AI Conversation Request Example** + +URL: http://your-domain/v1/chat/completions + +Request Example: + +```json +{ + "model": "gpt-3", + "messages": [ + { + "role": "user", + "content": "Hello, who are you?" + } + ], + "temperature": 0.3 +} +``` + +Response Example: + +```json +{ + "id": "c2518bd3-0f46-97d1-be34-bb5777cb3108", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "I am Qwen, an AI assistant developed by Alibaba Cloud. I can answer various questions, provide information, and engage in conversations with users. How can I assist you?" + }, + "finish_reason": "stop" + } + ], + "created": 1715175072, + "model": "qwen-turbo", + "object": "chat.completion", + "usage": { + "prompt_tokens": 24, + "completion_tokens": 33, + "total_tokens": 57 + } +} +``` + +**Multimodal Model API Request Example (Applicable to `qwen-vl-plus` and `qwen-vl-max` Models)** + +URL: http://your-domain/v1/chat/completions + +Request Example: + +```json +{ + "model": "gpt-4o", + "messages": [ + { + "role": "user", + "content": [ + { + "type": "image_url", + "image_url": { + "url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg" + } + }, + { + "type": "text", + "text": "Where is this picture from?" + } + ] + } + ], + "temperature": 0.3 +} +``` + +Response Example: + +```json +{ + "id": "17c5955d-af9c-9f28-bbde-293a9c9a3515", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": [ + { + "text": "This photo depicts a woman and a dog on a beach. As I cannot access specific geographical information, I cannot pinpoint the exact location of this beach. However, visually, it appears to be a sandy coastline along a coastal area with waves breaking on the shore. Such scenes can be found in many beautiful seaside locations worldwide. If you need more precise information, please provide additional context or descriptive details." + } + ] + }, + "finish_reason": "stop" + } + ], + "created": 1723949230, + "model": "qwen-vl-plus", + "object": "chat.completion", + "usage": { + "prompt_tokens": 1279, + "completion_tokens": 78 + } +} +``` + +**Text Embedding Request Example** + +URL: http://your-domain/v1/embeddings + +Request Example: + +```json +{ + "model": "text-embedding-v1", + "input": "Hello" +} +``` + +Response Example: + +```json +{ + "object": "list", + "data": [ + { + "object": "embedding", + "index": 0, + "embedding": [ + -1.0437825918197632, + 5.208984375, + 3.0483806133270264, + -1.7897135019302368, + -2.0107421875, + ..., + 0.8125, + -1.1759847402572632, + 0.8174641728401184, + 1.0432943105697632, + -0.5885213017463684 + ] + } + ], + "model": "text-embedding-v1", + "usage": { + "prompt_tokens": 1, + "total_tokens": 1 + } +} +``` + +### Using Qwen Service with Pure Text Context Information + +Using Qwen service while configuring pure text context information. + +**Configuration Information** + +```yaml +provider: + type: qwen + apiTokens: + - "YOUR_QWEN_API_TOKEN" + modelMapping: + "*": "qwen-turbo" + context: + - fileUrl: "http://file.default.svc.cluster.local/ai/context.txt", + serviceName: "file.dns", + servicePort: 80 +``` + +**Request Example** + +```json +{ + "model": "gpt-3", + "messages": [ + { + "role": "user", + "content": "Please summarize the content" + } + ], + "temperature": 0.3 +} +``` + +**Response Example** + +```json +{ + "id": "cmpl-77861a17681f4987ab8270dbf8001936", + "object": "chat.completion", + "created": 9756990, + "model": "moonshot-v1-128k", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "The content of this document is about..." + }, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 20181, + "completion_tokens": 439, + "total_tokens": 20620 + } +} +``` + +### Using Qwen Service with Native File Context + +Uploading files to Qwen in advance to use them as context when utilizing its AI service. + +**Configuration Information** + +```yaml +provider: + type: qwen + apiTokens: + - "YOUR_QWEN_API_TOKEN" + modelMapping: + "*": "qwen-long" # Qwen's file context can only be used in the qwen-long model + qwenFileIds: + - "file-fe-xxx" + - "file-fe-yyy" +``` + +**Request Example** + +```json +{ + "model": "gpt-4-turbo", + "messages": [ + { + "role": "user", + "content": "Please summarize the content" + } + ], + "temperature": 0.3 +} +``` + +**Response Example** + +```json +{ + "output": { + "choices": [ + { + "finish_reason": "stop", + "message": { + "role": "assistant", + "content": "You uploaded two files, `context.txt` and `context_2.txt`, which seem to contain information about..." + } + } + ] + }, + "usage": { + "total_tokens": 2023, + "output_tokens": 530, + "input_tokens": 1493 + }, + "request_id": "187e99ba-5b64-9ffe-8f69-01dafbaf6ed7" +} +``` +### Utilizing Moonshot with its Native File Context + +Upload files to Moonshot in advance and use its AI services based on file content. + +**Configuration Information** + +```yaml +provider: + type: moonshot + apiTokens: + - "YOUR_MOONSHOT_API_TOKEN" + moonshotFileId: "YOUR_MOONSHOT_FILE_ID", + modelMapping: + '*': "moonshot-v1-32k" +``` + +**Example Request** + +```json +{ + "model": "gpt-4-turbo", + "messages": [ + { + "role": "user", + "content": "Please summarize the content" + } + ], + "temperature": 0.3 +} +``` + +**Example Response** + +```json +{ + "id": "cmpl-e5ca873642ca4f5d8b178c1742f9a8e8", + "object": "chat.completion", + "created": 1872961, + "model": "moonshot-v1-128k", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "The content of the text is about a payment platform named ‘xxxx’..." + }, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 11, + "completion_tokens": 498, + "total_tokens": 509 + } +} +``` + +### Using OpenAI Protocol Proxy for Groq Service + +**Configuration Information** + +```yaml +provider: + type: groq + apiTokens: + - "YOUR_GROQ_API_TOKEN" +``` + +**Example Request** + +```json +{ + "model": "llama3-8b-8192", + "messages": [ + { + "role": "user", + "content": "Hello, who are you?" + } + ] +} +``` + +**Example Response** + +```json +{ + "id": "chatcmpl-26733989-6c52-4056-b7a9-5da791bd7102", + "object": "chat.completion", + "created": 1715917967, + "model": "llama3-8b-8192", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "😊 Ni Hao! (That's \"hello\" in Chinese!)\n\nI am LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner. I'm not a human, but a computer program designed to simulate conversations and answer questions to the best of my ability. I'm happy to chat with you in Chinese or help with any questions or topics you'd like to discuss! 😊" + }, + "logprobs": null, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 16, + "prompt_time": 0.005, + "completion_tokens": 89, + "completion_time": 0.104, + "total_tokens": 105, + "total_time": 0.109 + }, + "system_fingerprint": "fp_dadc9d6142", + "x_groq": { + "id": "req_01hy2awmcxfpwbq56qh6svm7qz" + } +} +``` + +### Using OpenAI Protocol Proxy for Claude Service + +**Configuration Information** + +```yaml +provider: + type: claude + apiTokens: + - "YOUR_CLAUDE_API_TOKEN" + version: "2023-06-01" +``` + +**Example Request** + +```json +{ + "model": "claude-3-opus-20240229", + "max_tokens": 1024, + "messages": [ + { + "role": "user", + "content": "Hello, who are you?" + } + ] +} +``` + +**Example Response** + +```json +{ + "id": "msg_01Jt3GzyjuzymnxmZERJguLK", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "Hello, I am a conversation system developed by Anthropic, a company specializing in artificial intelligence. My name is Claude, a friendly and knowledgeable chatbot. Nice to meet you! I can engage in discussions on various topics, answer questions, provide suggestions, and assist you. I'll do my best to give you helpful responses. I hope we have a pleasant exchange!" + }, + "finish_reason": "stop" + } + ], + "created": 1717385918, + "model": "claude-3-opus-20240229", + "object": "chat.completion", + "usage": { + "prompt_tokens": 16, + "completion_tokens": 126, + "total_tokens": 142 + } +} +``` + +### Using OpenAI Protocol Proxy for Hunyuan Service + +**Configuration Information** + +```yaml +provider: + type: "hunyuan" + hunyuanAuthKey: "" + apiTokens: + - "" + hunyuanAuthId: "" + timeout: 1200000 + modelMapping: + "*": "hunyuan-lite" +``` + +**Example Request** + +Request script: + +```sh + +curl --location 'http:///v1/chat/completions' \ +--header 'Content-Type: application/json' \ +--data '{ + "model": "gpt-3", + "messages": [ + { + "role": "system", + "content": "You are a professional developer!" + }, + { + "role": "user", + "content": "Hello, who are you?" + } + ], + "temperature": 0.3, + "stream": false +}' +``` + +**Example Response** + +```json +{ + "id": "fd140c3e-0b69-4b19-849b-d354d32a6162", + "choices": [ + { + "index": 0, + "delta": { + "role": "assistant", + "content": "Hello! I am a professional developer." + }, + "finish_reason": "stop" + } + ], + "created": 1717493117, + "model": "hunyuan-lite", + "object": "chat.completion", + "usage": { + "prompt_tokens": 15, + "completion_tokens": 9, + "total_tokens": 24 + } +} +``` + +### Using OpenAI Protocol Proxy for ERNIE Bot Service + +**Configuration Information** + +```yaml +provider: + type: baidu + apiTokens: + - "YOUR_BAIDU_API_TOKEN" + modelMapping: + 'gpt-3': "ERNIE-4.0" + '*': "ERNIE-4.0" +``` + +**Request Example** + +```json +{ + "model": "gpt-4-turbo", + "messages": [ + { + "role": "user", + "content": "Hello, who are you?" + } + ], + "stream": false +} +``` + +**Response Example** + +```json +{ + "id": "as-e90yfg1pk1", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "Hello, I am ERNIE Bot. I can interact with people, answer questions, assist in creation, and efficiently provide information, knowledge, and inspiration." + }, + "finish_reason": "stop" + } + ], + "created": 1717251488, + "model": "ERNIE-4.0", + "object": "chat.completion", + "usage": { + "prompt_tokens": 4, + "completion_tokens": 33, + "total_tokens": 37 + } +} +``` + +### Using OpenAI Protocol Proxy for MiniMax Service + +**Configuration Information** + +```yaml +provider: + type: minimax + apiTokens: + - "YOUR_MINIMAX_API_TOKEN" + modelMapping: + "gpt-3": "abab6.5g-chat" + "gpt-4": "abab6.5-chat" + "*": "abab6.5g-chat" + minimaxGroupId: "YOUR_MINIMAX_GROUP_ID" +``` + +**Request Example** + +```json +{ + "model": "gpt-4-turbo", + "messages": [ + { + "role": "user", + "content": "Hello, who are you?" + } + ], + "stream": false +} +``` + +**Response Example** + +```json +{ + "id": "02b2251f8c6c09d68c1743f07c72afd7", + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "Hello! I am MM Intelligent Assistant, a large language model developed by MiniMax. I can help answer questions, provide information, and engage in conversations. How can I assist you?", + "role": "assistant" + } + } + ], + "created": 1717760544, + "model": "abab6.5s-chat", + "object": "chat.completion", + "usage": { + "total_tokens": 106 + }, + "input_sensitive": false, + "output_sensitive": false, + "input_sensitive_type": 0, + "output_sensitive_type": 0, + "base_resp": { + "status_code": 0, + "status_msg": "" + } +} +``` + +### Using OpenAI Protocol Proxy for 360 Brain Services + +**Configuration Information** + +```yaml +provider: + type: ai360 + apiTokens: + - "YOUR_MINIMAX_API_TOKEN" + modelMapping: + "gpt-4o": "360gpt-turbo-responsibility-8k" + "gpt-4": "360gpt2-pro" + "gpt-3.5": "360gpt-turbo" + "text-embedding-3-small": "embedding_s1_v1.2" + "*": "360gpt-pro" +``` + +**Request Example** + +```json +{ + "model": "gpt-4o", + "messages": [ + { + "role": "system", + "content": "You are a professional developer!" + }, + { + "role": "user", + "content": "Hello, who are you?" + } + ] +} +``` + +**Response Example** + +```json +{ + "choices": [ + { + "message": { + "role": "assistant", + "content": "Hello, I am 360 Brain, a large language model. I can assist with answering various questions, providing information, engaging in conversations, and more. How can I assist you?" + }, + "finish_reason": "", + "index": 0 + } + ], + "created": 1724257207, + "id": "5e5c94a2-d989-40b5-9965-5b971db941fe", + "model": "360gpt-turbo", + "object": "", + "usage": { + "completion_tokens": 33, + "prompt_tokens": 24, + "total_tokens": 57 + }, + "messages": [ + { + "role": "system", + "content": "You are a professional developer!" + }, + { + "role": "user", + "content": "Hello, who are you?" + } + ], + "context": null +} +``` + +**Text Embedding Request Example** + +**URL**: http://your-domain/v1/embeddings + +**Request Example** + +```json +{ + "input":["Hello"], + "model":"text-embedding-3-small" +} +``` + +**Response Example** + +```json +{ + "data": [ + { + "embedding": [ + -0.011237, + -0.015433, + ..., + -0.028946, + -0.052778, + 0.003768, + -0.007917, + -0.042201 + ], + "index": 0, + "object": "" + } + ], + "model": "embedding_s1_v1.2", + "object": "", + "usage": { + "prompt_tokens": 2, + "total_tokens": 2 + } +} +``` + +### Using OpenAI Protocol Proxy for Cloudflare Workers AI Service + +**Configuration Information** + +```yaml +provider: + type: cloudflare + apiTokens: + - "YOUR_WORKERS_AI_API_TOKEN" + cloudflareAccountId: "YOUR_CLOUDFLARE_ACCOUNT_ID" + modelMapping: + "*": "@cf/meta/llama-3-8b-instruct" +``` + +**Request Example** + +```json +{ + "model": "gpt-3.5", + "max_tokens": 1024, + "messages": [ + { + "role": "user", + "content": "Who are you?" + } + ] +} +``` + +**Response Example** + +```json +{ + "id": "id-1720367803430", + "object": "chat.completion", + "created": 1720367803, + "model": "@cf/meta/llama-3-8b-instruct", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "I am LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner. I'm not a human, but a computer program designed to simulate conversation and answer questions to the best of my knowledge. I can be used to generate text on a wide range of topics, from science and history to entertainment and culture." + }, + "logprobs": null, + "finish_reason": "stop" + } + ] +} +``` + +### Using OpenAI Protocol Proxy for Spark Service + +**Configuration Information** + +```yaml +provider: + type: spark + apiTokens: + - "APIKey:APISecret" + modelMapping: + "gpt-4o": "generalv3.5" + "gpt-4": "generalv3" + "*": "general" +``` + +**Request Example** + +```json +{ + "model": "gpt-4o", + "messages": [ + { + "role": "system", + "content": "You are a professional developer!" + }, + { + "role": "user", + "content": "Hello, who are you?" + } + ], + "stream": false +} +``` + +**Response Example** + +```json +{ + "id": "cha000c23c6@dx190ef0b4b96b8f2532", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "Hello! I am a professional developer skilled in programming and problem-solving. What can I assist you with?" + } + } + ], + "created": 1721997415, + "model": "generalv3.5", + "object": "chat.completion", + "usage": { + "prompt_tokens": 10, + "completion_tokens": 19, + "total_tokens": 29 + } +} +``` + +### Utilizing OpenAI Protocol Proxy for Gemini Services + +**Configuration Information** + +```yaml +provider: + type: gemini + apiTokens: + - "YOUR_GEMINI_API_TOKEN" + modelMapping: + "*": "gemini-pro" + geminiSafetySetting: + "HARM_CATEGORY_SEXUALLY_EXPLICIT" :"BLOCK_NONE" + "HARM_CATEGORY_HATE_SPEECH" :"BLOCK_NONE" + "HARM_CATEGORY_HARASSMENT" :"BLOCK_NONE" + "HARM_CATEGORY_DANGEROUS_CONTENT" :"BLOCK_NONE" +``` + +**Request Example** + +```json +{ + "model": "gpt-3.5", + "messages": [ + { + "role": "user", + "content": "Who are you?" + } + ], + "stream": false +} +``` + +**Response Example** + +```json +{ + "id": "chatcmpl-b010867c-0d3f-40ba-95fd-4e8030551aeb", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "I am a large multi-modal model, trained by Google. I am designed to provide information and answer questions to the best of my abilities." + }, + "finish_reason": "stop" + } + ], + "created": 1722756984, + "model": "gemini-pro", + "object": "chat.completion", + "usage": { + "prompt_tokens": 5, + "completion_tokens": 29, + "total_tokens": 34 + } +} +``` + +### Utilizing OpenAI Protocol Proxy for DeepL Text Translation Service + +**Configuration Information** + +```yaml +provider: + type: deepl + apiTokens: + - "YOUR_DEEPL_API_TOKEN" + targetLang: "ZH" +``` + +**Request Example** +Here, `model` denotes the service tier of DeepL and can only be either `Free` or `Pro`. The `content` field contains the text to be translated; within `role: system`, `content` may include context that influences the translation but isn't translated itself. For instance, when translating product names, including a product description as context could enhance translation quality. + +```json +{ + "model": "Free", + "messages": [ + { + "role": "system", + "content": "money" + }, + { + "content": "sit by the bank" + }, + { + "content": "a bank in China" + } + ] +} +``` + +**Response Example** +```json +{ + "choices": [ + { + "index": 0, + "message": { "name": "EN", "role": "assistant", "content": "operate a gambling establishment" } + }, + { + "index": 1, + "message": { "name": "EN", "role": "assistant", "content": "Bank of China" } + } + ], + "created": 1722747752, + "model": "Free", + "object": "chat.completion", + "usage": {} +} +``` + +## Full Configuration Example + +### Kubernetes Example + +Here's a full plugin configuration example using the OpenAI protocol proxy for Groq services. + +```yaml +apiVersion: extensions.higress.io/v1alpha1 +kind: WasmPlugin +metadata: + name: ai-proxy-groq + namespace: higress-system +spec: + matchRules: + - config: + provider: + type: groq + apiTokens: + - "YOUR_API_TOKEN" + ingress: + - groq + url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-proxy:1.0.0 +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + annotations: + higress.io/backend-protocol: HTTPS + higress.io/destination: groq.dns + higress.io/proxy-ssl-name: api.groq.com + higress.io/proxy-ssl-server-name: "on" + labels: + higress.io/resource-definer: higress + name: groq + namespace: higress-system +spec: + ingressClassName: higress + rules: + - host: + http: + paths: + - backend: + resource: + apiGroup: networking.higress.io + kind: McpBridge + name: default + path: / + pathType: Prefix +--- +apiVersion: networking.higress.io/v1 +kind: McpBridge +metadata: + name: default + namespace: higress-system +spec: + registries: + - domain: api.groq.com + name: groq + port: 443 + type: dns +``` + +Access Example: + +```bash +curl "http:///v1/chat/completions" -H "Content-Type: application/json" -d '{ + "model": "llama3-8b-8192", + "messages": [ + { + "role": "user", + "content": "hello, who are you?" + } + ] +}' +``` + +### Docker-Compose Example + +`docker-compose.yml` configuration file: + +```yaml +version: '3.7' +services: + envoy: + image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/envoy:1.20 + entrypoint: /usr/local/bin/envoy + # Enables debug level logging for easier debugging + command: -c /etc/envoy/envoy.yaml --component-log-level wasm:debug + networks: + - higress-net + ports: + - "10000:10000" + volumes: + - ./envoy.yaml:/etc/envoy/envoy.yaml + - ./plugin.wasm:/etc/envoy/plugin.wasm +networks: + higress-net: {} +``` + +`envoy.yaml` configuration file: + +```yaml +admin: + address: + socket_address: + protocol: TCP + address: 0.0.0.0 + port_value: 9901 +static_resources: + listeners: + - name: listener_0 + address: + socket_address: + protocol: TCP + address: 0.0.0.0 + port_value: 10000 + filter_chains: + - filters: + - name: envoy.filters.network.http_connection_manager + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager + scheme_header_transformation: + scheme_to_overwrite: https + stat_prefix: ingress_http + # Outputs envoy logs to stdout + access_log: + - name: envoy.access_loggers.stdout + typed_config: + "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog + # Modify as needed + route_config: + name: local_route + virtual_hosts: + - name: local_service + domains: [ "*" ] + routes: + - match: + prefix: "/" + route: + cluster: claude + timeout: 300s + http_filters: + - name: claude + typed_config: + "@type": type.googleapis.com/udpa.type.v1.TypedStruct + type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm + value: + config: + name: claude + vm_config: + runtime: envoy.wasm.runtime.v8 + code: + local: + filename: /etc/envoy/plugin.wasm + configuration: + "@type": "type.googleapis.com/google.protobuf.StringValue" + value: | # Plugin configuration + { + "provider": { + "type": "claude", + "apiTokens": [ + "YOUR_API_TOKEN" + ] + } + } + - name: envoy.filters.http.router + clusters: + - name: claude + connect_timeout: 30s + type: LOGICAL_DNS + dns_lookup_family: V4_ONLY + lb_policy: ROUND_ROBIN + load_assignment: + cluster_name: claude + endpoints: + - lb_endpoints: + - endpoint: + address: + socket_address: + address: api.anthropic.com # Service address + port_value: 443 + transport_socket: + name: envoy.transport_sockets.tls + typed_config: + "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext + "sni": "api.anthropic.com" +``` + +Access Example: + +```bash +curl "http://localhost:10000/v1/chat/completions" -H "Content-Type: application/json" -d '{ + "model": "claude-3-opus-20240229", + "max_tokens": 1024, + "messages": [ + { + "role": "user", + "content": "hello, who are you?" + } + ] +}' +``` diff --git a/plugins/wasm-go/extensions/ai-quota/README.md b/plugins/wasm-go/extensions/ai-quota/README.md index 11ddf80359..71b45b610a 100644 --- a/plugins/wasm-go/extensions/ai-quota/README.md +++ b/plugins/wasm-go/extensions/ai-quota/README.md @@ -1,10 +1,21 @@ -# 功能说明 +--- +title: AI 配额管理 +keywords: [ AI网关, AI配额 ] +description: AI 配额管理插件配置参考 +--- + +## 功能说明 `ai-qutoa` 插件实现给特定 consumer 根据分配固定的 quota 进行 quota 策略限流，同时支持 quota 管理能力，包括查询 quota 、刷新 quota、增减 quota。 `ai-quota` 插件需要配合 认证插件比如 `key-auth`、`jwt-auth` 等插件获取认证身份的 consumer 名称，同时需要配合 `ai-statatistics` 插件获取 AI Token 统计信息。 -# 配置说明 +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`750` + +## 配置说明 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | |--------------------|-----------------|--------------------------------------| ---- |--------------------------------------------| @@ -25,9 +36,9 @@ -# 配置示例 +## 配置示例 -## 识别请求参数 apikey，进行区别限流 +### 识别请求参数 apikey，进行区别限流 ```yaml redis_key_prefix: "chat_quota:" admin_consumer: consumer3 @@ -39,19 +50,19 @@ redis: ``` -## 刷新 quota +### 刷新 quota 如果当前请求 url 的后缀符合 admin_path，例如插件在 example.com/v1/chat/completions 这个路由上生效，那么更新 quota 可以通过 curl https://example.com/v1/chat/completions/quota/refresh -H "Authorization: Bearer credential3" -d "consumer=consumer1"a=10000" Redis 中 key 为 chat_quota:consumer1 的值就会被刷新为 10000 -## 查询 quota +### 查询 quota 查询特定用户的 quota 可以通过 curl https://example.com/v1/chat/completions/quota?consumer=consumer1 -H "Authorization: Bearer credential3" 将返回： {"quota": 10000, "consumer": "consumer1"} -## 增减 quota +### 增减 quota 增减特定用户的 quota 可以通过 curl https://example.com/v1/chat/completions/quota/delta -d "consumer=consumer1&value=100" -H "Authorization: Bearer credential3" 这样 Redis 中 Key 为 chat_quota:consumer1 的值就会增加100，可以支持负数，则减去对应值。 diff --git a/plugins/wasm-go/extensions/ai-quota/README_EN.md b/plugins/wasm-go/extensions/ai-quota/README_EN.md new file mode 100644 index 0000000000..e136a75969 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-quota/README_EN.md @@ -0,0 +1,54 @@ +--- +title: AI Quota Management +keywords: [ AI Gateway, AI Quota ] +description: AI quota management plugin configuration reference +--- +## Function Description +The `ai-quota` plugin implements quota rate limiting based on fixed quotas allocated to specific consumers. It also supports quota management capabilities, including querying quotas, refreshing quotas, and increasing or decreasing quotas. The `ai-quota` plugin needs to work with authentication plugins such as `key-auth`, `jwt-auth`, etc., to obtain the consumer name associated with the authenticated identity, and it needs to work with the `ai-statistics` plugin to obtain AI Token statistical information. + +## Runtime Properties +Plugin execution phase: `default phase` +Plugin execution priority: `750` + +## Configuration Description +| Name | Data Type | Required Conditions | Default Value | Description | +|---------------------|------------------|--------------------------------------------|---------------|---------------------------------------------------| +| `redis_key_prefix` | string | Optional | chat_quota: | Quota redis key prefix | +| `admin_consumer` | string | Required | | Consumer name for managing quota management identity | +| `admin_path` | string | Optional | /quota | Prefix for the path to manage quota requests | +| `redis` | object | Yes | | Redis related configuration | +Explanation of each configuration field in `redis` +| Configuration Item | Type | Required | Default Value | Explanation | +|---------------------|------------------|----------|---------------------------------------------------------|-----------------------------------------------| +| service_name | string | Required | - | Redis service name, full FQDN name with service type, e.g., my-redis.dns, redis.my-ns.svc.cluster.local | +| service_port | int | No | Default value for static service is 80; others are 6379 | Service port for the redis service | +| username | string | No | - | Redis username | +| password | string | No | - | Redis password | +| timeout | int | No | 1000 | Redis connection timeout in milliseconds | + +## Configuration Example +### Identify request parameter apikey and apply rate limiting accordingly +```yaml +redis_key_prefix: "chat_quota:" +admin_consumer: consumer3 +admin_path: /quota +redis: + service_name: redis-service.default.svc.cluster.local + service_port: 6379 + timeout: 2000 +``` + +### Refresh Quota +If the suffix of the current request URL matches the admin_path, for example, if the plugin is effective on the route example.com/v1/chat/completions, then the quota can be updated via: +curl https://example.com/v1/chat/completions/quota/refresh -H "Authorization: Bearer credential3" -d "consumer=consumer1"a=10000" +The value of the key `chat_quota:consumer1` in Redis will be refreshed to 10000. + +### Query Quota +To query the quota of a specific user, you can use: +curl https://example.com/v1/chat/completions/quota?consumer=consumer1 -H "Authorization: Bearer credential3" +The response will return: {"quota": 10000, "consumer": "consumer1"} + +### Increase or Decrease Quota +To increase or decrease the quota of a specific user, you can use: +curl https://example.com/v1/chat/completions/quota/delta -d "consumer=consumer1&value=100" -H "Authorization: Bearer credential3" +This will increase the value of the key `chat_quota:consumer1` in Redis by 100, and negative values can also be supported, thus subtracting the corresponding value. diff --git a/plugins/wasm-go/extensions/ai-rag/README.md b/plugins/wasm-go/extensions/ai-rag/README.md index 204ad9ee03..8695eb10fa 100644 --- a/plugins/wasm-go/extensions/ai-rag/README.md +++ b/plugins/wasm-go/extensions/ai-rag/README.md @@ -1,9 +1,22 @@ -# 简介 +--- +title: AI RAG +keywords: [ AI网关, AI RAG ] +description: AI RAG 插件配置参考 +--- + +## 功能说明 + 通过对接阿里云向量检索服务实现LLM-RAG，流程如图所示： -# 配置说明 +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`400` + + +## 配置说明 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | |----------------|-----------------|------|-----|----------------------------------------------------------------------------------| | `dashscope.apiKey` | string | 必填 | - | 用于在访问通义千问服务时进行认证的令牌。 | @@ -20,7 +33,7 @@ 插件开启后，在使用链路追踪功能时，会在span的attribute中添加rag检索到的文档id信息，供排查问题使用。 -# 示例 +## 示例 ```yaml dashscope: @@ -54,4 +67,4 @@ dashvector: 经过RAG插件处理后LLM返回的结果为： ``` 海南追尾事故发生在海文高速公路文昌至海口方向37公里处。关于事故的具体原因，交警部门当时仍在进一步调查中，所以根据提供的信息无法确定事故的确切原因。人员伤亡情况是1人死亡（司机当场死亡），另有8人受伤（包括2名儿童和6名成人），所有受伤人员都被解救并送往医院进行治疗。 -``` \ No newline at end of file +``` diff --git a/plugins/wasm-go/extensions/ai-rag/README_EN.md b/plugins/wasm-go/extensions/ai-rag/README_EN.md new file mode 100644 index 0000000000..bde30517e8 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-rag/README_EN.md @@ -0,0 +1,61 @@ +--- +title: AI RAG +keywords: [ AI Gateway, AI RAG ] +description: AI RAG Plugin Configuration Reference +--- +## Function Description +Implement LLM-RAG by integrating with Alibaba Cloud Vector Search Service, as shown in the figure below: + + +## Running Attributes +Plugin execution phase: `Default Phase` +Plugin execution priority: `400` + +## Configuration Description +| Name | Data Type | Requirement | Default Value | Description | +|--------------------------|-----------|-------------|---------------|-------------------------------------------------------------------------------------------| +| `dashscope.apiKey` | string | Required | - | Token used for authentication when accessing Tongyi Qianwen service. | +| `dashscope.serviceFQDN` | string | Required | - | Tongyi Qianwen service name | +| `dashscope.servicePort` | int | Required | - | Tongyi Qianwen service port | +| `dashscope.serviceHost` | string | Required | - | Domain name for accessing Tongyi Qianwen service | +| `dashvector.apiKey` | string | Required | - | Token used for authentication when accessing Alibaba Cloud Vector Search Service. | +| `dashvector.serviceFQDN`| string | Required | - | Alibaba Cloud Vector Search service name | +| `dashvector.servicePort`| int | Required | - | Alibaba Cloud Vector Search service port | +| `dashvector.serviceHost`| string | Required | - | Domain name for accessing Alibaba Cloud Vector Search service | +| `dashvector.topk` | int | Required | - | Number of vectors to retrieve from Alibaba Cloud Vector Search | +| `dashvector.threshold` | float | Required | - | Vector distance threshold; documents above this threshold will be filtered out | +| `dashvector.field` | string | Required | - | Field name where documents are stored in Alibaba Cloud Vector Search | + +Once the plugin is enabled, while using the tracing feature, the document ID information retrieved by RAG will be added to the span's attributes for troubleshooting purposes. + +## Example +```yaml +dashscope: + apiKey: xxxxxxxxxxxxxxx + serviceFQDN: dashscope + servicePort: 443 + serviceHost: dashscope.aliyuncs.com +dashvector: + apiKey: xxxxxxxxxxxxxxxxxxxx + serviceFQDN: dashvector + servicePort: 443 + serviceHost: vrs-cn-xxxxxxxxxxxxxxx.dashvector.cn-hangzhou.aliyuncs.com + collection: xxxxxxxxxxxxxxx + topk: 1 + threshold: 0.4 + field: raw +``` +The [CEC-Corpus](https://github.com/shijiebei2009/CEC-Corpus) dataset contains 332 news reports on emergency events, along with annotation data. The original news text is extracted, vectorized, and then added to Alibaba Cloud Vector Search Service. For text vectorization tutorials, you can refer to [“Implementing Semantic Search Based on Vector Search Service and Lingji”](https://help.aliyun.com/document_detail/2510234.html). + +Below is an example enhanced using RAG, with the original request being: +``` +Where did the rear-end collision in Hainan occur? What was the cause? How many casualties were there? +``` +The result returned by LLM without RAG plugin processing was: +``` +I'm sorry, as an AI model, I cannot retrieve and update specific information on news events in real time, including details such as location, cause, and casualties. For such specific events, it is recommended that you consult the latest news reports or official announcements for accurate information. You can visit mainstream media websites, use news applications, or follow announcements from relevant government departments to get dynamic updates. +``` +After processing with RAG plugin, the result returned by LLM was: +``` +The rear-end collision in Hainan occurred on the Haiven Expressway, 37 kilometers from Wenchang to Haikou. Regarding the specific cause of the accident, traffic police were still conducting further investigations at the time, so the exact cause of the accident cannot be determined based on the provided information. The casualty situation is 1 death (the driver died on the spot) and 8 injuries (including 2 children and 6 adults). All injured persons were rescued and sent to the hospital for treatment. +``` diff --git a/plugins/wasm-go/extensions/ai-security-guard/README.md b/plugins/wasm-go/extensions/ai-security-guard/README.md index a7339b1e44..4961d527b3 100644 --- a/plugins/wasm-go/extensions/ai-security-guard/README.md +++ b/plugins/wasm-go/extensions/ai-security-guard/README.md @@ -1,6 +1,6 @@ -# 简介 +## 功能说明 -# 配置说明 +## 配置说明 | Name | Type | Requirement | Default | Description | | :-: | :-: | :-: | :-: | :-: | | serviceSource | string | requried | - | 服务来源，填dns | @@ -11,7 +11,7 @@ | sk | string | requried | - | 阿里云SK | -# 配置示例 +## 配置示例 ```yaml serviceSource: "dns" serviceName: "safecheck" @@ -19,4 +19,4 @@ servicePort: 443 domain: "green-cip.cn-shanghai.aliyuncs.com" ak: "XXXXXXXXX" sk: "XXXXXXXXXXXXXXX" -``` \ No newline at end of file +``` diff --git a/plugins/wasm-go/extensions/ai-token-ratelimit/README.md b/plugins/wasm-go/extensions/ai-token-ratelimit/README.md index 740191454f..239524fb73 100644 --- a/plugins/wasm-go/extensions/ai-token-ratelimit/README.md +++ b/plugins/wasm-go/extensions/ai-token-ratelimit/README.md @@ -1,10 +1,20 @@ -# 功能说明 +--- +title: AI Token限流 +keywords: [ AI网关, AI token限流 ] +description: AI Token限流插件配置参考 +--- + + +## 功能说明 `ai-token-ratelimit`插件实现了基于特定键值实现token限流，键值来源可以是 URL 参数、HTTP 请求头、客户端 IP 地址、consumer 名称、cookie中 key 名称 +## 运行属性 +插件执行阶段：`默认阶段` +插件执行优先级：`600` -# 配置说明 +## 配置说明 | 配置项 | 类型 | 必填 | 默认值 | 说明 | | ----------------------- | ------ | ---- | ------ |---------------------------------------------------------------------------| @@ -51,9 +61,9 @@ -# 配置示例 +## 配置示例 -## 识别请求参数 apikey，进行区别限流 +### 识别请求参数 apikey，进行区别限流 ```yaml rule_name: default_rule @@ -81,7 +91,7 @@ redis: -## 识别请求头 x-ca-key，进行区别限流 +### 识别请求头 x-ca-key，进行区别限流 ```yaml rule_name: default_rule @@ -109,7 +119,7 @@ redis: -## 根据请求头 x-forwarded-for 获取对端IP，进行区别限流 +### 根据请求头 x-forwarded-for 获取对端IP，进行区别限流 ```yaml rule_name: default_rule @@ -129,7 +139,7 @@ redis: service_name: redis.static ``` -## 识别consumer，进行区别限流 +### 识别consumer，进行区别限流 ```yaml rule_name: default_rule @@ -157,7 +167,7 @@ redis: -## 识别cookie中的键值对，进行区别限流 +### 识别cookie中的键值对，进行区别限流 ```yaml rule_name: default_rule @@ -183,4 +193,4 @@ rejected_code: 200 rejected_msg: '{"code":-1,"msg":"Too many requests"}' redis: service_name: redis.static -``` \ No newline at end of file +``` diff --git a/plugins/wasm-go/extensions/ai-token-ratelimit/README_EN.md b/plugins/wasm-go/extensions/ai-token-ratelimit/README_EN.md new file mode 100644 index 0000000000..0650bbf1b0 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-token-ratelimit/README_EN.md @@ -0,0 +1,170 @@ +--- +title: AI Token Rate Limiting +keywords: [ AI Gateway, AI Token Rate Limiting ] +description: AI Token Rate Limiting Plugin Configuration Reference +--- +## Function Description +The `ai-token-ratelimit` plugin implements token rate limiting based on specific key values. The key values can come from URL parameters, HTTP request headers, client IP addresses, consumer names, or key names in cookies. + +## Runtime Attributes +Plugin execution phase: `default phase` +Plugin execution priority: `600` + +## Configuration Description +| Configuration Item | Type | Required | Default Value | Description | +| ----------------------- | ----------------- | -------- | ------------- | ----------------------------------------------------------------------------- | +| rule_name | string | Yes | - | Name of the rate limiting rule, used to assemble the redis key based on the rule name + rate limiting type + rate limiting key name + actual value corresponding to the rate limiting key | +| rule_items | array of object | Yes | - | Rate limiting rule items. After matching the first rule_item, subsequent rules will be ignored based on the order in `rule_items` | +| rejected_code | int | No | 429 | The HTTP status code returned when the request is rate limited | +| rejected_msg | string | No | Too many requests | The response body returned when the request is rate limited | +| redis | object | Yes | - | Redis related configuration | + +Field descriptions for each item in `rule_items` +| Configuration Item | Type | Required | Default Value | Description | +| ------------------------ | ----------------- | --------------------------- | ------------- | ------------------------------------------------------------ | +| limit_by_header | string | No, optionally select one in `limit_by_*` | - | Configure the source HTTP header name for obtaining the rate limiting key value | +| limit_by_param | string | No, optionally select one in `limit_by_*` | - | Configure the source URL parameter name for obtaining the rate limiting key value | +| limit_by_consumer | string | No, optionally select one in `limit_by_*` | - | Rate limit by consumer name, no actual value needs to be added | +| limit_by_cookie | string | No, optionally select one in `limit_by_*` | - | Configure the source key name in cookies for obtaining the rate limiting key value | +| limit_by_per_header | string | No, optionally select one in `limit_by_*` | - | Match specific HTTP request headers according to rules and calculate rate limiting separately for each header. Configure the source HTTP header name for obtaining the rate limiting key value. Supports regular expressions or `*` when configuring `limit_keys` | +| limit_by_per_param | string | No, optionally select one in `limit_by_*` | - | Match specific URL parameters according to rules and calculate rate limiting separately for each parameter. Configure the source URL parameter name for obtaining the rate limiting key value. Supports regular expressions or `*` when configuring `limit_keys` | +| limit_by_per_consumer | string | No, optionally select one in `limit_by_*` | - | Match specific consumers according to rules and calculate rate limiting separately for each consumer. Rate limit by consumer name, no actual value needs to be added. Supports regular expressions or `*` when configuring `limit_keys` | +| limit_by_per_cookie | string | No, optionally select one in `limit_by_*` | - | Match specific cookies according to rules and calculate rate limiting separately for each cookie. Configure the source key name in cookies for obtaining the rate limiting key value. Supports regular expressions or `*` when configuring `limit_keys` | +| limit_by_per_ip | string | No, optionally select one in `limit_by_*` | - | Match specific IPs according to rules and calculate rate limiting separately for each IP. Configure the source IP parameter name for obtaining the rate limiting key value from request headers, `from-header-
`, such as `from-header-x-forwarded-for`. Directly get the remote socket IP by configuring `from-remote-addr` | +| limit_keys | array of object | Yes | - | Configure the number of rate limit requests after matching keys | + +Field descriptions for each item in `limit_keys` +| Configuration Item | Type | Required | Default Value | Description | +| ----------------------- | ----------------- | ------------------------------------------- | ------------- | ----------------------------------------------- | +| key | string | Yes | - | Matched key value. Types `limit_by_per_header`, `limit_by_per_param`, `limit_by_per_consumer`, `limit_by_per_cookie` support configuring regular expressions (beginning with regexp: followed by the regex) or `*` (representing all). Example regex: `regexp:^d.*` (all strings starting with d); `limit_by_per_ip` supports configuring IP addresses or IP segments | +| token_per_second | int | No, optionally select one in `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` | - | Allowed number of token requests per second | +| token_per_minute | int | No, optionally select one in `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` | - | Allowed number of token requests per minute | +| token_per_hour | int | No, optionally select one in `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` | - | Allowed number of token requests per hour | +| token_per_day | int | No, optionally select one in `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` | - | Allowed number of token requests per day | + +Field descriptions for each item in `redis` +| Configuration Item | Type | Required | Default Value | Description | +| ----------------------- | ----------------- | -------- | --------------------------------------------------------------- | ----------------------------------------------- | +| service_name | string | Required | - | Full FQDN name of the redis service, including service type, e.g., my-redis.dns, redis.my-ns.svc.cluster.local | +| service_port | int | No | Default value for static addresses (static service) is 80; otherwise, it is 6379 | Input the service port of the redis service | +| username | string | No | - | Redis username | +| password | string | No | - | Redis password | +| timeout | int | No | 1000 | Redis connection timeout in milliseconds | + +## Configuration Examples +### Identify request parameter apikey for differentiated rate limiting +```yaml +rule_name: default_rule +rule_items: + - limit_by_param: apikey + limit_keys: + - key: 9a342114-ba8a-11ec-b1bf-00163e1250b5 + token_per_minute: 10 + - key: a6a6d7f2-ba8a-11ec-bec2-00163e1250b5 + token_per_hour: 100 + - limit_by_per_param: apikey + limit_keys: + # Regular expression, matches all strings starting with a, each apikey corresponds to 10 qds + - key: "regexp:^a.*" + token_per_second: 10 + # Regular expression, matches all strings starting with b, each apikey corresponds to 100 qd + - key: "regexp:^b.*" + token_per_minute: 100 + # Fallback, matches all requests, each apikey corresponds to 1000 qdh + - key: "*" + token_per_hour: 1000 +redis: + service_name: redis.static +``` +### Identify request header x-ca-key for differentiated rate limiting +```yaml +rule_name: default_rule +rule_items: + - limit_by_header: x-ca-key + limit_keys: + - key: 102234 + token_per_minute: 10 + - key: 308239 + token_per_hour: 10 + - limit_by_per_header: x-ca-key + limit_keys: + # Regular expression, matches all strings starting with a, each apikey corresponds to 10 qds + - key: "regexp:^a.*" + token_per_second: 10 + # Regular expression, matches all strings starting with b, each apikey corresponds to 100 qd + - key: "regexp:^b.*" + token_per_minute: 100 + # Fallback, matches all requests, each apikey corresponds to 1000 qdh + - key: "*" + token_per_hour: 1000 +redis: + service_name: redis.static +``` +### Get the peer IP using the request header x-forwarded-for for differentiated rate limiting +```yaml +rule_name: default_rule +rule_items: + - limit_by_per_ip: from-header-x-forwarded-for + limit_keys: + # Exact IP + - key: 1.1.1.1 + token_per_day: 10 + # IP segment, matching IPs in this segment, each IP 100 qpd + - key: 1.1.1.0/24 + token_per_day: 100 + # Fallback, i.e., default each IP 1000 qpd + - key: 0.0.0.0/0 + token_per_day: 1000 +redis: + service_name: redis.static +``` +### Identify consumer for differentiated rate limiting +```yaml +rule_name: default_rule +rule_items: + - limit_by_consumer: '' + limit_keys: + - key: consumer1 + token_per_second: 10 + - key: consumer2 + token_per_hour: 100 + - limit_by_per_consumer: '' + limit_keys: + # Regular expression, matches all strings starting with a, each consumer corresponds to 10 qds + - key: "regexp:^a.*" + token_per_second: 10 + # Regular expression, matches all strings starting with b, each consumer corresponds to 100 qd + - key: "regexp:^b.*" + token_per_minute: 100 + # Fallback, matches all requests, each consumer corresponds to 1000 qdh + - key: "*" + token_per_hour: 1000 +redis: + service_name: redis.static +``` +### Identify key-value pairs in cookies for differentiated rate limiting +```yaml +rule_name: default_rule +rule_items: + - limit_by_cookie: key1 + limit_keys: + - key: value1 + token_per_minute: 10 + - key: value2 + token_per_hour: 100 + - limit_by_per_cookie: key1 + limit_keys: + # Regular expression, matches all strings starting with a, each value in cookie corresponds to 10 qds + - key: "regexp:^a.*" + token_per_second: 10 + # Regular expression, matches all strings starting with b, each value in cookie corresponds to 100 qd + - key: "regexp:^b.*" + token_per_minute: 100 + # Fallback, matches all requests, each value in cookie corresponds to 1000 qdh + - key: "*" + token_per_hour: 1000 +rejected_code: 200 +rejected_msg: '{"code":-1,"msg":"Too many requests"}' +redis: + service_name: redis.static +``` diff --git a/plugins/wasm-go/extensions/ai-transformer/README.md b/plugins/wasm-go/extensions/ai-transformer/README.md index 9b098a14f4..28dd8abdd4 100644 --- a/plugins/wasm-go/extensions/ai-transformer/README.md +++ b/plugins/wasm-go/extensions/ai-transformer/README.md @@ -1,7 +1,19 @@ -# 简介 -低代码开发插件，通过LLM对请求/响应的header以及body进行修改。 +--- +title: AI 请求响应转换 +keywords: [higress,AI transformer] +description: AI 请求响应转换插件配置参考 +--- -# 配置说明 + +## 功能说明 +AI 请求响应转换插件，通过LLM对请求/响应的header以及body进行修改。 + +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`410` + +## 配置说明 | Name | Type | Requirement | Default | Description | | :- | :- | :- | :- | :- | | request.enable | bool | requried | - | 是否在request阶段开启转换 | @@ -12,7 +24,7 @@ | provider.domain | string | requried | - | LLM服务域名 | | provider.apiKey | string | requried | - | 阿里云dashscope服务的API Key | -# 配置示例 +## 配置示例 ```yaml request: enable: false diff --git a/plugins/wasm-go/extensions/ai-transformer/README_EN.md b/plugins/wasm-go/extensions/ai-transformer/README_EN.md new file mode 100644 index 0000000000..e8eb5d31d0 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-transformer/README_EN.md @@ -0,0 +1,85 @@ +--- +title: AI Request-Response Transformation +keywords: [higress, AI transformer] +description: AI Request-Response Transformation plugin configuration reference +--- +## Function Description +The AI Request-Response Transformation plugin modifies the header and body of requests/responses using LLM. + +## Execution Attributes +Plugin execution phase: `Authentication Phase` +Plugin execution priority: `410` + +## Configuration Description +| Name | Type | Requirement | Default | Description | +| :- | :- | :- | :- | :- | +| request.enable | bool | required | - | Whether to enable transformation in the request phase | +| request.prompt | string | required | - | Prompt used for transformation in the request phase | +| response.enable | bool | required | - | Whether to enable transformation in the response phase | +| response.prompt | string | required | - | Prompt used for transformation in the response phase | +| provider.serviceName | string | required | - | DNS type service name, currently only supports Qwen | +| provider.domain | string | required | - | LLM service domain | +| provider.apiKey | string | required | - | Alibaba Cloud Dashscope service API Key | + +## Configuration Example +```yaml +request: + enable: false + prompt: "If the request path starts with /httpbin, please remove the /httpbin prefix and do not change anything else." +response: + enable: true + prompt: "Please modify the following HTTP response information with the requirements: 1. change content-type to application/json; 2. convert body from xml to json; 3. remove content-length." +provider: + serviceName: qwen + domain: dashscope.aliyuncs.com + apiKey: xxxxxxxxxxxxx +``` + +Accessing the original httpbin's /xml interface yields: +``` + + + + + + Wake up to WonderWidgets! + + + + Overview + Why WonderWidgets are great + + Who buys WonderWidgets + + +``` + +Using the above configuration, accessing the httpbin's /xml interface through the gateway yields: +``` +{ + "slideshow": { + "title": "Sample Slide Show", + "date": "Date of publication", + "author": "Yours Truly", + "slides": [ + { + "type": "all", + "title": "Wake up to WonderWidgets!" + }, + { + "type": "all", + "title": "Overview", + "items": [ + "Why WonderWidgets are great", + "", + "Who buys WonderWidgets" + ] + } + ] + } +} +``` diff --git a/plugins/wasm-go/extensions/basic-auth/README.md b/plugins/wasm-go/extensions/basic-auth/README.md index e8b37b674e..6a39fd1481 100644 --- a/plugins/wasm-go/extensions/basic-auth/README.md +++ b/plugins/wasm-go/extensions/basic-auth/README.md @@ -14,12 +14,17 @@ description: Basic 认证插件配置参考 ## 配置字段 -### 全局配置 +**注意：** + +- 在一个规则里，鉴权配置和认证配置不可同时存在 +- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ----------- | --------------- | -------- | ------ | ---------------------------------------------------- | -| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | -| `global_auth` | bool | 选填 | - | 若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制; 若不配置则仅当没有域名和路由配置时全局生效（兼容机制） | +### 认证配置 + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | -------- | ------ | ---------------------------------------------------- | +| `global_auth` | bool | 选填（**仅实例级别配置**） | - | 只能在实例级别配置，若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制，若不配置则仅当没有域名和路由配置时全局生效（兼容老用户使用习惯）。 | +| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | `consumers`中每一项的配置字段说明如下： @@ -28,22 +33,20 @@ description: Basic 认证插件配置参考 | `credential` | string | 必填 | - | 配置该consumer的访问凭证 | | `name` | string | 必填 | - | 配置该consumer的名称 | -### 域名和路由级配置 +### 鉴权配置（非必需） | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | | `allow` | array of string | 必填 | - | 对于符合匹配条件的请求，配置允许访问的consumer名称 | -**注意：** -- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 - ## 配置示例 -### 对特定路由或域名开启认证和鉴权 +### 全局配置认证和路由粒度进行鉴权 以下配置将对网关特定路由或域名开启 Basic Auth 认证和鉴权，注意凭证信息中的用户名和密码之间使用":"分隔，`credential`字段不能重复 -**全局配置** + +在实例级别做如下插件配置： ```yaml consumers: @@ -54,9 +57,6 @@ consumers: global_auth: false ``` - -**路由级配置** - 对 route-a 和 route-b 这两个路由做如下配置： ```yaml @@ -75,7 +75,7 @@ allow: 此例指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将允许`name`为`consumer2`的调用者访问，其他调用者不允许访问。 -#### 根据该配置，下列请求可以允许访问： +根据该配置，下列请求可以允许访问： **请求指定用户名密码** @@ -89,7 +89,7 @@ curl -H 'Authorization: Basic YWRtaW46MTIzNDU2' http://xxx.hello.com/test 认证鉴权通过后，请求的header中会被添加一个`X-Mse-Consumer`字段，在此例中其值为`consumer1`，用以标识调用方的名称 -#### 下列请求将拒绝访问： +下列请求将拒绝访问： **请求未提供用户名密码，返回401** ```bash @@ -111,4 +111,4 @@ curl -u guest:abc http://xxx.hello.com/test | ----------- |--------------------------------------------------------------------------------| ---------------------- | | 401 | Request denied by Basic Auth check. No Basic Authentication information found. | 请求未提供凭证 | | 401 | Request denied by Basic Auth check. Invalid username and/or password. | 请求凭证无效 | -| 403 | Request denied by Basic Auth check. Unauthorized consumer. | 请求的调用方无访问权限 | \ No newline at end of file +| 403 | Request denied by Basic Auth check. Unauthorized consumer. | 请求的调用方无访问权限 | diff --git a/plugins/wasm-go/extensions/basic-auth/README_EN.md b/plugins/wasm-go/extensions/basic-auth/README_EN.md index 880f961edd..b5cc71b98b 100644 --- a/plugins/wasm-go/extensions/basic-auth/README_EN.md +++ b/plugins/wasm-go/extensions/basic-auth/README_EN.md @@ -1,119 +1,99 @@ --- -title: Basic Auth -keywords: [higress, basic auth] -description: Basic authentication plug-in configuration reference +title: Basic Authentication +keywords: [higress,basic auth] +description: Basic authentication plugin configuration reference --- +## Function Description +The `basic-auth` plugin implements authentication and authorization based on the HTTP Basic Auth standard. -## Description -`basic-auth` plugin implements the function of authentication based on the HTTP Basic Auth standard. +## Operation Attributes +Plugin execution stage: `Authentication Phase` +Plugin execution priority: `320` ## Configuration Fields - -| Name | Type | Requirement | Default Value | Description | -| ----------- | --------------- | -------- | ------ | ---------------------------------------------------- | -| `consumers` | array of object | Required | - | Caller of the service for authentication of requests | -| `_rules_` | array of object | Optional | - | Configure access permission list for specific routes or domains to authenticate requests | - -Filed descriptions of `consumers` items: - -| Name | Type | Requirement | Default Value | Description | -| ------------ | ------ | ----------- | ------------- | ------------------------------------- | -| `credential` | string | Required | - | Credential for this consumer's access | -| `name` | string | Required | - | Name of this consumer | - -Configuration field descriptions for each item in `_rules_` are as follows: - -| Field Name | Data Type | Requirement | Default | Description | -| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- | -| `_match_route_` | array of string | One of `_match_route_` or `_match_domain_` | - | Configure the routes to match for request authorization | -| `_match_domain_` | array of string | One of `_match_route_` , `_match_domain_` | - | Configure the domains to match for request authorization | -| `allow` | array of string | Required | - | Configure the consumer names allowed to access requests that match the match condition | - **Note:** - -- If the `_rules_` field is not configured, authentication is enabled for all routes of the current gateway instance by default; -- For authenticated requests, `X-Mse-Consumer` field will be added to the request header to identify the name of the caller. - -## Configuration Samples - -### Enable Authentication and Authorization for specific routes or domains - -The following configuration will enable Basic Auth authentication and authorization for specific routes or domains of the gateway. Note that the username and password in the credential information are separated by a ":", and the `credential` field cannot be repeated. - - - +- In one rule, authentication configurations and authorization configurations cannot coexist. +- For requests that pass authentication, the request header will include an `X-Mse-Consumer` field to identify the caller's name. + +### Authentication Configuration +| Name | Data Type | Requirements | Default Value | Description | +| ------------- | ---------------- | ------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `global_auth` | bool | Optional (**instance-level only**) | - | Can only be configured at the instance level. If set to true, the authentication mechanism will take effect globally; if set to false, it will only take effect for the configured domains and routes. If not configured, it will only take effect globally when there are no domain and route configurations (compatible with old user habits). | +| `consumers` | array of object | Required | - | Configures the service callers for request authentication. | + +Each configuration field in `consumers` is described as follows: +| Name | Data Type | Requirements | Default Value | Description | +| ------------ | --------- | ------------ | ------------- | ------------------------------- | +| `credential` | string | Required | - | Configures the access credentials for this consumer. | +| `name` | string | Required | - | Configures the name of this consumer. | + +### Authorization Configuration (Optional) +| Name | Data Type | Requirements | Default Value | Description | +| ---------------- | ---------------- | ---------------------------------------------------- | -------------- | -------------------------------------------------------- | +| `allow` | array of string | Required | - | Configures the consumer names allowed to access for matching requests. | + +## Configuration Example +### Global Authentication and Route Granularity Authorization +The following configuration will enable Basic Auth authentication and authorization for specific routes or domains of the gateway. Note that the username and password in the credential information are separated by ":", and the `credential` field cannot be duplicated. + +Make the following plugin configuration at the instance level: ```yaml -# use the _rules_ field for fine-grained rule configuration. consumers: - credential: 'admin:123456' name: consumer1 - credential: 'guest:abc' name: consumer2 -_rules_: -# rule 1: match by the route name. - - _match_route_: - - route-a - - route-b - allow: - - consumer1 -# rule 2: match by the domain. - - _match_domain_: - - "*.example.com" - - test.com - allow: - - consumer2 +global_auth: false ``` -In this sample, `route-a` and `route-b` specified in `_match_route_` are the route names filled in when creating gateway routes. When these two routes are matched, the caller with `name` as `consumer1` is allowed to access, and other callers are not allowed to access. -The `*.example.com` and `test.com` specified in `_match_domain_` are used to match the domain name of the request. When the domain name is matched, the caller with `name` as `consumer2` is allowed to access, and other callers are not allowed to access. +For routes `route-a` and `route-b`, configure as follows: +```yaml +allow: +- consumer1 +``` +For the domains `*.example.com` and `test.com`, configure as follows: +```yaml +allow: +- consumer2 +``` -#### According to this configuration, the following requests are allowed: +If configured in the console, the specified `route-a` and `route-b` refer to the route names filled in when creating the routes in the console. When matching these two routes, callers with the name `consumer1` will be allowed access, while other callers will not. -**Requests with specified username and password** +The specified `*.example.com` and `test.com` are used to match the request domain. When a match is found, callers with the name `consumer2` will be allowed access, while other callers will not. +Based on this configuration, the following requests may be allowed access: +**Request with specified username and password** ```bash -# Assuming the following request will match with route-a -# Use -u option of curl to specify the credentials +# Assuming the following request matches the route-a route +# Using curl's -u parameter to specify curl -u admin:123456 http://xxx.hello.com/test -# Or specify the Authorization request header directly with the credentials in base64 encoding +# Or directly specify the Authorization request header with the username and password encoded in base64 curl -H 'Authorization: Basic YWRtaW46MTIzNDU2' http://xxx.hello.com/test ``` -A `X-Mse-Consumer` field will be added to the headers of the request, and its value in this example is `consumer1`, used to identify the name of the caller when passed authentication and authorization. - -#### The following requests will be denied: +After successful authentication, the request header will have an added `X-Mse-Consumer` field, which in this case is `consumer1` to identify the caller's name. -**Requests without providing username and password, returning 401** +The following requests will be denied access: +**Request without username and password, returns 401** ```bash curl http://xxx.hello.com/test ``` -**Requests with incorrect username or password, returning 401** + +**Request with incorrect username and password, returns 401** ```bash curl -u admin:abc http://xxx.hello.com/test ``` -**Requests matched with a caller who has no access permission, returning 403** + +**Caller matched by username and password has no access, returns 403** ```bash -# consumer2 is not in the allow list of route-a +# consumer2 is not in the allow list for route-a curl -u guest:abc http://xxx.hello.com/test ``` -### Enable basic auth for gateway instance - -The following configuration does not specify the `_rules_` field, so Basic Auth authentication will be effective for the whole gateway instance. - -```yaml -consumers: -- credential: 'admin:123456' - name: consumer1 -- credential: 'guest:abc' - name: consumer2 -``` - -## Error Codes - -| HTTP Status Code | Error Info | Reason | -| ----------- |--------------------------------------------------------------------------------| ---------------------- | -| 401 | Request denied by Basic Auth check. No Basic Authentication information found. | Credentials not provided in the request | -| 401 | Request denied by Basic Auth check. Invalid username and/or password. | Invalid username and/or password | -| 403 | Request denied by Basic Auth check. Unauthorized consumer. | Unauthorized consumer | \ No newline at end of file +## Related Error Codes +| HTTP Status Code | Error Message | Reason Description | +| ---------------- | ------------------------------------------------------------------------------------- | -------------------------------- | +| 401 | Request denied by Basic Auth check. No Basic Authentication information found. | Request did not provide credentials. | +| 401 | Request denied by Basic Auth check. Invalid username and/or password. | Request credentials are invalid. | +| 403 | Request denied by Basic Auth check. Unauthorized consumer. | The caller making the request does not have access. | diff --git a/plugins/wasm-go/extensions/bot-detect/README.md b/plugins/wasm-go/extensions/bot-detect/README.md index b56fba8e8f..11311313dd 100644 --- a/plugins/wasm-go/extensions/bot-detect/README.md +++ b/plugins/wasm-go/extensions/bot-detect/README.md @@ -1,11 +1,21 @@ -

- English | 中文 -

+--- +title: Bot 拦截 +keywords: [higress,bot detect] +description: Bot 拦截插件配置参考 +--- + + +## 功能说明 -# 功能说明 `bot-detect`插件可以用于识别并阻止互联网爬虫对站点资源的爬取 -# 配置字段 +## 运行属性 + +插件执行阶段：`授权阶段` +插件执行优先级：`310` + + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- | -------- | -------- | @@ -33,9 +43,9 @@ (CSimpleSpider|Cityreview Robot|CrawlDaddy|CrawlFire|Finderbots|Index crawler|Job Roboter|KiwiStatus Spider|Lijit Crawler|QuerySeekerSpider|ScollSpider|Trends Crawler|USyd-NLP-Spider|SiteCat Webbot|BotName\/\$BotVersion|123metaspider-Bot|1470\.net crawler|50\.nu|8bo Crawler Bot|Aboundex|Accoona-[A-z]{1,30}-Agent|AdsBot-Google(?:-[a-z]{1,30}|)|altavista|AppEngine-Google|archive.{0,30}\.org_bot|archiver|Ask Jeeves|[Bb]ai[Dd]u[Ss]pider(?:-[A-Za-z]{1,30})(?:-[A-Za-z]{1,30}|)|bingbot|BingPreview|blitzbot|BlogBridge|Bloglovin|BoardReader Blog Indexer|BoardReader Favicon Fetcher|boitho.com-dc|BotSeer|BUbiNG|\b\w{0,30}favicon\w{0,30}\b|\bYeti(?:-[a-z]{1,30}|)|Catchpoint(?: bot|)|[Cc]harlotte|Checklinks|clumboot|Comodo HTTP\(S\) Crawler|Comodo-Webinspector-Crawler|ConveraCrawler|CRAWL-E|CrawlConvera|Daumoa(?:-feedfetcher|)|Feed Seeker Bot|Feedbin|findlinks|Flamingo_SearchEngine|FollowSite Bot|furlbot|Genieo|gigabot|GomezAgent|gonzo1|(?:[a-zA-Z]{1,30}-|)Googlebot(?:-[a-zA-Z]{1,30}|)|Google SketchUp|grub-client|gsa-crawler|heritrix|HiddenMarket|holmes|HooWWWer|htdig|ia_archiver|ICC-Crawler|Icarus6j|ichiro(?:/mobile|)|IconSurf|IlTrovatore(?:-Setaccio|)|InfuzApp|Innovazion Crawler|InternetArchive|IP2[a-z]{1,30}Bot|jbot\b|KaloogaBot|Kraken|Kurzor|larbin|LEIA|LesnikBot|Linguee Bot|LinkAider|LinkedInBot|Lite Bot|Llaut|lycos|Mail\.RU_Bot|masscan|masidani_bot|Mediapartners-Google|Microsoft .{0,30} Bot|mogimogi|mozDex|MJ12bot|msnbot(?:-media {0,2}|)|msrbot|Mtps Feed Aggregation System|netresearch|Netvibes|NewsGator[^/]{0,30}|^NING|Nutch[^/]{0,30}|Nymesis|ObjectsSearch|OgScrper|Orbiter|OOZBOT|PagePeeker|PagesInventory|PaxleFramework|Peeplo Screenshot Bot|PlantyNet_WebRobot|Pompos|Qwantify|Read%20Later|Reaper|RedCarpet|Retreiver|Riddler|Rival IQ|scooter|Scrapy|Scrubby|searchsight|seekbot|semanticdiscovery|SemrushBot|Simpy|SimplePie|SEOstats|SimpleRSS|SiteCon|Slackbot-LinkExpanding|Slack-ImgProxy|Slurp|snappy|Speedy Spider|Squrl Java|Stringer|TheUsefulbot|ThumbShotsBot|Thumbshots\.ru|Tiny Tiny RSS|Twitterbot|WhatsApp|URL2PNG|Vagabondo|VoilaBot|^vortex|Votay bot|^voyager|WASALive.Bot|Web-sniffer|WebThumb|WeSEE:[A-z]{1,30}|WhatWeb|WIRE|WordPress|Wotbox|www\.almaden\.ibm\.com|Xenu(?:.s|) Link Sleuth|Xerka [A-z]{1,30}Bot|yacy(?:bot|)|YahooSeeker|Yahoo! Slurp|Yandex\w{1,30}|YodaoBot(?:-[A-z]{1,30}|)|YottaaMonitor|Yowedo|^Zao|^Zao-Crawler|ZeBot_www\.ze\.bz|ZooShot|ZyBorg)(?:[ /]v?(\d+)(?:\.(\d+)(?:\.(\d+)|)|)|) ``` -# 配置示例 +## 配置示例 -## 放行原本命中爬虫规则的请求 +### 放行原本命中爬虫规则的请求 ```yaml allow: - ".*Go-http-client.*" @@ -44,7 +54,7 @@ allow: 若不作该配置，默认的 Golang 网络库请求会被视做爬虫，被禁止访问 -## 增加爬虫判断 +### 增加爬虫判断 ```yaml deny: - "spd-tools.*" @@ -55,4 +65,4 @@ deny: ```bash curl http://example.com -H 'User-Agent: spd-tools/1.1' curl http://exmaple.com -H 'User-Agent: spd-tools' -``` \ No newline at end of file +``` diff --git a/plugins/wasm-go/extensions/bot-detect/README_EN.md b/plugins/wasm-go/extensions/bot-detect/README_EN.md index 44eab957ad..1fbaf1a6a6 100644 --- a/plugins/wasm-go/extensions/bot-detect/README_EN.md +++ b/plugins/wasm-go/extensions/bot-detect/README_EN.md @@ -1,22 +1,26 @@ -

- English | 中文 -

- -# Description -`bot-detect` plugin can be used to identify and prevent web crawlers from crawling websites. - -# Configuration Fields - -| Name | Type | Requirement | Default Value | Description | -| -------- | -------- | -------- | -------- | -------- | -| allow | array of string | Optional | - | A regular expression to match the User-Agent request header and will allow access if the match hits | -| deny | array of string | Optional | - | A regular expression to match the User-Agent request header and will block the request if the match hits | -| blocked_code | number | Optional | 403 | The HTTP status code returned when a request is blocked | -| blocked_message | string | Optional | - | The HTTP response Body returned when a request is blocked | - -If field `allow` and field `deny` are not configured at the same time, the default logic to identify crawlers will be executed. By configuring the `allow` field, requests that would otherwise hit the default logic can be allowed. The judgement can be extended by configuring the `deny` field - -The default set of crawler judgment regular expressions is as follows： +--- +title: Bot Detect +keywords: [higress, bot detect] +description: Bot detect plugin configuration reference +--- +## Function Description +The `bot-detect` plugin can be used to identify and block internet crawlers from accessing site resources. + +## Running Properties +Plugin Execution Phase: `Authorization Phase` +Plugin Execution Priority: `310` + +## Configuration Fields +| Name | Data Type | Required | Default Value | Description | +| ----------------- | ------------------- | --------------| --------------| ---------------------------------------------------------- | +| allow | array of string | Optional | - | Regular expressions to match the User-Agent request header; requests matching will be allowed to access. | +| deny | array of string | Optional | - | Regular expressions to match the User-Agent request header; requests matching will be blocked. | +| blocked_code | number | Optional | 403 | HTTP status code returned when a request is blocked. | +| blocked_message | string | Optional | - | HTTP response body returned when a request is blocked. | + +The `allow` and `deny` fields can both be left unconfigured, in which case the default crawler identification logic will be executed. Configuring the `allow` field can allow requests that would otherwise hit the default crawler identification logic. Configuring the `deny` field can add additional crawler identification logic. + +The default crawler identification regular expression set is as follows: ```bash # Bots General matcher 'name/0.0' @@ -33,26 +37,23 @@ The default set of crawler judgment regular expressions is as follows： (CSimpleSpider|Cityreview Robot|CrawlDaddy|CrawlFire|Finderbots|Index crawler|Job Roboter|KiwiStatus Spider|Lijit Crawler|QuerySeekerSpider|ScollSpider|Trends Crawler|USyd-NLP-Spider|SiteCat Webbot|BotName\/\$BotVersion|123metaspider-Bot|1470\.net crawler|50\.nu|8bo Crawler Bot|Aboundex|Accoona-[A-z]{1,30}-Agent|AdsBot-Google(?:-[a-z]{1,30}|)|altavista|AppEngine-Google|archive.{0,30}\.org_bot|archiver|Ask Jeeves|[Bb]ai[Dd]u[Ss]pider(?:-[A-Za-z]{1,30})(?:-[A-Za-z]{1,30}|)|bingbot|BingPreview|blitzbot|BlogBridge|Bloglovin|BoardReader Blog Indexer|BoardReader Favicon Fetcher|boitho.com-dc|BotSeer|BUbiNG|\b\w{0,30}favicon\w{0,30}\b|\bYeti(?:-[a-z]{1,30}|)|Catchpoint(?: bot|)|[Cc]harlotte|Checklinks|clumboot|Comodo HTTP\(S\) Crawler|Comodo-Webinspector-Crawler|ConveraCrawler|CRAWL-E|CrawlConvera|Daumoa(?:-feedfetcher|)|Feed Seeker Bot|Feedbin|findlinks|Flamingo_SearchEngine|FollowSite Bot|furlbot|Genieo|gigabot|GomezAgent|gonzo1|(?:[a-zA-Z]{1,30}-|)Googlebot(?:-[a-zA-Z]{1,30}|)|Google SketchUp|grub-client|gsa-crawler|heritrix|HiddenMarket|holmes|HooWWWer|htdig|ia_archiver|ICC-Crawler|Icarus6j|ichiro(?:/mobile|)|IconSurf|IlTrovatore(?:-Setaccio|)|InfuzApp|Innovazion Crawler|InternetArchive|IP2[a-z]{1,30}Bot|jbot\b|KaloogaBot|Kraken|Kurzor|larbin|LEIA|LesnikBot|Linguee Bot|LinkAider|LinkedInBot|Lite Bot|Llaut|lycos|Mail\.RU_Bot|masscan|masidani_bot|Mediapartners-Google|Microsoft .{0,30} Bot|mogimogi|mozDex|MJ12bot|msnbot(?:-media {0,2}|)|msrbot|Mtps Feed Aggregation System|netresearch|Netvibes|NewsGator[^/]{0,30}|^NING|Nutch[^/]{0,30}|Nymesis|ObjectsSearch|OgScrper|Orbiter|OOZBOT|PagePeeker|PagesInventory|PaxleFramework|Peeplo Screenshot Bot|PlantyNet_WebRobot|Pompos|Qwantify|Read%20Later|Reaper|RedCarpet|Retreiver|Riddler|Rival IQ|scooter|Scrapy|Scrubby|searchsight|seekbot|semanticdiscovery|SemrushBot|Simpy|SimplePie|SEOstats|SimpleRSS|SiteCon|Slackbot-LinkExpanding|Slack-ImgProxy|Slurp|snappy|Speedy Spider|Squrl Java|Stringer|TheUsefulbot|ThumbShotsBot|Thumbshots\.ru|Tiny Tiny RSS|Twitterbot|WhatsApp|URL2PNG|Vagabondo|VoilaBot|^vortex|Votay bot|^voyager|WASALive.Bot|Web-sniffer|WebThumb|WeSEE:[A-z]{1,30}|WhatWeb|WIRE|WordPress|Wotbox|www\.almaden\.ibm\.com|Xenu(?:.s|) Link Sleuth|Xerka [A-z]{1,30}Bot|yacy(?:bot|)|YahooSeeker|Yahoo! Slurp|Yandex\w{1,30}|YodaoBot(?:-[A-z]{1,30}|)|YottaaMonitor|Yowedo|^Zao|^Zao-Crawler|ZeBot_www\.ze\.bz|ZooShot|ZyBorg)(?:[ /]v?(\d+)(?:\.(\d+)(?:\.(\d+)|)|)|) ``` -# Configuration Samples - -## Release Requests that would otherwise Hit the Crawler Rules +## Configuration Example +### Allowing Requests That Hit the Crawler Rules ```yaml allow: - ".*Go-http-client.*" ``` -Without this configuration, the default Golang web library request will be treated as a crawler and access will be denied. - +If this configuration is not made, requests from the default Golang network library will be treated as crawlers and blocked. -## Add Crawler Judgement +### Adding Crawler Identification ```yaml deny: - "spd-tools.*" ``` -According to this configuration, the following requests will be denied: - +With this configuration, the following requests will be blocked: ```bash curl http://example.com -H 'User-Agent: spd-tools/1.1' curl http://exmaple.com -H 'User-Agent: spd-tools' -``` \ No newline at end of file +``` diff --git a/plugins/wasm-go/extensions/cache-control/README.md b/plugins/wasm-go/extensions/cache-control/README.md index ea386bac40..9837e7ab31 100644 --- a/plugins/wasm-go/extensions/cache-control/README.md +++ b/plugins/wasm-go/extensions/cache-control/README.md @@ -1,14 +1,25 @@ -# 功能说明 +--- +title: 浏览器缓存控制 +keywords: [higress,浏览器缓存控制] +description: 浏览器缓存控制插件配置参考 +--- + +## 功能说明 `cache-control`插件实现了基于 URL 文件后缀来为请求的响应头部添加 `Expires` 和 `Cache-Control` 头部，从而方便浏览器对特定后缀的文件进行缓存，例如 `jpg`、`png` 等图片文件。 -# 配置字段 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`420` + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | |---------|--------|-----------------------------------------------------------------------------------------------------|-|--------------------------| | suffix | string | 选填，表示匹配的文件后缀名，例如 `jpg`、`png` 等。
如果需要匹配多种后缀，需要用 `\|` 进行分割，例如 `png\|jpg`。
如果不填写，表示匹配所有后缀 | - | 配置用于匹配的请求文件后缀 | | expires | string | 必填，表示缓存的最长时间。
当填入的字符串为数字时，单位为秒，例如需要缓存1小时，需填写 3600。
另外，还可以填写 epoch 或 max
，与 nginx 中语义相同。 | - | 配置缓存的最大时间 | -# 配置示例 +## 配置示例 1. 缓存后缀为 `jpg`, `png`, `jpeg` 的文件，缓存时间为一小时 ```yaml suffix: jpg|png|jpeg diff --git a/plugins/wasm-go/extensions/cache-control/README_EN.md b/plugins/wasm-go/extensions/cache-control/README_EN.md new file mode 100644 index 0000000000..d2e0d29748 --- /dev/null +++ b/plugins/wasm-go/extensions/cache-control/README_EN.md @@ -0,0 +1,33 @@ +--- +title: Browser Cache Control +keywords: [higress, browser cache control] +description: Browser cache control plugin configuration reference +--- +## Function Description +The `cache-control` plugin implements adding `Expires` and `Cache-Control` headers to the response based on the URL file extensions, making it easier for the browser to cache files with specific extensions, such as `jpg`, `png`, and other image files. + +## Runtime Attributes +Plugin execution phase: `Authentication Phase` +Plugin execution priority: `420` + +## Configuration Fields +| Name | Data Type | Requirements | Default Value | Description | +|-----------|-------------|----------------------------------------------------------------------------------------------------------|---------------|-----------------------------------| +| suffix | string | Optional, indicates the file extensions to match, such as `jpg`, `png`, etc.
If multiple extensions are needed, separate them with `\|`, for example `png\|jpg`.
If not specified, it matches all extensions. | - | Configures the request file extensions to match | +| expires | string | Required, indicates the maximum caching time.
When the input string is a number, the unit is seconds; for example, if you want to cache for 1 hour, enter 3600.
You can also enter epoch or max
, with the same semantics as in nginx. | - | Configures the maximum caching time | + +## Configuration Example +1. Cache files with extensions `jpg`, `png`, `jpeg`, with a caching time of one hour +```yaml +suffix: jpg|png|jpeg +expires: 3600 +``` +With this configuration, the following requests will have `Expires` and `Cache-Control` fields added to the response headers, with an expiration time of 1 hour later. +```bash +curl http://example.com/test.png +curl http://example.com/test.jpg +``` +2. Cache all files, with a maximum caching time of `"Thu, 31 Dec 2037 23:55:55 GMT"` +```yaml +expires: max +``` diff --git a/plugins/wasm-go/extensions/cluster-key-rate-limit/README.md b/plugins/wasm-go/extensions/cluster-key-rate-limit/README.md index e7c51afa0f..883e2535c2 100644 --- a/plugins/wasm-go/extensions/cluster-key-rate-limit/README.md +++ b/plugins/wasm-go/extensions/cluster-key-rate-limit/README.md @@ -9,6 +9,11 @@ description: 基于 Key 集群限流插件配置参考 `cluster-key-rate-limit` 插件基于 Redis 实现集群限流，适用于需要跨多个 Higress Gateway 实例实现全局一致速率限制的场景。 限流所使用的 Key 可以来源于 URL 参数、HTTP 请求头、客户端 IP 地址、消费者名称或 Cookie 中的 Key。 +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`20` + ## 配置说明 | 配置项 | 类型 | 必填 | 默认值 | 说明 | diff --git a/plugins/wasm-go/extensions/cluster-key-rate-limit/README_EN.md b/plugins/wasm-go/extensions/cluster-key-rate-limit/README_EN.md new file mode 100644 index 0000000000..4a4dcf8633 --- /dev/null +++ b/plugins/wasm-go/extensions/cluster-key-rate-limit/README_EN.md @@ -0,0 +1,183 @@ +--- +title: Key-Based Cluster Rate Limiting +keywords: [higress, rate-limit] +description: Configuration reference for the Key-Based Cluster Rate Limiting plugin +--- +## Function Description +The `cluster-key-rate-limit` plugin implements cluster rate limiting based on Redis, suitable for scenarios that require global consistent rate limiting across multiple Higress Gateway instances. + +The Key used for rate limiting can originate from URL parameters, HTTP request headers, client IP addresses, consumer names, or keys in cookies. + +## Execution Attributes +Plugin Execution Phase: `default phase` +Plugin Execution Priority: `20` + +## Configuration Description +| Configuration Item | Type | Required | Default Value | Description | +|---------------------------|---------------|----------|---------------|-------------------------------------------------------------------------------------------| +| rule_name | string | Yes | - | The name of the rate limiting rule. The Redis key is constructed using rule name + rate limit type + limit key name + actual value of the limit key. | +| rule_items | array of object| Yes | - | Rate limiting rule items. The first matching `rule_item` based on the order under `rule_items` will trigger the rate limiting, and subsequent rules will be ignored. | +| show_limit_quota_header | bool | No | false | Whether to display `X-RateLimit-Limit` (total requests allowed) and `X-RateLimit-Remaining` (remaining requests that can be sent) in the response headers. | +| rejected_code | int | No | 429 | HTTP status code returned when a request is rate limited. | +| rejected_msg | string | No | Too many requests | Response body returned when a request is rate limited. | +| redis | object | Yes | - | Redis related configuration. | + +Description of configuration fields for each item in `rule_items`. +| Configuration Item | Type | Required | Default Value | Description | +|---------------------------|---------------|------------------------|---------------|-------------------------------------------------------------------------------------------------------| +| limit_by_header | string | No, one of `limit_by_*` | - | The name of the HTTP request header from which to retrieve the rate limiting key value. | +| limit_by_param | string | No, one of `limit_by_*` | - | The name of the URL parameter from which to retrieve the rate limiting key value. | +| limit_by_consumer | string | No, one of `limit_by_*` | - | Applies rate limiting based on consumer name without needing to add an actual value. | +| limit_by_cookie | string | No, one of `limit_by_*` | - | The name of the key in the Cookie from which to retrieve the rate limiting key value. | +| limit_by_per_header | string | No, one of `limit_by_*` | - | Matches specific HTTP request headers according to the rules and calculates rate limits for each header. The `limit_keys` configuration supports regular expressions or `*`. | +| limit_by_per_param | string | No, one of `limit_by_*` | - | Matches specific URL parameters according to the rules and calculates rate limits for each parameter. The `limit_keys` configuration supports regular expressions or `*`. | +| limit_by_per_consumer | string | No, one of `limit_by_*` | - | Matches specific consumers according to the rules and calculates rate limits for each consumer. The `limit_keys` configuration supports regular expressions or `*`. | +| limit_by_per_cookie | string | No, one of `limit_by_*` | - | Matches specific cookies according to the rules and calculates rate limits for each cookie. The `limit_keys` configuration supports regular expressions or `*`. | +| limit_by_per_ip | string | No, one of `limit_by_*` | - | Matches specific IPs according to the rules and calculates rate limits for each IP. Retrieve via IP parameter name from request headers, defined as `from-header-{header name}`, e.g., `from-header-x-forwarded-for`. To get the remote socket IP directly, use `from-remote-addr`. | +| limit_keys | array of object | Yes | - | Configures the limit counts after matching key values. | + +Description of configuration fields for each item in `limit_keys`. +| Configuration Item | Type | Required | Default Value | Description | +|---------------------------|---------------|------------------------------------------------------------------|---------------|--------------------------------------------------------------------| +| key | string | Yes | - | Matched key value; types `limit_by_per_header`, `limit_by_per_param`, `limit_by_per_consumer`, `limit_by_per_cookie` support regular expression configurations (starting with regexp: followed by a regular expression) or `*` (representing all), e.g., `regexp:^d.*` (all strings starting with d); `limit_by_per_ip` supports configuring IP addresses or IP segments. | +| query_per_second | int | No, one of `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day` is optional. | - | Allowed number of requests per second. | +| query_per_minute | int | No, one of `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day` is optional. | - | Allowed number of requests per minute. | +| query_per_hour | int | No, one of `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day` is optional. | - | Allowed number of requests per hour. | +| query_per_day | int | No, one of `query_per_second`, `query_per_minute`, `query_per_hour`, `query_per_day` is optional. | - | Allowed number of requests per day. | + +Description of configuration fields for each item in `redis`. +| Configuration Item | Type | Required | Default Value | Description | +|---------------------------|---------------|----------|------------------------------------------------------------|---------------------------------------------------------------------------| +| service_name | string | Required | - | Full FQDN name of the Redis service, including service type, e.g., my-redis.dns, redis.my-ns.svc.cluster.local. | +| service_port | int | No | 80 for static services; otherwise 6379 | Service port for the Redis service. | +| username | string | No | - | Redis username. | +| password | string | No | - | Redis password. | +| timeout | int | No | 1000 | Redis connection timeout in milliseconds. | + +## Configuration Examples + +### Distinguish rate limiting based on the request parameter apikey +```yaml +rule_name: default_rule +rule_items: +- limit_by_param: apikey + limit_keys: + - key: 9a342114-ba8a-11ec-b1bf-00163e1250b5 + query_per_minute: 10 + - key: a6a6d7f2-ba8a-11ec-bec2-00163e1250b5 + query_per_hour: 100 +- limit_by_per_param: apikey + limit_keys: + # Regular expression, matches all strings starting with a, each apikey corresponds to 10qds. + - key: "regexp:^a.*" + query_per_second: 10 + # Regular expression, matches all strings starting with b, each apikey corresponds to 100qd. + - key: "regexp:^b.*" + query_per_minute: 100 + # As a fallback, matches all requests, each apikey corresponds to 1000qdh. + - key: "*" + query_per_hour: 1000 +redis: + service_name: redis.static +show_limit_quota_header: true +``` + +### Distinguish rate limiting based on the header x-ca-key +```yaml +rule_name: default_rule +rule_items: +- limit_by_header: x-ca-key + limit_keys: + - key: 102234 + query_per_minute: 10 + - key: 308239 + query_per_hour: 10 +- limit_by_per_header: x-ca-key + limit_keys: + # Regular expression, matches all strings starting with a, each apikey corresponds to 10qds. + - key: "regexp:^a.*" + query_per_second: 10 + # Regular expression, matches all strings starting with b, each apikey corresponds to 100qd. + - key: "regexp:^b.*" + query_per_minute: 100 + # As a fallback, matches all requests, each apikey corresponds to 1000qdh. + - key: "*" + query_per_hour: 1000 +redis: + service_name: redis.static +show_limit_quota_header: true +``` + +### Distinguish rate limiting based on the client IP from the request header x-forwarded-for +```yaml +rule_name: default_rule +rule_items: +- limit_by_per_ip: from-header-x-forwarded-for + limit_keys: + # Exact IP + - key: 1.1.1.1 + query_per_day: 10 + # IP segment, for IPs matching this segment, each IP corresponds to 100qpd. + - key: 1.1.1.0/24 + query_per_day: 100 + # As a fallback, defaults to 1000 qpd for each IP. + - key: 0.0.0.0/0 + query_per_day: 1000 +redis: + service_name: redis.static +show_limit_quota_header: true +``` + +### Distinguish rate limiting based on consumers +```yaml +rule_name: default_rule +rule_items: +- limit_by_consumer: '' + limit_keys: + - key: consumer1 + query_per_second: 10 + - key: consumer2 + query_per_hour: 100 +- limit_by_per_consumer: '' + limit_keys: + # Regular expression, matches all strings starting with a, each consumer corresponds to 10qds. + - key: "regexp:^a.*" + query_per_second: 10 + # Regular expression, matches all strings starting with b, each consumer corresponds to 100qd. + - key: "regexp:^b.*" + query_per_minute: 100 + # As a fallback, matches all requests, each consumer corresponds to 1000qdh. + - key: "*" + query_per_hour: 1000 +redis: + service_name: redis.static +show_limit_quota_header: true +``` + +### Distinguish rate limiting based on key-value pairs in cookies +```yaml +rule_name: default_rule +rule_items: + - limit_by_cookie: key1 + limit_keys: + - key: value1 + query_per_minute: 10 + - key: value2 + query_per_hour: 100 + - limit_by_per_cookie: key1 + limit_keys: + # Regular expression, matches all strings starting with a, each cookie's value corresponds to 10qds. + - key: "regexp:^a.*" + query_per_second: 10 + # Regular expression, matches all strings starting with b, each cookie's value corresponds to 100qd. + - key: "regexp:^b.*" + query_per_minute: 100 + # As a fallback, matches all requests, each cookie's value corresponds to 1000qdh. + - key: "*" + query_per_hour: 1000 +rejected_code: 200 +rejected_msg: '{"code":-1,"msg":"Too many requests"}' +redis: + service_name: redis.static +show_limit_quota_header: true +``` diff --git a/plugins/wasm-go/extensions/cors/README.md b/plugins/wasm-go/extensions/cors/README.md index 0015481e0c..51f36c26a0 100644 --- a/plugins/wasm-go/extensions/cors/README.md +++ b/plugins/wasm-go/extensions/cors/README.md @@ -1,27 +1,38 @@ -# 功能说明 +--- +title: 跨域资源共享 +keywords: [higress,cors] +description: 跨域资源共享插件配置参考 +--- + +## 功能说明 `cors` 插件可以为服务端启用 CORS（Cross-Origin Resource Sharing，跨域资源共享）的返回 http 响应头。 -# 配置字段 +## 运行属性 + +插件执行阶段：`授权阶段` +插件执行优先级：`340` + +## 配置字段 -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -|-----------------------|-----------------|-------|---------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| allow_origins | array of string | 选填 | * | 允许跨域访问的 Origin，格式为 scheme://host:port，示例如 http://example.com:8081。当 allow_credentials 为 false 时，可以使用 * 来表示允许所有 Origin 通过 | -| allow_origin_patterns | array of string | 选填 | - | 允许跨域访问的 Origin 模式匹配， 用 * 匹配域名或者端口，
比如 http://*.example.com -- 匹配域名， http://*.example.com:[8080,9090] -- 匹配域名和指定端口， http://*.example.com:[*] -- 匹配域名和所有端口。单独 * 表示匹配所有域名和端口 | -| allow_methods | array of string | 选填 | GET, PUT, POST, DELETE, PATCH, OPTIONS | 允许跨域访问的 Method，比如：GET，POST 等。可以使用 * 来表示允许所有 Method。 | -| allow_headers | array of string | 选填 | DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With，
If-Modified-Since,Cache-Control,Content-Type,Authorization | 允许跨域访问时请求方携带哪些非 CORS 规范以外的 Header。可以使用 * 来表示允许任意 Header。 | -| expose_headers | array of string | 选填 | - | 允许跨域访问时响应方携带哪些非 CORS 规范以外的 Header。可以使用 * 来表示允许任意 Header。 | -| allow_credentials | bool | 选填 | false | 是否允许跨域访问的请求方携带凭据（如 Cookie 等）。根据 CORS 规范，如果设置该选项为 true，在 allow_origins 不能使用 *， 替换成使用 allow_origin_patterns * | -| max_age | number | 选填 | 86400秒 | 浏览器缓存 CORS 结果的最大时间，单位为秒。
在这个时间范围内，浏览器会复用上一次的检查结果 | +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +|-----------------------|-----------------|----------|----------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| allow_origins | array of string | 选填 | * | 允许跨域访问的 Origin，格式为 scheme://host:port，示例如 http://example.com:8081。当 allow_credentials 为 false 时，可以使用 * 来表示允许所有 Origin 通过 | +| allow_origin_patterns | array of string | 选填 | - | 允许跨域访问的 Origin 模式匹配， 用 * 匹配域名或者端口，
比如 http://*.example.com -- 匹配域名， http://*.example.com:[8080,9090] -- 匹配域名和指定端口， http://*.example.com:[*] -- 匹配域名和所有端口。单独 * 表示匹配所有域名和端口 | +| allow_methods | array of string | 选填 | GET, PUT, POST, DELETE, PATCH, OPTIONS | 允许跨域访问的 Method，比如：GET，POST 等。可以使用 * 来表示允许所有 Method。 | +| allow_headers | array of string | 选填 | DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With，
If-Modified-Since,Cache-Control,Content-Type,Authorization | 允许跨域访问时请求方携带哪些非 CORS 规范以外的 Header。可以使用 * 来表示允许任意 Header。 | +| expose_headers | array of string | 选填 | - | 允许跨域访问时响应方携带哪些非 CORS 规范以外的 Header。可以使用 * 来表示允许任意 Header。 | +| allow_credentials | bool | 选填 | false | 是否允许跨域访问的请求方携带凭据（如 Cookie 等）。根据 CORS 规范，如果设置该选项为 true，在 allow_origins 不能使用 *， 替换成使用 allow_origin_patterns * | +| max_age | number | 选填 | 86400秒 | 浏览器缓存 CORS 结果的最大时间，单位为秒。
在这个时间范围内，浏览器会复用上一次的检查结果 | > 注意 > * allow_credentials 是一个很敏感的选项，请谨慎开启。开启之后，allow_credentials 和 allow_origins 为 * 不能同时使用，同时设置时， allow_origins 值为 "*" 生效。 > * allow_origins 和 allow_origin_patterns 可以同时设置， 先检查 allow_origins 是否匹配，然后再检查 allow_origin_patterns 是否匹配 > * 非法 CORS 请求， HTTP 状态码返回是 403， 返回体内容为 "Invalid CORS request" -# 配置示例 +## 配置示例 -## 允许所有跨域访问, 不允许请求方携带凭据 +### 允许所有跨域访问, 不允许请求方携带凭据 ```yaml allow_origins: - '*' @@ -35,7 +46,7 @@ allow_credentials: false max_age: 7200 ``` -## 允许所有跨域访问,同时允许请求方携带凭据 +### 允许所有跨域访问,同时允许请求方携带凭据 ```yaml allow_origin_patterns: - '*' @@ -49,7 +60,7 @@ allow_credentials: true max_age: 7200 ``` -## 允许特定子域,特定方法，特定请求头跨域访问，同时允许请求方携带凭据 +### 允许特定子域,特定方法，特定请求头跨域访问，同时允许请求方携带凭据 ```yaml allow_origin_patterns: - http://*.example.com @@ -69,9 +80,9 @@ allow_credentials: true max_age: 7200 ``` -# 测试 +## 测试 -## 测试配置 +### 测试配置 ```yaml apiVersion: networking.higress.io/v1 @@ -143,9 +154,9 @@ spec: imagePullPolicy: Always ``` -## 请求测试 +### 请求测试 -### 简单请求 +#### 简单请求 ```shell curl -v -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example.com" http://127.0.0.1/anything/get\?foo\=1 @@ -156,7 +167,7 @@ curl -v -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example > access-control-allow-credentials: true ``` -### 预检请求 +#### 预检请求 ```shell curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example.com" -H "Access-Control-Request-Method: POST" -H "Access-Control-Request-Headers: Content-Type, Token" http://127.0.0.1/anything/get\?foo\=1 @@ -177,7 +188,7 @@ curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org:9090" -H "Host: http * Closing connection 0 ``` -### 非法 CORS Origin 预检请求 +#### 非法 CORS Origin 预检请求 ```shell curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org" -H "Host: httpbin.example.com" -H "Access-Control-Request-Method: GET" http://127.0.0.1/anything/get\?foo\=1 @@ -193,7 +204,7 @@ curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org" -H "Host: httpbin.e Invalid CORS request ``` -### 非法 CORS Method 预检请求 +#### 非法 CORS Method 预检请求 ```shell curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example.com" -H "Access-Control-Request-Method: DELETE" http://127.0.0.1/anything/get\?foo\=1 @@ -209,7 +220,7 @@ curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org:9090" -H "Host: http Invalid CORS request ``` -### 非法 CORS Header 预检请求 +#### 非法 CORS Header 预检请求 ```shell curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example.com" -H "Access-Control-Request-Method: GET" -H "Access-Control-Request-Headers: TokenView" http://127.0.0.1/anything/get\?foo\=1 @@ -225,6 +236,6 @@ Invalid CORS request Invalid CORS request ``` -# 参考文档 +## 参考文档 - https://www.ruanyifeng.com/blog/2016/04/cors.html - https://fetch.spec.whatwg.org/#http-cors-protocol diff --git a/plugins/wasm-go/extensions/cors/README_EN.md b/plugins/wasm-go/extensions/cors/README_EN.md new file mode 100644 index 0000000000..45abc08cf0 --- /dev/null +++ b/plugins/wasm-go/extensions/cors/README_EN.md @@ -0,0 +1,223 @@ +--- +title: Cross-Origin Resource Sharing +keywords: [higress,cors] +description: Cross-Origin Resource Sharing plugin configuration reference +--- +## Function Description +The `cors` plugin can enable CORS (Cross-Origin Resource Sharing) HTTP response headers for the server. + +## Execution Attributes +Plugin execution phase: `Authorization Phase` +Plugin execution priority: `340` + +## Configuration Fields +| Name | Data Type | Required | Default Value | Description | +|-----------------------|------------------|----------|-----------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| allow_origins | array of string | Optional | * | Allowed Origins for cross-origin access, formatted as scheme://host:port, for example, http://example.com:8081. When allow_credentials is false, * can be used to allow all Origins through. | +| allow_origin_patterns | array of string | Optional | - | Patterns for matching allowed Origins for cross-origin access, using * to match domain or port,
for example http://*.example.com -- matches domain, http://*.example.com:[8080,9090] -- matches domain and specified ports, http://*.example.com:[*] -- matches domain and all ports. A single * indicates matching all domains and ports. | +| allow_methods | array of string | Optional | GET, PUT, POST, DELETE, PATCH, OPTIONS | Allowed Methods for cross-origin access, for example: GET, POST, etc. * can be used to indicate all Methods are allowed. | +| allow_headers | array of string | Optional | DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,
If-Modified-Since,Cache-Control,Content-Type,Authorization | Allowed Headers for the requester to carry that are not part of CORS specifications during cross-origin access. * can be used to indicate any Header is allowed. | +| expose_headers | array of string | Optional | - | Allowed Headers for the responder to carry that are not part of CORS specifications during cross-origin access. * can be used to indicate any Header is allowed. | +| allow_credentials | bool | Optional | false | Whether to allow the requester to carry credentials (e.g. Cookies) during cross-origin access. According to CORS specifications, if this option is set to true, * cannot be used for allow_origins, replace it with allow_origin_patterns. | +| max_age | number | Optional | 86400 seconds | Maximum time for browsers to cache CORS results, in seconds.
Within this time frame, browsers will reuse the previous inspection results. | +> Note +> * allow_credentials is a very sensitive option, please enable it with caution. Once enabled, allow_credentials and allow_origins cannot both be *, if both are set, the allow_origins value of "*" takes effect. +> * allow_origins and allow_origin_patterns can be set simultaneously. First, check if allow_origins matches, then check if allow_origin_patterns matches. +> * Illegal CORS requests will return HTTP status code 403, with the response body content as "Invalid CORS request". + +## Configuration Examples +### Allow all cross-origin access, without allowing the requester to carry credentials +```yaml +allow_origins: + - '*' +allow_methods: + - '*' +allow_headers: + - '*' +expose_headers: + - '*' +allow_credentials: false +max_age: 7200 +``` + +### Allow all cross-origin access, while allowing the requester to carry credentials +```yaml +allow_origin_patterns: + - '*' +allow_methods: + - '*' +allow_headers: + - '*' +expose_headers: + - '*' +allow_credentials: true +max_age: 7200 +``` + +### Allow specific subdomains, specific methods, and specific request headers for cross-origin access, while allowing the requester to carry credentials +```yaml +allow_origin_patterns: + - http://*.example.com + - http://*.example.org:[8080,9090] +allow_methods: + - GET + - PUT + - POST + - DELETE +allow_headers: + - Token + - Content-Type + - Authorization +expose_headers: + - '*' +allow_credentials: true +max_age: 7200 +``` + +## Testing +### Test Configuration +```yaml +apiVersion: networking.higress.io/v1 +kind: McpBridge +metadata: + name: mcp-cors-httpbin + namespace: higress-system +spec: + registries: + - domain: httpbin.org + name: httpbin + port: 80 + type: dns +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + annotations: + higress.io/destination: httpbin.dns + higress.io/upstream-vhost: "httpbin.org" + higress.io/backend-protocol: HTTP + name: ingress-cors-httpbin + namespace: higress-system +spec: + ingressClassName: higress + rules: + - host: httpbin.example.com + http: + paths: + - backend: + resource: + apiGroup: networking.higress.io + kind: McpBridge + name: mcp-cors-httpbin + path: / + pathType: Prefix +--- +apiVersion: extensions.higress.io/v1alpha1 +kind: WasmPlugin +metadata: + name: wasm-cors-httpbin + namespace: higress-system +spec: + defaultConfigDisable: true + matchRules: + - config: + allow_origins: + - http://httpbin.example.net + allow_origin_patterns: + - http://*.example.com:[*] + - http://*.example.org:[9090,8080] + allow_methods: + - GET + - POST + - PATCH + allow_headers: + - Content-Type + - Token + - Authorization + expose_headers: + - X-Custom-Header + - X-Env-UTM + allow_credentials: true + max_age: 3600 + configDisable: false + ingress: + - ingress-cors-httpbin + url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/cors:1.0.0 + imagePullPolicy: Always +``` + +### Request Testing +#### Simple Request +```shell +curl -v -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example.com" http://127.0.0.1/anything/get\?foo\=1 +< HTTP/1.1 200 OK +> x-cors-version: 1.0.0 +> access-control-allow-origin: http://httpbin2.example.org:9090 +> access-control-expose-headers: X-Custom-Header,X-Env-UTM +> access-control-allow-credentials: true +``` + +#### Preflight Request +```shell +curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example.com" -H "Access-Control-Request-Method: POST" -H "Access-Control-Request-Headers: Content-Type, Token" http://127.0.0.1/anything/get\?foo\=1 +< HTTP/1.1 200 OK +< x-cors-version: 1.0.0 +< access-control-allow-origin: http://httpbin2.example.org:9090 +< access-control-allow-methods: GET,POST,PATCH +< access-control-allow-headers: Content-Type,Token,Authorization +< access-control-expose-headers: X-Custom-Header,X-Env-UTM +< access-control-allow-credentials: true +< access-control-max-age: 3600 +< date: Tue, 23 May 2023 11:41:28 GMT +< server: istio-envoy +< content-length: 0 +< +* Connection #0 to host 127.0.0.1 left intact +* Closing connection 0 +``` + +#### Illegal CORS Origin Preflight Request +```shell +curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org" -H "Host: httpbin.example.com" -H "Access-Control-Request-Method: GET" http://127.0.0.1/anything/get\?foo\=1 + HTTP/1.1 403 Forbidden +< content-length: 70 +< content-type: text/plain +< x-cors-version: 1.0.0 +< date: Tue, 23 May 2023 11:27:01 GMT +< server: istio-envoy +< +* Connection #0 to host 127.0.0.1 left intact +Invalid CORS request +``` + +#### Illegal CORS Method Preflight Request +```shell +curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example.com" -H "Access-Control-Request-Method: DELETE" http://127.0.0.1/anything/get\?foo\=1 +< HTTP/1.1 403 Forbidden +< content-length: 49 +< content-type: text/plain +< x-cors-version: 1.0.0 +< date: Tue, 23 May 2023 11:28:51 GMT +< server: istio-envoy +< +* Connection #0 to host 127.0.0.1 left intact +Invalid CORS request +``` + +#### Illegal CORS Header Preflight Request +```shell +curl -v -X OPTIONS -H "Origin: http://httpbin2.example.org:9090" -H "Host: httpbin.example.com" -H "Access-Control-Request-Method: GET" -H "Access-Control-Request-Headers: TokenView" http://127.0.0.1/anything/get\?foo\=1 +< HTTP/1.1 403 Forbidden +< content-length: 52 +< content-type: text/plain +< x-cors-version: 1.0.0 +< date: Tue, 23 May 2023 11:31:03 GMT +< server: istio-envoy +< +* Connection #0 to host 127.0.0.1 left intact +Invalid CORS request +``` + +## Reference Documents +- https://www.ruanyifeng.com/blog/2016/04/cors.html +- https://fetch.spec.whatwg.org/#http-cors-protocol diff --git a/plugins/wasm-go/extensions/custom-response/README.md b/plugins/wasm-go/extensions/custom-response/README.md index ffeb981668..e59863441d 100644 --- a/plugins/wasm-go/extensions/custom-response/README.md +++ b/plugins/wasm-go/extensions/custom-response/README.md @@ -1,11 +1,19 @@ -

- English | 中文 -

+--- +title: 自定义应答 +keywords: [higress,customn response] +description: 自定义应答插件配置参考 +--- -# 功能说明 + +## 功能说明 `custom-response`插件支持配置自定义的响应，包括自定义 HTTP 应答状态码、HTTP 应答头，以及 HTTP 应答 Body。可以用于 Mock 响应，也可以用于判断特定状态码后给出自定义应答，例如在触发网关限流策略时实现自定义响应。 -# 配置字段 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`910` + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- | -------- | -------- | @@ -14,9 +22,9 @@ | body | string | 选填 | - | 自定义 HTTP 应答 Body | | enable_on_status | array of number | 选填 | - | 匹配原始状态码，生成自定义响应，不填写时，不判断原始状态码 | -# 配置示例 +## 配置示例 -## Mock 应答场景 +### Mock 应答场景 ```yaml status_code: 200 @@ -38,7 +46,7 @@ Content-Length: 17 {"hello":"world"} ``` -## 触发限流时自定义响应 +### 触发限流时自定义响应 ```yaml enable_on_status: @@ -58,27 +66,3 @@ Location: https://example.com 从而实现基于浏览器 302 重定向机制，将限流后的用户引导到其他页面，比如可以是一个 CDN 上的静态页面。 如果希望触发限流时，正常返回其他应答，参考 Mock 应答场景配置相应的字段即可。 - -## 对特定路由或域名开启 -```yaml -# 使用 matchRules 字段进行细粒度规则配置 -matchRules: -# 规则一：按 Ingress 名称匹配生效 -- ingress: - - default/foo - - default/bar - body: "{\"hello\":\"world\"}" -# 规则二：按域名匹配生效 -- domain: - - "*.example.com" - - test.com - enable_on_status: - - 429 - status_code: 200 - headers: - - Content-Type=application/json - body: "{\"errmsg\": \"rate limited\"}" -``` -此例 `ingress` 中指定的 `default/foo` 和 `default/bar` 对应 default 命名空间下名为 foo 和 bar 的 Ingress，当匹配到这两个 Ingress 时，将使用此段配置； -此例 `domain` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将使用此段配置； -配置的匹配生效顺序，将按照 `matchRules` 下规则的排列顺序，匹配第一个规则后生效对应配置，后续规则将被忽略。 diff --git a/plugins/wasm-go/extensions/custom-response/README_EN.md b/plugins/wasm-go/extensions/custom-response/README_EN.md index d8df25153c..365696e20a 100644 --- a/plugins/wasm-go/extensions/custom-response/README_EN.md +++ b/plugins/wasm-go/extensions/custom-response/README_EN.md @@ -1,84 +1,54 @@ -

- English | 中文 -

+--- +title: Custom Response +keywords: [higress, custom response] +description: Custom response plugin configuration reference +--- +## Function Description +The `custom-response` plugin supports the configuration of custom responses, including custom HTTP response status codes, HTTP response headers, and HTTP response bodies. It can be used for Mock responses or for providing custom responses based on specific status codes, such as implementing custom responses when triggering the gateway rate-limiting policy. -# Description -`custom-response` plugin implements a function of sending custom responses, including custom HTTP response status codes, HTTP response headers and HTTP response body, which can be used in the scenarios of response mocking and sending a custom response for specific status codes, such as customizing the response for rate-limited requests. +## Running Attributes +Plugin Execution Phase: `Authentication Phase` -# Configuration Fields +Plugin Execution Priority: `910` -| Name | Type | Requirement | Default Value | Description | +## Configuration Fields +| Name | Data Type | Requirements | Default Value | Description | | -------- | -------- | -------- | -------- | -------- | | status_code | number | Optional | 200 | Custom HTTP response status code | -| headers | array of string | Optional | - | Custom HTTP response header. Key and value shall be separated using `=`. | +| headers | array of string | Optional | - | Custom HTTP response headers, keys and values separated by `=` | | body | string | Optional | - | Custom HTTP response body | -| enable_on_status | array of number | Optional | - | The original response status code to match. Generate the custom response only the actual status code matches the configuration. Ignore the status code match if left unconfigured. | - -# Configuration Samples - -## Mock Responses +| enable_on_status | array of number | Optional | - | Match original status codes to generate custom responses; if not specified, the original status code is not checked | +## Configuration Example +### Mock Response Scenario ```yaml status_code: 200 headers: - Content-Type=application/json - Hello=World body: "{\"hello\":\"world\"}" - ``` - -According to the configuration above, all the requests will get the following custom response: - +With this configuration, the request will return the following custom response: ```text HTTP/1.1 200 OK Content-Type: application/json Hello: World Content-Length: 17 - {"hello":"world"} ``` - -## Send a Custom Response when Rate-Limited - +### Custom Response on Rate Limiting ```yaml -enable_on_status: +enable_on_status: - 429 status_code: 302 headers: - Location=https://example.com ``` - -When rate-limited, normally gateway will return a status code of `429` . Now, rate-limited requests will get the following custom response: - +When the gateway rate limiting is triggered, it generally returns the `429` status code, and the request will return the following custom response: ```text HTTP/1.1 302 Found Location: https://example.com ``` +This achieves the goal of redirecting users who have been rate-limited to another page based on the browser's 302 redirect mechanism, which could be a static page on a CDN. -So based on the 302 redirecting mechanism provided by browsers, this can redirect rate-limited users to other pages, for example, a static page hosted on CDN. - -If you'd like to send other responses when rate-limited, please add other fields into the configuration, referring to the Mock Responses scenario. - -## Only Enabled for Specific Routes or Domains -```yaml -# Use matchRules field for fine-grained rule configurations -matchRules: -# Rule 1: Match by Ingress name -- ingress: - - default/foo - - default/bar - body: "{\"hello\":\"world\"}" -# Rule 2: Match by domain -- domain: - - "*.example.com" - - test.com - enable_on_status: - - 429 - status_code: 200 - headers: - - Content-Type=application/json - body: "{\"errmsg\": \"rate limited\"}" -``` -In the rule sample of `ingress`, `default/foo` and `default/bar` are the Ingresses named foo and bar in the default namespace. When the current Ingress names matches the configuration, the rule following shall be applied. -In the rule sample of `domain`, `*.example.com` and `test.com` are the domain names used for request matching. When the current domain name matches the configuration, the rule following shall be applied. -All rules shall be checked following the order of items in the `matchRules` field, The first matched rule will be applied. All remained will be ignored. +If you wish to return other responses normally when rate limiting is triggered, just refer to the Mock response scenario to configure the relevant fields accordingly. diff --git a/plugins/wasm-go/extensions/de-graphql/README.md b/plugins/wasm-go/extensions/de-graphql/README.md index 647a61206f..6fe3815cdb 100644 --- a/plugins/wasm-go/extensions/de-graphql/README.md +++ b/plugins/wasm-go/extensions/de-graphql/README.md @@ -1,97 +1,17 @@ -# DeGraphQL - -## GraphQL - -### GraphQL 端点 - -REST API 有多个端点，GraphQL API 只有一个端点。 - -```shell -https://api.github.com/graphql -``` -### 与 GraphQL 通信 - -由于 GraphQL 操作由多行 JSON 组成，可以使用 curl 或任何其他采用 HTTP 的库。 - -在 REST 中，HTTP 谓词确定执行的操作。 在 GraphQL 中，执行查询要提供 JSON 请求体，因此 HTTP 谓词为 POST。 唯一的例外是内省查询，它是一种简单的 GET 到终结点查询。 - -### GraphQL POST 请求参数 - -标准的 GraphQL POST 请求情况如下： - -- 添加 HTTP 请求头： Content-Type: application/json -- 使用 JSON 格式的请求体 -- JSON 请求体包含三个字段 - - query：查询文档，必填 - - variables：变量，选填 - - operationName：操作名称，选填，查询文档有多个操作时必填 - -```json -{ - "query": "{viewer{name}}", - "operationName": "", - "variables": { - "name": "value" - } -} -``` - -### GraphQL 基本参数类型 - -- 基本参数类型包含： String, Int, Float, Boolean -- [类型]代表数组，例如：[Int]代表整型数组 -- GraphQL 基本参数传递 - - 小括号内定义形参，注意：参数需要定义类型 - - !（叹号）代表参数不能为空 - -```shell -query ($owner : String!, $name : String!) { - repository(owner: $owner, name: $name) { - name - forkCount - description - } -} -``` - - -### GitHub GraphQL 测试 - -使用 curl 命令查询 GraphQL， 用有效 JSON 请求体发出 POST 请求。 有效请求体必须包含一个名为 query 的字符串。 - -```shell - -curl https://api.github.com/graphql -X POST \ --H "Authorization: bearer " \ --d "{\"query\": \"query { viewer { login }}\"}" - -{ - "data": { - "viewer": { - "login": "2456868764" - } - } -} -``` +--- +title: DeGraphQL +keywords: [higress, DeGraphQL] +description: DeGraphQL 插件配置参考 +--- -```shell -curl 'https://api.github.com/graphql' -X POST \ --H 'Authorization: bearer ' \ --d '{"query":"query ($owner: String!, $name: String!) {\n repository(owner: $owner, name: $name) {\n name\n forkCount\n description\n }\n}\n","variables":{"owner":"2456868764","name":"higress"}}' +## 功能说明 -{ - "data": { - "repository": { - "name": "higress", - "forkCount": 149, - "description": "Next-generation Cloud Native Gateway | 下一代云原生网关" - } - } -} -``` +`de-graphql`插件通过将URIs映射到GraphQL查询，从而可以将GraphQL上游转换为传统服务进行访问 +## 运行属性 -## DeGraphQL 插件 +插件执行阶段：`认证阶段` +插件执行优先级：`430` ### 参数配置 @@ -182,6 +102,96 @@ curl "http://localhost/api?owner=alibaba&name=higress" -H "Authorization: Bearer } ``` +## GraphQL介绍 + +### GraphQL 端点 + +REST API 有多个端点，GraphQL API 只有一个端点。 + +```shell +https://api.github.com/graphql +``` +### 与 GraphQL 通信 + +由于 GraphQL 操作由多行 JSON 组成，可以使用 curl 或任何其他采用 HTTP 的库。 + +在 REST 中，HTTP 谓词确定执行的操作。 在 GraphQL 中，执行查询要提供 JSON 请求体，因此 HTTP 谓词为 POST。 唯一的例外是内省查询，它是一种简单的 GET 到终结点查询。 + +### GraphQL POST 请求参数 + +标准的 GraphQL POST 请求情况如下： + +- 添加 HTTP 请求头： Content-Type: application/json +- 使用 JSON 格式的请求体 +- JSON 请求体包含三个字段 + - query：查询文档，必填 + - variables：变量，选填 + - operationName：操作名称，选填，查询文档有多个操作时必填 + +```json +{ + "query": "{viewer{name}}", + "operationName": "", + "variables": { + "name": "value" + } +} +``` + +### GraphQL 基本参数类型 + +- 基本参数类型包含： String, Int, Float, Boolean +- [类型]代表数组，例如：[Int]代表整型数组 +- GraphQL 基本参数传递 + - 小括号内定义形参，注意：参数需要定义类型 + - !（叹号）代表参数不能为空 + +```shell +query ($owner : String!, $name : String!) { + repository(owner: $owner, name: $name) { + name + forkCount + description + } +} +``` + + +### GitHub GraphQL 测试 + +使用 curl 命令查询 GraphQL， 用有效 JSON 请求体发出 POST 请求。 有效请求体必须包含一个名为 query 的字符串。 + +```shell + +curl https://api.github.com/graphql -X POST \ +-H "Authorization: bearer " \ +-d "{\"query\": \"query { viewer { login }}\"}" + +{ + "data": { + "viewer": { + "login": "2456868764" + } + } +} +``` + +```shell +curl 'https://api.github.com/graphql' -X POST \ +-H 'Authorization: bearer ' \ +-d '{"query":"query ($owner: String!, $name: String!) {\n repository(owner: $owner, name: $name) {\n name\n forkCount\n description\n }\n}\n","variables":{"owner":"2456868764","name":"higress"}}' + +{ + "data": { + "repository": { + "name": "higress", + "forkCount": 149, + "description": "Next-generation Cloud Native Gateway | 下一代云原生网关" + } + } +} +``` + ## 参考文档 - https://github.com/graphql/graphql-spec diff --git a/plugins/wasm-go/extensions/de-graphql/README_EN.md b/plugins/wasm-go/extensions/de-graphql/README_EN.md new file mode 100644 index 0000000000..715e645298 --- /dev/null +++ b/plugins/wasm-go/extensions/de-graphql/README_EN.md @@ -0,0 +1,199 @@ +--- +title: DeGraphQL +keywords: [higress, DeGraphQL] +description: DeGraphQL 插件配置参考 +--- + +## 功能说明 + +`de-graphql`插件通过将URIs映射到GraphQL查询，从而可以将GraphQL上游转换为传统服务进行访问 + +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`430` + +### 参数配置 + +| 参数 | 描述 | 默认 | +|:----------------|:------------------------|:-----------| +| `gql` | graphql 查询 | 不能为空 | +| `endpoint` | graphql 查询端点 | `/graphql` | +| `timeout` | 查询连接超时，单位毫秒 | `5000` | +| `domain` | 服务域名，当服务来源是dns配置 | | + +### 插件使用 + +https://github.com/alibaba/higress/issues/268 + +- 测试配置 +```yaml +apiVersion: networking.higress.io/v1 +kind: McpBridge +metadata: + name: default + namespace: higress-system +spec: + registries: + - domain: api.github.com + name: github + port: 443 + type: dns +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + annotations: + higress.io/destination: github.dns + higress.io/upstream-vhost: "api.github.com" + higress.io/backend-protocol: HTTPS + name: github-api + namespace: higress-system +spec: + ingressClassName: higress + rules: + - http: + paths: + - backend: + resource: + apiGroup: networking.higress.io + kind: McpBridge + name: default + path: /api + pathType: Prefix +--- +apiVersion: extensions.higress.io/v1alpha1 +kind: WasmPlugin +metadata: + name: de-graphql-github-api + namespace: higress-system +spec: + matchRules: + - ingress: + - github-api + config: + timeout: 5000 + endpoint: /graphql + domain: api.github.com + gql: | + query ($owner:String! $name:String!){ + repository(owner:$owner, name:$name) { + name + forkCount + description + } + } + url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/de-graphql:1.0.0 +``` + +- 测试结果 + +```shell +curl "http://localhost/api?owner=alibaba&name=higress" -H "Authorization: Bearer some-token" + +{ + "data": { + "repository": { + "description": "Next-generation Cloud Native Gateway", + "forkCount": 149, + "name": "higress" + } + } +} +``` + +## GraphQL介绍 + +### GraphQL 端点 + +REST API 有多个端点，GraphQL API 只有一个端点。 + +```shell +https://api.github.com/graphql +``` +### 与 GraphQL 通信 + +由于 GraphQL 操作由多行 JSON 组成，可以使用 curl 或任何其他采用 HTTP 的库。 + +在 REST 中，HTTP 谓词确定执行的操作。 在 GraphQL 中，执行查询要提供 JSON 请求体，因此 HTTP 谓词为 POST。 唯一的例外是内省查询，它是一种简单的 GET 到终结点查询。 + +### GraphQL POST 请求参数 + +标准的 GraphQL POST 请求情况如下： + +- 添加 HTTP 请求头： Content-Type: application/json +- 使用 JSON 格式的请求体 +- JSON 请求体包含三个字段 + - query：查询文档，必填 + - variables：变量，选填 + - operationName：操作名称，选填，查询文档有多个操作时必填 + +```json +{ + "query": "{viewer{name}}", + "operationName": "", + "variables": { + "name": "value" + } +} +``` + +### GraphQL 基本参数类型 + +- 基本参数类型包含： String, Int, Float, Boolean +- [类型]代表数组，例如：[Int]代表整型数组 +- GraphQL 基本参数传递 + - 小括号内定义形参，注意：参数需要定义类型 + - !（叹号）代表参数不能为空 + +```shell +query ($owner : String!, $name : String!) { + repository(owner: $owner, name: $name) { + name + forkCount + description + } +} +``` + + +### GitHub GraphQL 测试 + +使用 curl 命令查询 GraphQL， 用有效 JSON 请求体发出 POST 请求。 有效请求体必须包含一个名为 query 的字符串。 + +```shell + +curl https://api.github.com/graphql -X POST \ +-H "Authorization: bearer " \ +-d "{\"query\": \"query { viewer { login }}\"}" + +{ + "data": { + "viewer": { + "login": "2456868764" + } + } +} +``` + +```shell +curl 'https://api.github.com/graphql' -X POST \ +-H 'Authorization: bearer ' \ +-d '{"query":"query ($owner: String!, $name: String!) {\n repository(owner: $owner, name: $name) {\n name\n forkCount\n description\n }\n}\n","variables":{"owner":"2456868764","name":"higress"}}' + +{ + "data": { + "repository": { + "name": "higress", + "forkCount": 149, + "description": "Next-generation Cloud Native Gateway | 下一代云原生网关" + } + } +} +``` + +## 参考文档 + +- https://github.com/graphql/graphql-spec +- https://docs.github.com/zh/graphql/guides/forming-calls-with-graphql +- https://github.com/altair-graphql/altair diff --git a/plugins/wasm-go/extensions/ext-auth/README.md b/plugins/wasm-go/extensions/ext-auth/README.md index 24dd39e570..4b16734aa4 100644 --- a/plugins/wasm-go/extensions/ext-auth/README.md +++ b/plugins/wasm-go/extensions/ext-auth/README.md @@ -8,6 +8,10 @@ description: Ext 认证插件实现了调用外部授权服务进行认证鉴权 `ext-auth` 插件实现了向外部授权服务发送鉴权请求，以检查客户端请求是否得到授权。该插件实现时参考了Envoy原生的[ext_authz filter](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_authz_filter)，实现了原生filter中对接HTTP服务的部分能力 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`360` ## 配置字段 @@ -281,4 +285,4 @@ Content-Length: 0 | x-forwarded-method | 原始请求的方法，比如get/post/delete/patch | | x-forwarded-host | 原始请求的host | | x-forwarded-uri | 原始请求的path，包含路径参数，比如/v1/app?test=true | -| x-forwarded-for | 原始请求的客户端IP地址 | \ No newline at end of file +| x-forwarded-for | 原始请求的客户端IP地址 | diff --git a/plugins/wasm-go/extensions/ext-auth/README_EN.md b/plugins/wasm-go/extensions/ext-auth/README_EN.md new file mode 100644 index 0000000000..eda7e3ccc2 --- /dev/null +++ b/plugins/wasm-go/extensions/ext-auth/README_EN.md @@ -0,0 +1,242 @@ +--- +title: External Authentication +keywords: [higress, auth] +description: The Ext authentication plugin implements the functionality to call external authorization services for authentication and authorization. +--- +## Function Description +The `ext-auth` plugin implements the ability to send authorization requests to external authorization services to check whether client requests are authorized. This plugin is based on the [ext_authz filter](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_authz_filter) of Envoy, and implements part of the capability of connecting HTTP services found in the native filter. + +## Runtime Attributes +Plugin Execution Phase: `Authentication Phase` +Plugin Execution Priority: `360` + +## Configuration Fields +| Name | Data Type | Required | Default Value | Description | +| ------------------------------- | --------- | -------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `http_service` | object | Yes | - | Configuration for the external authorization service | +| `failure_mode_allow` | bool | No | false | When set to true, client requests will still be accepted even if communication with the authorization service fails, or if the authorization service returns an HTTP 5xx error. | +| `failure_mode_allow_header_add` | bool | No | false | When both `failure_mode_allow` and `failure_mode_allow_header_add` are set to true, if communication with the authorization service fails or if the authorization service returns an HTTP 5xx error, the request header will add `x-envoy-auth-failure-mode-allowed: true`. | +| `status_on_error` | int | No | 403 | Sets the HTTP status code returned to the client when the authorization service is unreachable or returns a status code of 5xx. The default status code is `403`. | + +Description of each configuration field under `http_service` +| Name | Data Type | Required | Default Value | Description | +| ------------------------ | --------- | -------- | ------------- | -------------------------------------------- | +| `endpoint_mode` | string | No | envoy | Choose one from `envoy`, `forward_auth`. | +| `endpoint` | object | Yes | - | HTTP service information for sending authorization requests. | +| `timeout` | int | No | 1000 | Timeout for connecting to `ext-auth` service, in milliseconds. | +| `authorization_request` | object | No | - | Configuration for sending authorization requests. | +| `authorization_response` | object | No | - | Configuration for processing authorization responses. | + +Description of each configuration field under `endpoint` +| Name | Data Type | Required | Default Value | Description | +|------------|-----------|----------|---------------|----------------------------------------------------------------------------------------------| +| `service_name` | string | Required | - | The full FQDN name of the authorization service, e.g., `ext-auth.dns`, `ext-auth.my-ns.svc.cluster.local`. | +| `service_port` | int | No | 80 | The service port of the authorization service. | +| `path_prefix` | string | Required if `endpoint_mode` is `envoy` | | When `endpoint_mode` is `envoy`, this is the request path prefix sent by the client to the authorization service. | +| `request_method` | string | No | GET | When `endpoint_mode` is `forward_auth`, this is the HTTP Method sent by the client to the authorization service. | +| `path` | string | Required if `endpoint_mode` is `forward_auth` | - | When `endpoint_mode` is `forward_auth`, this is the request path sent by the client to the authorization service. | + +Description of each configuration field under `authorization_request` +| Name | Data Type | Required | Default Value | Description | +|-------------------------|------------------------|----------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `allowed_headers` | array of StringMatcher | No | - | When set, client request headers that match will be added to the request to the authorization service. In addition to user-defined header matching rules, the `Authorization` HTTP header will automatically be included in the authorization service request. (If `endpoint_mode` is `forward_auth`, the original request's path will be set to `X-Original-Uri`, and the original request's HTTP Method will be set to `X-Original-Method`.) | +| `headers_to_add` | `map[string]string` | No | - | Sets the list of request headers to include in the request to the authorization service. Note that headers with the same name from the client request will be overwritten. | +| `with_request_body` | bool | No | false | Buffers the client request body and sends it in the authorization request (does not take effect for HTTP Methods GET, OPTIONS, HEAD). | +| `max_request_body_bytes`| int | No | 10MB | Sets the maximum size of the client request body to be stored in memory. When the request body reaches the value set in this field, an HTTP 413 status code will be returned, and the authorization process will not be initiated. Note that this setting takes precedence over the configuration of `failure_mode_allow`. | + +Description of each configuration field under `authorization_response` +| Name | Data Type | Required | Default Value | Description | +|---------------------------|------------------------|----------|---------------|-----------------------------------------------------------------------------------------| +| `allowed_upstream_headers` | array of StringMatcher | No | - | When set, response headers from the authorization request that match will be added to the original client request headers. Note that headers with the same name will be overwritten. | +| `allowed_client_headers` | array of StringMatcher | No | - | If not set, when a request is denied, all response headers from the authorization request will be added to the client response. When set, in the case of a denied request, response headers from the authorization request that match will be added to the client response. | + +Configuration of the `StringMatcher` type, when using `array of StringMatcher`, will be configured in the order defined in the array +| Name | Data Type | Required | Default Value | Description | +|------------|-----------|------------------------------------------------------------------|---------------|------------------| +| `exact` | string | No, choose one from `exact`, `prefix`, `suffix`, `contains`, `regex` | - | Exact match | +| `prefix` | string | No, choose one from `exact`, `prefix`, `suffix`, `contains`, `regex` | - | Prefix match | +| `suffix` | string | No, choose one from `exact`, `prefix`, `suffix`, `contains`, `regex` | - | Suffix match | +| `contains` | string | No, choose one from `exact`, `prefix`, `suffix`, `contains`, `regex` | - | Contains | +| `regex` | string | No, choose one from `exact`, `prefix`, `suffix`, `contains`, `regex` | - | Regex match | + +## Configuration Example +Assuming the `ext-auth` service in Kubernetes has a serviceName of `ext-auth`, port `8090`, path of `/auth`, and namespace of `backend`, it supports two types of `endpoint_mode`: + +- When `endpoint_mode` is `envoy`, the authorization request will use the original HTTP Method and the configured `path_prefix` as a prefix combined with the original request path. +- When `endpoint_mode` is `forward_auth`, the authorization request will use the configured `request_method` as the HTTP Method and the configured `path` as the request path. + +### Example for endpoint_mode being envoy +#### Example 1 +Configuration of the `ext-auth` plugin: +```yaml +http_service: + endpoint_mode: envoy + endpoint: + service_name: ext-auth.backend.svc.cluster.local + service_port: 8090 + path_prefix: /auth + timeout: 1000 +``` + +Using the following request through the gateway, when the `ext-auth` plugin is enabled: +```shell +curl -X POST http://localhost:8082/users?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5 -X GET -H "foo: bar" -H "Authorization: xxx" +``` + +**Request to `ext-auth` service successful:** +The `ext-auth` service will receive the following authorization request: +```shell +POST /auth/users?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5 HTTP/1.1 +Host: ext-auth +Authorization: xxx +Content-Length: 0 +``` + +**Request to `ext-auth` service failed:** +When calling the `ext-auth` service gets a 5xx response, the client will receive an HTTP response code of 403 along with all response headers returned by the `ext-auth` service. + +If the `ext-auth` service returns response headers like `x-auth-version: 1.0` and `x-auth-failed: true`, these will be passed to the client: +```shell +HTTP/1.1 403 Forbidden +x-auth-version: 1.0 +x-auth-failed: true +date: Tue, 16 Jul 2024 00:19:41 GMT +server: istio-envoy +content-length: 0 +``` + +When `ext-auth` is unreachable or returns a status code of 5xx, the client request will be denied with the status code configured in `status_on_error`. If the `ext-auth` service returns other HTTP status codes, the client request will be denied with the returned status code. If `allowed_client_headers` is configured, response headers with corresponding matching items will be added to the client response. + +#### Example 2 +Configuration of the `ext-auth` plugin: +```yaml +http_service: + authorization_request: + allowed_headers: + - exact: x-auth-version + headers_to_add: + x-envoy-header: true + authorization_response: + allowed_upstream_headers: + - exact: x-user-id + - exact: x-auth-version + endpoint_mode: envoy + endpoint: + service_name: ext-auth.backend.svc.cluster.local + service_port: 8090 + path_prefix: /auth + timeout: 1000 +``` + +Using the following request through the gateway, when the `ext-auth` plugin is enabled: +```shell +curl -X POST http://localhost:8082/users?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5 -X GET -H "foo: bar" -H "Authorization: xxx" +``` + +The `ext-auth` service will receive the following authorization request: +```shell +POST /auth/users?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5 HTTP/1.1 +Host: ext-auth +Authorization: xxx +X-Auth-Version: 1.0 +x-envoy-header: true +Content-Length: 0 +``` + +If the response headers from the `ext-auth` service contain `x-user-id` and `x-auth-version`, these two headers will be included in the upstream request when the gateway calls upstream. + +### Example for endpoint_mode being forward_auth +#### Example 1 +Configuration of the `ext-auth` plugin: +```yaml +http_service: + endpoint_mode: forward_auth + endpoint: + service_name: ext-auth.backend.svc.cluster.local + service_port: 8090 + path: /auth + request_method: POST + timeout: 1000 +``` + +Using the following request through the gateway, when the `ext-auth` plugin is enabled: +```shell +curl -i http://localhost:8082/users?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5 -X GET -H "foo: bar" -H "Authorization: xxx" +``` + +**Request to `ext-auth` service successful:** +The `ext-auth` service will receive the following authorization request: +```shell +POST /auth HTTP/1.1 +Host: ext-auth +Authorization: xxx +X-Original-Uri: /users?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5 +X-Original-Method: GET +Content-Length: 0 +``` + +**Request to `ext-auth` service failed:** +When calling the `ext-auth` service gets a 5xx response, the client will receive an HTTP response code of 403 along with all response headers returned by the `ext-auth` service. + +If the `ext-auth` service returns response headers like `x-auth-version: 1.0` and `x-auth-failed: true`, these will be passed to the client: +```shell +HTTP/1.1 403 Forbidden +x-auth-version: 1.0 +x-auth-failed: true +date: Tue, 16 Jul 2024 00:19:41 GMT +server: istio-envoy +content-length: 0 +``` + +When `ext-auth` is unreachable or returns a status code of 5xx, the client request will be denied with the status code configured in `status_on_error`. If the `ext-auth` service returns other HTTP status codes, the client request will be denied with the returned status code. If `allowed_client_headers` is configured, response headers with corresponding matching items will be added to the client response. + +#### Example 2 +Configuration of the `ext-auth` plugin: +```yaml +http_service: + authorization_request: + allowed_headers: + - exact: x-auth-version + headers_to_add: + x-envoy-header: true + authorization_response: + allowed_upstream_headers: + - exact: x-user-id + - exact: x-auth-version + endpoint_mode: forward_auth + endpoint: + service_name: ext-auth.backend.svc.cluster.local + service_port: 8090 + path: /auth + request_method: POST + timeout: 1000 +``` + +Using the following request through the gateway, when the `ext-auth` plugin is enabled: +```shell +curl -i http://localhost:8082/users?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5 -X GET -H "foo: bar" -H "Authorization: xxx" -H "X-Auth-Version: 1.0" +``` + +The `ext-auth` service will receive the following authorization request: +```shell +POST /auth HTTP/1.1 +Host: ext-auth +Authorization: xxx +X-Original-Uri: /users?apikey=9a342114-ba8a-11ec-b1bf-00163e1250b5 +X-Original-Method: GET +X-Auth-Version: 1.0 +x-envoy-header: true +Content-Length: 0 +``` + +If the response headers from the `ext-auth` service contain `x-user-id` and `x-auth-version`, these two headers will be included in the upstream request when the gateway calls upstream. + +#### x-forwarded-* header +When `endpoint_mode` is `forward_auth`, higress will automatically generate and send the following headers to the authorization service. +| Header | Description | +|--------------------|------------------------------------------| +| x-forwarded-proto | The original request scheme, e.g., http/https | +| x-forwarded-method | The original request method, e.g., get/post/delete/patch | +| x-forwarded-host | The original request host | +| x-forwarded-uri | The original request path, including path parameters, e.g., /v1/app?test=true | +| x-forwarded-for | The original request client IP address | diff --git a/plugins/wasm-go/extensions/frontend-gray/README.md b/plugins/wasm-go/extensions/frontend-gray/README.md index dce90802c6..6c5db2c3ac 100644 --- a/plugins/wasm-go/extensions/frontend-gray/README.md +++ b/plugins/wasm-go/extensions/frontend-gray/README.md @@ -1,7 +1,18 @@ -# frontend-gray 前端灰度插件 +--- +title: 前端灰度 +keywords: [higress,frontend gray] +description: 前端灰度插件配置参考 +--- + ## 功能说明 `frontend-gray`插件实现了前端用户灰度的的功能，通过此插件，不但可以用于业务`A/B实验`，同时通过`可灰度`配合`可监控`,`可回滚`策略保证系统发布运维的稳定性。 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`450` + + ## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | |----------------|--------------|----|-----|----------------------------------------------------------------------------------------------------| diff --git a/plugins/wasm-go/extensions/frontend-gray/README_EN.md b/plugins/wasm-go/extensions/frontend-gray/README_EN.md new file mode 100644 index 0000000000..b3a652d449 --- /dev/null +++ b/plugins/wasm-go/extensions/frontend-gray/README_EN.md @@ -0,0 +1,162 @@ +--- +title: Frontend Gray +keywords: [higress, frontend gray] +description: Frontend gray plugin configuration reference +--- +## Function Description +The `frontend-gray` plugin implements the functionality of user gray release on the frontend. Through this plugin, it can be used for business `A/B testing`, while the `gradual release` combined with `monitorable` and `rollback` strategies ensures the stability of system release operations. + +## Runtime Attributes +Plugin execution phase: `Authentication Phase` +Plugin execution priority: `450` + +## Configuration Fields +| Name | Data Type | Requirements | Default Value | Description | +|-----------------|-------------------|---------------|---------------|-------------------------------------------------------------------------------------------------------------| +| `grayKey` | string | Optional | - | The unique identifier of the user ID, which can be from Cookie or Header, such as userid. If not provided, uses `rules[].grayTagKey` and `rules[].grayTagValue` to filter gray release rules. | +| `graySubKey` | string | Optional | - | User identity information may be output in JSON format, for example: `userInfo:{ userCode:"001" }`, in the current example, `graySubKey` is `userCode`. | +| `rules` | array of object | Required | - | User-defined different gray release rules, adapted to different gray release scenarios. | +| `rewrite` | object | Required | - | Rewrite configuration, generally used for OSS/CDN frontend deployment rewrite configurations. | +| `baseDeployment`| object | Optional | - | Configuration of the Base baseline rules. | +| `grayDeployments` | array of object | Optional | - | Configuration of the effective rules for gray release, as well as the effective versions. | + +`rules` field configuration description: +| Name | Data Type | Requirements | Default Value | Description | +|------------------|-------------------|---------------|---------------|--------------------------------------------------------------------------------------------| +| `name` | string | Required | - | Unique identifier for the rule name, associated with `deploy.gray[].name` for effectiveness. | +| `grayKeyValue` | array of string | Optional | - | Whitelist of user IDs. | +| `grayTagKey` | string | Optional | - | Label key for user classification tagging, derived from Cookie. | +| `grayTagValue` | array of string | Optional | - | Label value for user classification tagging, derived from Cookie. | + +`rewrite` field configuration description: +> `indexRouting` homepage rewrite and `fileRouting` file rewrite essentially use prefix matching, for example, `/app1`: `/mfe/app1/{version}/index.html` represents requests with the prefix /app1 routed to `/mfe/app1/{version}/index.html` page, where `{version}` represents the version number, which will be dynamically replaced by `baseDeployment.version` or `grayDeployments[].version` during execution. +> `{version}` will be replaced dynamically during execution by the frontend version from `baseDeployment.version` or `grayDeployments[].version`. + +| Name | Data Type | Requirements | Default Value | Description | +|------------------|-------------------|---------------|---------------|---------------------------------------| +| `host` | string | Optional | - | Host address, if OSS set to the VPC internal access address. | +| `notFoundUri` | string | Optional | - | 404 page configuration. | +| `indexRouting` | map of string to string | Optional | - | Defines the homepage rewrite routing rules. Each key represents the homepage routing path, and the value points to the redirect target file. For example, the key `/app1` corresponds to the value `/mfe/app1/{version}/index.html`. If the effective version is `0.0.1`, the access path is `/app1`, it redirects to `/mfe/app1/0.0.1/index.html`. | +| `fileRouting` | map of string to string | Optional | - | Defines resource file rewrite routing rules. Each key represents the resource access path, and the value points to the redirect target file. For example, the key `/app1/` corresponds to the value `/mfe/app1/{version}`. If the effective version is `0.0.1`, the access path is `/app1/js/a.js`, it redirects to `/mfe/app1/0.0.1/js/a.js`. | + +`baseDeployment` field configuration description: +| Name | Data Type | Requirements | Default Value | Description | +|------------------|-------------------|---------------|---------------|-------------------------------------------------------------------------------------------| +| `version` | string | Required | - | The version number of the Base version, as a fallback version. | + +`grayDeployments` field configuration description: +| Name | Data Type | Requirements | Default Value | Description | +|------------------|-------------------|---------------|---------------|----------------------------------------------------------------------------------------------| +| `version` | string | Required | - | Version number of the Gray version, if the gray rules are hit, this version will be used. If it is a non-CDN deployment, add `x-higress-tag` to the header. | +| `backendVersion` | string | Required | - | Gray version for the backend, which will add `x-mse-tag` to the header of `XHR/Fetch` requests. | +| `name` | string | Required | - | Rule name associated with `rules[].name`. | +| `enabled` | boolean | Required | - | Whether to activate the current gray release rule. | + +## Configuration Example +### Basic Configuration +```yml +grayKey: userid +rules: +- name: inner-user + grayKeyValue: + - '00000001' + - '00000005' +- name: beta-user + grayKeyValue: + - '00000002' + - '00000003' + grayTagKey: level + grayTagValue: + - level3 + - level5 +baseDeployment: + version: base +grayDeployments: + - name: beta-user + version: gray + enabled: true +``` + +The unique identifier of the user in the cookie is `userid`, and the current gray release rule has configured the `beta-user` rule. +When the following conditions are met, the version `version: gray` will be used: +- `userid` in the cookie equals `00000002` or `00000003` +- Users whose `level` in the cookie equals `level3` or `level5` +Otherwise, use version `version: base`. + +### User Information Exists in JSON +```yml +grayKey: appInfo +graySubKey: userId +rules: +- name: inner-user + grayKeyValue: + - '00000001' + - '00000005' +- name: beta-user + grayKeyValue: + - '00000002' + - '00000003' + grayTagKey: level + grayTagValue: + - level3 + - level5 +baseDeployment: + version: base +grayDeployments: + - name: beta-user + version: gray + enabled: true +``` + +The cookie contains JSON data for `appInfo`, which includes the field `userId` as the current unique identifier. +The current gray release rule has configured the `beta-user` rule. +When the following conditions are met, the version `version: gray` will be used: +- `userid` in the cookie equals `00000002` or `00000003` +- Users whose `level` in the cookie equals `level3` or `level5` +Otherwise, use version `version: base`. + +### Rewrite Configuration +> Generally used in CDN deployment scenarios. +```yml +grayKey: userid +rules: +- name: inner-user + grayKeyValue: + - '00000001' + - '00000005' +- name: beta-user + grayKeyValue: + - '00000002' + - '00000003' + grayTagKey: level + grayTagValue: + - level3 + - level5 +rewrite: + host: frontend-gray.oss-cn-shanghai-internal.aliyuncs.com + notFoundUri: /mfe/app1/dev/404.html + indexRouting: + /app1: '/mfe/app1/{version}/index.html' + /: '/mfe/app1/{version}/index.html', + fileRouting: + /: '/mfe/app1/{version}' + /app1/: '/mfe/app1/{version}' +baseDeployment: + version: base +grayDeployments: + - name: beta-user + version: gray + enabled: true +``` + +`{version}` will be dynamically replaced with the actual version during execution. + +#### indexRouting: Homepage Route Configuration +Accessing `/app1`, `/app123`, `/app1/index.html`, `/app1/xxx`, `/xxxx` will route to '/mfe/app1/{version}/index.html'. + +#### fileRouting: File Route Configuration +The following file mappings are effective: +- `/js/a.js` => `/mfe/app1/v1.0.0/js/a.js` +- `/js/template/a.js` => `/mfe/app1/v1.0.0/js/template/a.js` +- `/app1/js/a.js` => `/mfe/app1/v1.0.0/js/a.js` +- `/app1/js/template/a.js` => `/mfe/app1/v1.0.0/js/template/a.js` diff --git a/plugins/wasm-go/extensions/geo-ip/README.md b/plugins/wasm-go/extensions/geo-ip/README.md index e1debe7359..975bc8b902 100644 --- a/plugins/wasm-go/extensions/geo-ip/README.md +++ b/plugins/wasm-go/extensions/geo-ip/README.md @@ -1,8 +1,20 @@ -# 功能说明 +--- +title: IP 地理位置 +keywords: [higress,geo ip] +description: IP 地理位置插件配置参考 +--- + + +## 功能说明 `geo-ip`本插件实现了通过用户ip查询出地理位置信息，然后通过请求属性和新添加的请求头把地理位置信息传递给后续插件。 -# 配置字段 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`440` + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- | -------- | -------- | | ip_protocol | string | 否 | ipv4 | 可选值：1. ipv4：只对ipv4用户请求查找地理位置信息，传递给后续插件。而ipv6用户的请求会跳过该插件，继续由后续插件处理。 2. ipv6：(未来实现后)只对ipv6用户查找地理位置信息，传递给后续插件。而ipv4用户的请求会跳过该插件，继续由后续插件处理。（目前是跳过插件，请求由后续插件处理。） @@ -10,7 +22,7 @@ | ip_header_name | string | 否 | x-forwarded-for | 当`ip_source_type`为`header`时，指定自定义IP来源头 | -# 配置示例 +## 配置示例 ```yaml ip_protocol: ipv4 @@ -18,7 +30,7 @@ ip_source_type: header ip_header_name: X-Real-Ip ``` -# 生成geoCidr.txt的说明 +## 生成geoCidr.txt的说明 在generateCidr目录里包含的ip.merge.txt文件是github上ip2region项目的全世界的ip网段库。 ipRange2Cidr.go 是把ip网段转换成多个cidr的程序。转换出的cidr 和地理位置信息存在 /data/geoCidr.txt文件里。geo-ip插件会在Higress启动读配置阶段读取geoCidr.txt文件并且解析到radixtree数据结构的内存里，以便以后查询用户ip对应的地理位置信息。转换程序运行命令如下： @@ -26,13 +38,5 @@ ip_header_name: X-Real-Ip go run generateCidr/ipRange2Cidr.go ``` -# property 的使用方式 +## property 的使用方式 在geo-ip插件里调用proxywasm.SetProperty() 分别把country、city、province、isp设置进请求属性里，以便后续插件可以调用proxywasm.GetProperty()获取该请求的用户ip对应的地理信息。 - -# ip网段转换成cidr列表的单元测试 -在 generateCidr 目录里的 ipRange2Cidr_test.go 是ip网段转换成cidr 列表的单元测试程序。在 generateCidr 目录里运行命令 go test 。通过的情况显示如下： - -``bash -PASS -ok higress/plugins/wasm-go/extensions/geo-ip/generateCidr 0.018s -``` diff --git a/plugins/wasm-go/extensions/geo-ip/README_EN.md b/plugins/wasm-go/extensions/geo-ip/README_EN.md new file mode 100644 index 0000000000..5749dc1876 --- /dev/null +++ b/plugins/wasm-go/extensions/geo-ip/README_EN.md @@ -0,0 +1,34 @@ +--- +title: IP Geolocation +keywords: [higress,geo ip] +description: IP Geolocation Plugin Configuration Reference +--- +## Function Description +The `geo-ip` plugin allows querying geographical location information based on the user's IP address, and then passes this geographical information to subsequent plugins through request attributes and newly added request headers. + +## Runtime Properties +Plugin Execution Phase: `Authentication Phase` +Plugin Execution Priority: `440` + +## Configuration Fields +| Name | Data Type | Requirement | Default Value | Description | +| -------- | ----------- | ----------- | ------------------ | ------------ | +| ip_protocol | string | No | ipv4 | Optional values: 1. ipv4: Only queries geographical location information for ipv4 user requests, passing it to subsequent plugins. Requests from ipv6 users will skip this plugin and be processed by later plugins. 2. ipv6: (To be implemented in the future) Only queries geographical location information for ipv6 users, passing it to subsequent plugins. Requests from ipv4 users will skip this plugin and be processed by later plugins. (Currently skips the plugin; requests are handled by subsequent plugins.) | +| ip_source_type | string | No | origin-source | Optional values: 1. Peer socket IP: `origin-source`; 2. Retrieved via header: `header` | +| ip_header_name | string | No | x-forwarded-for | When `ip_source_type` is `header`, specify the custom IP source header. | + +## Configuration Example +```yaml +ip_protocol: ipv4 +ip_source_type: header +ip_header_name: X-Real-Ip +``` + +## Explanation for Generating geoCidr.txt +The ip.merge.txt file included in the generateCidr directory is the global IP segment database from the ip2region project on GitHub. The ipRange2Cidr.go program converts IP segments into multiple CIDRs. The converted CIDRs and geographical location information are stored in the /data/geoCidr.txt file. The geo-ip plugin will read the geoCidr.txt file during the configuration stage when Higress starts and parse it into the radixtree data structure in memory for future queries of geographical location information corresponding to user IP addresses. The command to run the conversion program is as follows: +```bash +go run generateCidr/ipRange2Cidr.go +``` + +## Usage of Properties +In the geo-ip plugin, call proxywasm.SetProperty() to set country, city, province, and isp into request attributes so that subsequent plugins can use proxywasm.GetProperty() to obtain the geographical information corresponding to the user's IP for that request. diff --git a/plugins/wasm-go/extensions/ip-restriction/README.md b/plugins/wasm-go/extensions/ip-restriction/README.md index 9f6bab8a81..1f397bb194 100644 --- a/plugins/wasm-go/extensions/ip-restriction/README.md +++ b/plugins/wasm-go/extensions/ip-restriction/README.md @@ -1,9 +1,21 @@ -# 功能说明 +--- +title: IP 访问限制 +keywords: [higress, ip restriction] +description: IP 访问限制插件配置参考 +--- + +## 功能说明 `ip-restriction `插件可以通过将 IP 地址列入白名单或黑名单来限制对服务或路由的访问.支持对单个 IP 地址、多个 IP 地址和类似 10.10.10.0/24 的 CIDR范围的限制. -# 配置说明 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`210` + + +## 配置说明 | 配置项 | 类型 | 必填 | 默认值 | 说明 | |----------------|--------|----|-----------------------------|------------------------------------------| diff --git a/plugins/wasm-go/extensions/ip-restriction/README_EN.md b/plugins/wasm-go/extensions/ip-restriction/README_EN.md new file mode 100644 index 0000000000..29aaa370d6 --- /dev/null +++ b/plugins/wasm-go/extensions/ip-restriction/README_EN.md @@ -0,0 +1,37 @@ +--- +title: IP Access Restriction +keywords: [higress, ip restriction] +description: IP access restriction plugin configuration reference +--- +## Function Description +The `ip-restriction` plugin can restrict access to services or routes by whitelisting or blacklisting IP addresses. It supports restrictions on a single IP address, multiple IP addresses, and CIDR ranges like 10.10.10.0/24. + +## Running Attributes +Plugin execution phase: `Authentication Phase` + +Plugin execution priority: `210` + +## Configuration Description +| Configuration Item | Type | Required | Default Value | Description | +|---------------------|---------|----------|---------------------------------|---------------------------------------------| +| ip_source_type | string | No | origin-source | Optional values: 1. Peer socket IP: `origin-source`; 2. Get from header: `header` | +| ip_header_name | string | No | x-forwarded-for | When `ip_source_type` is `header`, specify the custom IP source header | +| allow | array | No | [] | Whitelist | +| deny | array | No | [] | Blacklist | +| status | int | No | 403 | HTTP status code when access is denied | +| message | string | No | Your IP address is blocked. | Return message when access is denied | + +```yaml +ip_source_type: origin-source +allow: + - 10.0.0.1 + - 192.168.0.0/16 +``` + +```yaml +ip_source_type: header +ip_header_name: x-real-iP +deny: + - 10.0.0.1 + - 192.169.0.0/16 +``` diff --git a/plugins/wasm-go/extensions/key-auth/README.md b/plugins/wasm-go/extensions/key-auth/README.md index a06abfac6c..786124e033 100644 --- a/plugins/wasm-go/extensions/key-auth/README.md +++ b/plugins/wasm-go/extensions/key-auth/README.md @@ -1,15 +1,32 @@ -# 功能说明 +--- +title: Key 认证 +keywords: [higress,key auth] +description: Key 认证插件配置参考 +--- + +## 功能说明 `key-auth`插件实现了基于 API Key 进行认证鉴权的功能，支持从 HTTP 请求的 URL 参数或者请求头解析 API Key，同时验证该 API Key 是否有权限访问。 -# 配置字段 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`310` + +## 配置字段 + +**注意：** + +- 在一个规则里，鉴权配置和认证配置不可同时存在 +- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 -| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | -| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | -| `global_auth` | bool | 选填 | - | 若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制; 若不配置则仅当没有域名和路由配置时全局生效（兼容机制） | -| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | -| `keys` | array of string | 必填 | - | API Key 的来源字段名称，可以是 URL 参数或者 HTTP 请求头名称 | -| `in_query` | bool | `in_query` 和 `in_header` 至少有一个为 true | true | 配置 true 时，网关会尝试从 URL 参数中解析 API Key | -| `in_header` | bool | `in_query` 和 `in_header` 至少有一个为 true | true | 配置 true 时，网关会尝试从 HTTP 请求头中解析 API Key | +### 认证配置 +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `global_auth` | bool | 选填（**仅实例级别配置**） | - | 只能在实例级别配置，若配置为true，则全局生效认证机制; 若配置为false，则只对做了配置的域名和路由生效认证机制，若不配置则仅当没有域名和路由配置时全局生效（兼容老用户使用习惯）。 | +| `consumers` | array of object | 必填 | - | 配置服务的调用者，用于对请求进行认证 | +| `keys` | array of string | 必填 | - | API Key 的来源字段名称，可以是 URL 参数或者 HTTP 请求头名称 | +| `in_query` | bool | `in_query` 和 `in_header` 至少有一个为 true | true | 配置 true 时，网关会尝试从 URL 参数中解析 API Key | +| `in_header` | bool | `in_query` 和 `in_header` 至少有一个为 true | true | 配置 true 时，网关会尝试从 HTTP 请求头中解析 API Key | `consumers`中每一项的配置字段说明如下： @@ -18,18 +35,22 @@ | `credential` | string | 必填 | - | 配置该consumer的访问凭证 | | `name` | string | 必填 | - | 配置该consumer的名称 | +### 鉴权配置（非必需） -**注意：** -- 对于通过认证鉴权的请求，请求的header会被添加一个`X-Mse-Consumer`字段，用以标识调用者的名称。 +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- | +| `allow` | array of string | 选填(**非实例级别配置**) | - | 只能在路由或域名等细粒度规则上配置，对于符合匹配条件的请求，配置允许访问的 consumer，从而实现细粒度的权限控制 | -# 配置示例 +## 配置示例 -## 对特定路由或域名开启 +### 全局配置认证和路由粒度进行鉴权 -以下配置将对网关特定路由或域名开启 Key Auth 认证和鉴权，注意`credential`字段不能重复 +以下配置将对网关特定路由或域名开启Key Auth认证和鉴权。credential字段不能重复。 + +在实例级别做如下插件配置： ```yaml -global_auth: true +global_auth: false consumers: - credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 name: consumer1 @@ -40,8 +61,6 @@ keys: - x-api-key ``` -**路由级配置** - 对 route-a 和 route-b 这两个路由做如下配置： ```yaml @@ -56,10 +75,16 @@ allow: - consumer2 ``` -### 根据该配置，下列请求可以允许访问： +**说明：** -假设以下请求会匹配到route-a这条路由 +此例指定的route-a和route-b即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将允许name为consumer1的调用者访问，其他调用者不允许访问。 + +此例指定的*.example.com和test.com用于匹配请求的域名，当发现域名匹配时，将允许name为consumer2的调用者访问，其他调用者不被允许访问。 +根据该配置，下列请求可以允许访问： + +假设以下请求会匹配到route-a这条路由 +n **将 API Key 设置在 url 参数中** ```bash curl http://xxx.hello.com/test?apikey=2bda943c-ba2b-11ec-ba07-00163e1250b5 @@ -71,7 +96,7 @@ curl http://xxx.hello.com/test -H 'x-api-key: 2bda943c-ba2b-11ec-ba07-00163e125 认证鉴权通过后，请求的header中会被添加一个`X-Mse-Consumer`字段，在此例中其值为`consumer1`，用以标识调用方的名称 -### 下列请求将拒绝访问： +下列请求将拒绝访问： **请求未提供 API Key，返回401** ```bash @@ -88,31 +113,24 @@ curl http://xxx.hello.com/test?apikey=926d90ac-ba2e-11ec-ab68-00163e1250b5 curl http://xxx.hello.com/test?apikey=c8c8e9ca-558e-4a2d-bb62-e700dcc40e35 ``` -## 网关实例级别开启 +### 网关实例级别开启 -以下配置未指定`matchRules`字段，因此将对网关实例级别开启全局 Key Auth 认证. +以下配置将对网关实例级别开启 Basic Auth 认证，所有请求均需要经过认证后才能访问。 ```yaml -defaultConfig - consumers: - - credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 - name: consumer1 - - credential: c8c8e9ca-558e-4a2d-bb62-e700dcc40e35 - name: consumer2 - keys: - - apikey - in_query: true +global_auth: true +consumers: +- credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 + name: consumer1 +- credential: c8c8e9ca-558e-4a2d-bb62-e700dcc40e35 + name: consumer2 +keys: +- apikey +- x-api-key ``` -开启`matchRules`方式如下： -```yaml -matchRules: - - config: - allow: - - consumer1 -``` -# 相关错误码 +## 相关错误码 | HTTP 状态码 | 出错信息 | 原因说明 | | ----------- | --------------------------------------------------------- | ----------------------- | diff --git a/plugins/wasm-go/extensions/key-auth/README_EN.md b/plugins/wasm-go/extensions/key-auth/README_EN.md index bb5ed77711..3abccc4f66 100644 --- a/plugins/wasm-go/extensions/key-auth/README_EN.md +++ b/plugins/wasm-go/extensions/key-auth/README_EN.md @@ -1,34 +1,47 @@ -# Features -The `key-auth` plug-in implements the authentication function based on the API Key, supports parsing the API Key from the URL parameter or request header of the HTTP request, and verifies whether the API Key has permission to access. - -# Configuration field - -| Name | Data Type | Parameter requirements | Default| Description | -| ----------- | --------------- | -------------------------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------- | -| `global_auth` | bool | Optional | - | If configured to true, the authentication mechanism will take effect globally; if configured to false, the authentication mechanism will only take effect for the configured domain names and routes; if not configured, the authentication mechanism will only take effect globally when no domain names and routes are configured (compatibility mechanism) | -| `consumers` | array of object | Required | - | Configure the caller of the service to authenticate the request. | -| `keys` | array of string | Required | - | The name of the source field of the API Key, which can be a URL parameter or an HTTP request header name. | -| `in_query` | bool | At least one of `in_query` and `in_header` must be true. | true | When configured true, the gateway will try to parse the API Key from the URL parameters. | -| `in_header` | bool | The same as above. | true | The same as above. | - -The configuration fields of each item in `consumers` are described as follows: - -| Name | Data Type | Parameter requirements | Default | Description | -| ------------ | --------- | -----------------------| ------ | ------------------------------------------- | -| `credential` | string | Required | - | Configure the consumer's access credentials. | -| `name` | string | Required | - | Configure the name of the consumer. | - -**Warning：** -- For a request that passes authentication, an `X-Mse-Consumer` field will be added to the request header to identify the name of the caller. - -# Example configuration - -## Enabled for specific routes or domains - -The following configuration will enable Key Auth authentication and authentication for gateway-specific routes or domain names. Note that the `credential` field can not be repeated. - +--- +title: Key Authentication +keywords: [higress,key auth] +description: Key Authentication Plugin Configuration Reference +--- +## Function Description +The `key-auth` plugin implements authentication based on API Key, supporting the parsing of the API Key from HTTP request URL parameters or request headers, while also verifying whether the API Key has permission to access the resource. + +## Runtime Properties +Plugin Execution Phase: `Authentication Phase` +Plugin Execution Priority: `310` + +## Configuration Fields +**Note:** +- Authentication and authorization configurations cannot coexist within a single rule. +- For requests that are authenticated, a header field `X-Mse-Consumer` will be added to identify the caller's name. + +### Authentication Configuration +| Name | Data Type | Requirements | Default Value | Description | +| ------------- | ---------------- | ----------------------------------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `global_auth` | bool | Optional (**Instance-Level Configuration Only**) | - | Can only be configured at the instance level; if set to true, the authentication mechanism takes effect globally; if set to false, it only applies to the configured hostnames and routes. If not configured, it will only take effect globally when no hostname and route configurations are present (to maintain compatibility with older user habits). | +| `consumers` | array of object | Required | - | Configures the service callers for request authentication. | +| `keys` | array of string | Required | - | Source field names for the API Key, which can be URL parameters or HTTP request header names. | +| `in_query` | bool | At least one of `in_query` and `in_header` must be true | true | When configured as true, the gateway will attempt to parse the API Key from URL parameters. | +| `in_header` | bool | At least one of `in_query` and `in_header` must be true | true | When configured as true, the gateway will attempt to parse the API Key from HTTP request headers. | + +The configuration field descriptions for each item in `consumers` are as follows: +| Name | Data Type | Requirements | Default Value | Description | +| ------------ | --------- | ------------ | ------------- | ------------------------------ | +| `credential` | string | Required | - | Configures the access credential for this consumer. | +| `name` | string | Required | - | Configures the name for this consumer. | + +### Authorization Configuration (Optional) +| Name | Data Type | Requirements | Default Value | Description | +| ----------- | ---------------- | ----------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `allow` | array of string | Optional (**Non-Instance Level Configuration**) | - | Can only be configured on fine-grained rules such as routes or hostnames; specifies the allowed consumers for matching requests, allowing for fine-grained permission control. | + +## Configuration Example +### Global Configuration for Authentication and Granular Route Authorization +The following configuration will enable Key Auth authentication and authorization for specific routes or hostnames in the gateway. The `credential` field must not repeat. + +At the instance level, do the following plugin configuration: ```yaml -global_auth: true +global_auth: false consumers: - credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 name: consumer1 @@ -37,50 +50,62 @@ consumers: keys: - apikey - x-api-key -in_query: true ``` -The `route-a` and `route-b` specified in `_match_route_` in this example are the route names filled in when creating the gateway route. When these two routes are matched, calls whose `name` is `consumer1` will be allowed Access by callers, other callers are not allowed to access; +For routes route-a and route-b, do the following configuration: +```yaml +allow: +- consumer1 +``` -`*.example.com` and `test.com` specified in `_match_domain_` in this example are used to match the domain name of the request. When the domain name matches, the caller whose `name` is `consumer2` will be allowed to access, and other calls access is not allowed. +For the hostnames *.example.com and test.com, do the following configuration: +```yaml +allow: +- consumer2 +``` + +**Note:** +The routes route-a and route-b specified in this example refer to the route names filled in when creating the gateway routes. When matched with these two routes, requests from the caller named consumer1 will be allowed while others will be denied. -### Depending on this configuration, the following requests would allow access： +The specified hostnames *.example.com and test.com are used to match the request's domain name. When a domain name is matched, callers named consumer2 will be allowed while others will be denied. -Assume that the following request will match the route-a route: +Based on this configuration, the following requests will be allowed: -**Set the API Key in the url parameter** +Assuming the following request matches route-a: +**Setting API Key in URL Parameters** ```bash curl http://xxx.hello.com/test?apikey=2bda943c-ba2b-11ec-ba07-00163e1250b5 ``` -**Set the API Key in the http request header** + +**Setting API Key in HTTP Request Headers** ```bash curl http://xxx.hello.com/test -H 'x-api-key: 2bda943c-ba2b-11ec-ba07-00163e1250b5' ``` -After the authentication is passed, an `X-Mse-Consumer` field will be added to the header of the request. In this example, its value is `consumer1`, which is used to identify the name of the caller. +After successful authentication and authorization, the request's header will have an added `X-Mse-Consumer` field with the value `consumer1`, to identify the name of the caller. -### The following requests will deny access： - -**The request does not provide an API Key, return 401** +The following requests will be denied access: +**Request without an API Key returns 401** ```bash curl http://xxx.hello.com/test ``` -**The API Key provided by the request is not authorized to access, return 401** + +**Request with an invalid API Key returns 401** ```bash curl http://xxx.hello.com/test?apikey=926d90ac-ba2e-11ec-ab68-00163e1250b5 ``` -**The caller matched according to the API Key provided in the request has no access rights, return 403** +**Caller matched with provided API Key has no access rights, returns 403** ```bash # consumer2 is not in the allow list of route-a curl http://xxx.hello.com/test?apikey=c8c8e9ca-558e-4a2d-bb62-e700dcc40e35 ``` -## Gateway instance level enabled - -The following configuration does not specify the `matchRules` field, so Key Auth authentication will be enabled at the gateway instance level. +### Enabling at the Instance Level +The following configuration will enable Basic Auth authentication at the instance level for the gateway, requiring all requests to pass authentication before accessing. ```yaml +global_auth: true consumers: - credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5 name: consumer1 @@ -88,22 +113,13 @@ consumers: name: consumer2 keys: - apikey -in_query: true -``` - -configuration specify the `matchRules` field, like： -```yaml -matchRules: - - config: - allow: - - consumer1 +- x-api-key ``` -# Error code - -| HTTP status code | Error information | Reason | -| ---------------- | --------------------------------------------------------- | -------------------------------------------- | -| 401 | Muti API key found in request. | Muti API provided by request Key. | -| 401 | No API key found in request. | API not provided by request Key. | -| 401 | Request denied by Key Auth check. Invalid API key. | Current API Key access is not allowed. | -| 403 | Request denied by Key Auth check. Unauthorized consumer. | The requested caller does not have access. | +## Related Error Codes +| HTTP Status Code | Error Message | Reason Explanation | +| ---------------- | ---------------------------------------------------------- | --------------------------------- | +| 401 | Request denied by Key Auth check. Multiple API keys found in request | Multiple API Keys provided in the request. | +| 401 | Request denied by Key Auth check. No API key found in request | API Key not provided in the request. | +| 401 | Request denied by Key Auth check. Invalid API key | The current API Key is not authorized for access. | +| 403 | Request denied by Key Auth check. Unauthorized consumer | The caller does not have access permissions. | diff --git a/plugins/wasm-go/extensions/oidc/README.md b/plugins/wasm-go/extensions/oidc/README.md index 43102f0b2b..4cc76e9041 100644 --- a/plugins/wasm-go/extensions/oidc/README.md +++ b/plugins/wasm-go/extensions/oidc/README.md @@ -1,140 +1,19 @@ -# OIDC Wasm 插件 +--- +title: OIDC 认证 +keywords: [higress, oidc] +description: OIDC 认证插件配置参考 +--- -## 简介 +## 功能说明 -本仓库提供了一个高度可集成的Wasm插件，以支持OpenID Connect（OIDC）身份认证。同时，该插件强化了对跨站请求伪造（CSRF）攻击的防御能力，并支持OpenID Connect协议中的注销端点（Logout Endpoint）以及刷新令牌（Refresh Token）机制。在经过Wasm插件进行OIDC验证后的请求将携带 `Authorization` 头部，包含相应的访问令牌（Access Token）。 +本插件支持OpenID Connect（OIDC）身份认证。同时，该插件强化了对跨站请求伪造（CSRF）攻击的防御能力，并支持OpenID Connect协议中的注销端点（Logout Endpoint）以及刷新令牌（Refresh Token）机制。在经过Wasm插件进行OIDC验证后的请求将携带 `Authorization` 头部，包含相应的访问令牌（Access Token）。 -### OIDC 流程图 +## 运行属性 -

- oidc_process -

+插件执行阶段：`认证阶段` +插件执行优先级：`350` -### OIDC 流程解析 - -#### 用户未登录 - -1. 模拟用户访问对应服务 API - - ```shell - curl --url "foo.bar.com/headers" - ``` - -2. Higress 重定向到 OIDC Provider 登录页同时携带 client_id、response_type、scope 等 OIDC 认证的参数并设置 csrf cookie 防御CSRF 攻击 - - ```shell - curl --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/authorize"\ - --url-query "approval_prompt=force" \ - --url-query "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ - --url-query "redirect_uri=http%3A%2F%2Ffoo.bar.com%2Foauth2%2Fcallback" \ - --url-query "response_type=code" \ - --url-query "scope=openid+email+offline_access" \ - --url-query "state=nT06xdCqn4IqemzBRV5hmO73U_hCjskrH_VupPqdcdw%3A%2Ffoo" \ - --header "Set-Cookie: _oauth2_proxy_csrf=LPruATEDgcdmelr8zScD_ObhsbP4zSzvcgmPlcNDcJpFJ0OvhxP2hFotsU-kZnYxd5KsIjzeIXGTOjf8TKcbTHbDIt-aQoZORXI_0id3qeY0Jt78223DPeJ1xBqa8VO0UiEOUFOR53FGxirJOdKFxaAvxDFb1Ok=|1718962455|V1QGWyjQ4hMNOQ4Jtf17HeQJdVqHdt5d65uraFduMIU=; Path=/; Expires=Fri, 21 Jun 2024 08:06:20 GMT; HttpOnly" - ``` - -3. 重定向到登录页 - -![keycloak_login](https://gw.alicdn.com/imgextra/i4/O1CN01HLcl7r1boXwwnzGqA_!!6000000003512-0-tps-3840-2160.jpg) - -4. 用户输入用户名密码登录完成 - -5. 携带授权重定向到 Higress 并携带了 state 参数用于验证 csrf cookie ，授权码用于交换 token - - ```shell - curl --url "http://foo.bar.com/oauth2/callback" \ - --url-query "state=nT06xdCqn4IqemzBRV5hmO73U_hCjskrH_VupPqdcdw%3A%2Ffoo" \ - --url-query "code=0bdopoS2c2lx95u7iO0OH9kY1TvaEdJHo4lB6CT2_qVFm" - ``` - -6. 校验 csrf cookie 中加密存储的 state 值与 url 参数中的 state 值必须相同 - -7. 利用授权交换 id_token 和 access_token - - ```shell - curl -X POST \ - --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/oauth/token" \ - --data "grant_type=authorization_code" \ - --data "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ - --data "client_secret=ekqv5XoZuMFtYms1NszEqRx03qct6BPvGeJUeptNG4y09PrY16BKT9IWezTrrhJJ" \ - --data "redirect_uri=http%3A%2F%2Ffoo.bar.com%2Foauth2%2Fcallback" \ - --data "code=0bdopoS2c2lx95u7iO0OH9kY1TvaEdJHo4lB6CT2_qVFm" \ - ``` - - 返回的请求里包含了 id_token, access_token，refresh_token 用于后续刷新 token - - ```json - { - "access_token": "eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIiwiaXNzIjoiaHR0cHM6Ly9kZXYtbzQzeGIxbXo3eWE3YWNoNC51cy5hdXRoMC5jb20vIn0..WP_WRVM-y3fM1sN4.fAQqtKoKZNG9Wj0OhtrMgtsjTJ2J72M2klDRd9SvUKGbiYsZNPmIl_qJUf81D3VIjD59o9xrOOJIzXTgsfFVA2x15g-jBlNh68N7dyhXu9237Tbplweu1jA25IZDSnjitQ3pbf7xJVIfPnWcrzl6uT8G1EP-omFcl6AQprV2FoKFMCGFCgeafuttppKe1a8mpJDj7AFLPs-344tT9mvCWmI4DuoLFh0PiqMMJBByoijRSxcSdXLPxZng84j8JVF7H6mFa-dj-icP-KLy6yvzEaRKz_uwBzQCzgYK434LIpqw_PRuN3ClEsenwRgIsNdVjvKcoAysfoZhmRy9BQaE0I7qTohSBFNX6A.mgGGeeWgugfXcUcsX4T5dQ", - "refresh_token": "GrZ1f2JvzjAZQzSXmyr1ScWbv8aMFBvzAXHBUSiILcDEG", - "id_token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Imc1Z1ExSF9ZbTY0WUlvVkQwSVpXTCJ9.eyJlbWFpbCI6IjE2MDExNTYyNjhAcXEuY29tIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJpc3MiOiJodHRwczovL2Rldi1vNDN4YjFtejd5YTdhY2g0LnVzLmF1dGgwLmNvbS8iLCJhdWQiOiJZYWdGcVJEOXRmTklhYWM1QmFtamhzU2F0anJBbnNuWiIsImlhdCI6MTcxOTE5ODYzOCwiZXhwIjoxNzE5MjM0NjM4LCJzdWIiOiJhdXRoMHw2NjVkNzFlNzRjMTMxMTc3YmU2NmU2MDciLCJzaWQiOiJjdDJVOF9ZUS16VDdFOGkwRTNNeUstejc5ZGlWUWhhVSJ9.gfzXKJ0FeqzYqOUDLQHWcUG19IOLqkpLN09xTmIat0umrlGV5VNSumgWH3XJmmwnhdb8AThH3Jf-7kbRJzu4rM-BbGbFTRBTzNHeUajFOFrIgld5VENQ_M_sXHkTp0psWKSr9vF24kmilCfSbvC5lBKjt878ljZ7-xteWuaUYOMUdcJb4DSv0-zjX01sonJxYamTlhji3M4TAW7VwhwqyZt8dBhVSNaRw1wUKj-M1JrBDLyx65sroZtSqVA0udIrqMHEbWYb2de7JjzlqG003HRMzwOm7OXgEd5ZVFqgmBLosgixOU5DJ4A26nlqK92Sp6VqDMRvA-3ym8W_m-wJ_A", - "scope": "openid email offline_access", - "expires_in": 86400, - "token_type": "Bearer" - } - ``` - -8. 将获得的 id_token, access_token, refresh_token 加密存储在cookie _oauth2_proxy中 - -9. 重定向到用户访问的后端服务并设置 cookie，用于后续用户登录状态的验证，同时清除 cookie _oauth2_proxy_csrf - - ```json - "Set-Cookie": [ - "_oauth2_proxy_csrf=; Path=/; Expires=Mon, 24 Jun 2024 02:17:39 GMT; HttpOnly", - "_oauth2_proxy=8zM_Pcfpp_gesKFe4SMg08o5Iv0A8WAOQOmG1-vZBbQ56UggYVC0Cu-gFMEoxJZU5q1O5vqRlVBizlLetgVjRCksGVbttwl8tQ7h5YiyIubbbtvF1T4JzLh3QfzUUrwbB-VznOkh8qLbjAhddocecjBt4rMiDyceKXqMr4eO5TUEMx4vHtJYnTYalMeTYhGXk5MNSyrdZX9NnQnkdrCjiOQM13ggwob2nYwhGWaAlgzFSWkgkdtBy2Cl_YMWZ8_gKk9rDX289-JrJyGpr5k9O9RzRhZoY2iE3Mcr8-Q37RTji1Ga22QO-XkAcSaGqY1Qo7jLdmgZTYKC5JvtdLc4rj3vcbveYxU7R3Pt2vEribQjKTh4Sqb0aA03p4cxXyZN4SUfBW1NAOm4JLPUhKJy8frqC9_E0nVqPvpvnacaoQs8WkX2zp75xHoMa3SD6KZhQ5JUiPEiNkOaUsyafLvht6lLkNDhgzW3BP2czoe0DCDBLnsot0jH-qQpMZYkaGr-ZnRKI1OPl1vHls3mao5juOAW1VB2A9aughgc8SJ55IFZpMfFMdHdTDdMqPODkItX2PK44GX-pHeLxkOqrzp3GHtMInpL5QIQlTuux3erm3CG-ntlUE7JBtN2T9LEb8XfIFu58X9_vzMun4JQlje2Thi9_taI_z1DSaTtvNNb54wJfSPwYCCl4OsH-BacVmPQhH6TTZ6gP2Qsm5TR2o1U2D9fuVkSM-OPCG9l3tILambIQwC3vofMW6X8SIFSmhJUDvN7NbwxowBiZ6Y7GJRZlAk_GKDkpsdrdIvC67QqczZFphRVnm6qi-gPO41APCbcO6fgTwyOhbP3RrZZKWSIqWJYhNE3_Sfkf0565H7sC7Hc8XUUjJvP3WnjKS9x7KwzWa-dsUjV3-Q-VNl-rXTguVNAIirYK-qrMNMZGCRcJqcLnUF0V_J2lVmFyVsSlE3t0sDw2xmbkOwDptXFOjQL5Rb4esUMYdCBWFajBfvUtcZEFtYhD0kb6VcbjXO3NCVW5qKh_l9C9SRCc7TG1vcRAqUQlRXHacTGWfcWsuQkCJ3Mp_oWaDxs1GRDykQYxAn5sTICovThWEU2C6o75grWaNrkj5NU-0eHh3ryvxLmGLBOXZV9OQhtKShWmUgywSWMxOHOuZAqdAPULc8KheuGFjXYp-RnCbFYWePJmwzfQw89kSkj1KUZgMYwKEjSz62z2qc9KLczomv76ortQzvo4Hv9kaW6xVuQj5R5Oq6_WMBOqsmUMzcXpxCIOGjcdcZRBc0Fm09Uy9oV1PRqvAE4PGtfyrCaoqILBix8UIww63B07YGwzQ-hAXDysBK-Vca2x7GmGdXsNXXcTgu00bdsjtHZPDBBWGfL3g_rMAXr2vWyvK4CwNjcaPAmrlF3geHPwbIePT0hskBboX1v1bsuhzsai7rGM4r53pnb1ZEoTQDa1B-HyokFgo14XiwME0zE1ifpNzefjpkz1YY2krJlqfCydNwoKaTit4tD2yHlnxAeFF9iIrxzSKErNUFpmyLa7ge7V33vhEH-6k5oBTLE2Q2BrC6aAkLCcPwU9xv_SzBDQPRY0MEYv3kGF03Swo1crRbGh-aifYX9NiHDsmG6r1vAnx0MAOw2Jzuz2x6SSdfBrzlcoWBlrwiZzd9kAKq75n1Uy9uzZ8SRnkBrEZySHBwEbu196VklkRE0jqwC-e3wWNNuviSOfwkVeX-7QdOoO10yw9VK2sW52lFvIEf4chv_ta7bGfAZOWBjpktG6ZLD81SE6A88zpqG2SysSyNMp9hl-umG-5sFsjCn_c9E8bDvwkUOUVb9bNqhBDsZgR0BNPawiOZjmyfhzmwmWf-zgFzfFSV6BvOwNRi3sCOHTsWcuk9NBQ_YK8CpNkVl3WeIBSDfidimuC_QV9UWKs1GPk35ZRkM4zKtLY2JsBFWKaDy_P80TcOzcMBoP8gIBClXZ-WUqfE8s1yyc4jrq-qL1_wJ24ef1O9FktsbyZiDKXw2vnqsT8-g_hCeG-unrT1ZFscf8oNdqczARHX-K4vKH2k3uIqEx1M=|1719199056|2rsgdUIClHNEpxBLlHOVRYup6e4oKensQfljtmn4B80=; Path=/; Expires=Mon, 01 Jul 2024 03:17:36 GMT; HttpOnly" - ] - ``` - -10. 校验是否存在 cookie 存储了用户的 token 信息同时查看是否过期 - -11. 使用含有 Authorization 头部存储用户的 access_token 访问相应的 API - - ```shell - curl --url "foo.bar.com/headers" - --header "Authorization: Bearer eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIiwiaXNzIjoiaHR0cHM6Ly9kZXYtbzQzeGIxbXo3eWE3YWNoNC51cy5hdXRoMC5jb20vIn0..WP_WRVM-y3fM1sN4.fAQqtKoKZNG9Wj0OhtrMgtsjTJ2J72M2klDRd9SvUKGbiYsZNPmIl_qJUf81D3VIjD59o9xrOOJIzXTgsfFVA2x15g-jBlNh68N7dyhXu9237Tbplweu1jA25IZDSnjitQ3pbf7xJVIfPnWcrzl6uT8G1EP-omFcl6AQprV2FoKFMCGFCgeafuttppKe1a8mpJDj7AFLPs-344tT9mvCWmI4DuoLFh0PiqMMJBByoijRSxcSdXLPxZng84j8JVF7H6mFa-dj-icP-KLy6yvzEaRKz_uwBzQCzgYK434LIpqw_PRuN3ClEsenwRgIsNdVjvKcoAysfoZhmRy9BQaE0I7qTohSBFNX6A.mgGGeeWgugfXcUcsX4T5dQ" - ``` - -12. 后端服务根据 access_token 获取用户授权信息并返回对应的 HTTP 响应 - - ```json - { - "email": "******", - "email_verified": false, - "iss": "https://dev-o43xb1mz7ya7ach4.us.auth0.com/", - "aud": "YagFqRD9tfNIaac5BamjhsSatjrAnsnZ", - "iat": 1719198638, - "exp": 1719234638, - "sub": "auth0|665d71e74c131177be66e607", - "sid": "ct2U8_YQ-zT7E8i0E3MyK-z79diVQhaU" - } - ``` - -#### 用户令牌刷新 - -1. 模拟用户访问对应服务 API - -```shell -curl --url "foo.bar.com/headers" -``` - -2. 验证令牌的过期时间 -3. 如果在 cookie 中检测到存在 refresh_token，则可以访问相应的接口以交换新的 id_token 和 access_token - -```shell -curl -X POST \ - --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/oauth/token" \ - --data "grant_type=refresh_token" \ - --data "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ - --data "client_secret=ekqv5XoZuMFtYms1NszEqRx03qct6BPvGeJUeptNG4y09PrY16BKT9IWezTrrhJJ" \ - --data "refresh_token=GrZ1f2JvzjAZQzSXmyr1ScWbv8aMFBvzAXHBUSiILcDEG" -``` - -4. 携带 Authorization 的标头对应 access_token 访问对应 API -5. 后端服务根据 access_token 获取用户授权信息并返回对应的 HTTP 响应 - -## 配置 - -### 配置项 +## 配置字段 | Option | Type | Description | Default | | ----------------------------- | ------------ | ------------------------------------------------------------ | ----------------- | @@ -166,6 +45,8 @@ curl -X POST \ | match_rule_path | string | match rule path such as `/headers` | | | match_rule_type | string | match rule type can be `exact` or `prefix` or `regex` | | +## 使用方式 + ### 生成 Cookie 密钥 ``` python @@ -413,4 +294,133 @@ match_list: #### 登陆成功跳转到服务页面 -![aliyun_result](https://gw.alicdn.com/imgextra/i3/O1CN015pGvi51eakt3pFS8Y_!!6000000003888-0-tps-3840-2160.jpg) \ No newline at end of file +![aliyun_result](https://gw.alicdn.com/imgextra/i3/O1CN015pGvi51eakt3pFS8Y_!!6000000003888-0-tps-3840-2160.jpg) + +### OIDC 流程图 + +

+ oidc_process +

+ +### OIDC 流程解析 + +#### 用户未登录 + +1. 模拟用户访问对应服务 API + + ```shell + curl --url "foo.bar.com/headers" + ``` + +2. Higress 重定向到 OIDC Provider 登录页同时携带 client_id、response_type、scope 等 OIDC 认证的参数并设置 csrf cookie 防御CSRF 攻击 + + ```shell + curl --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/authorize"\ + --url-query "approval_prompt=force" \ + --url-query "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ + --url-query "redirect_uri=http%3A%2F%2Ffoo.bar.com%2Foauth2%2Fcallback" \ + --url-query "response_type=code" \ + --url-query "scope=openid+email+offline_access" \ + --url-query "state=nT06xdCqn4IqemzBRV5hmO73U_hCjskrH_VupPqdcdw%3A%2Ffoo" \ + --header "Set-Cookie: _oauth2_proxy_csrf=LPruATEDgcdmelr8zScD_ObhsbP4zSzvcgmPlcNDcJpFJ0OvhxP2hFotsU-kZnYxd5KsIjzeIXGTOjf8TKcbTHbDIt-aQoZORXI_0id3qeY0Jt78223DPeJ1xBqa8VO0UiEOUFOR53FGxirJOdKFxaAvxDFb1Ok=|1718962455|V1QGWyjQ4hMNOQ4Jtf17HeQJdVqHdt5d65uraFduMIU=; Path=/; Expires=Fri, 21 Jun 2024 08:06:20 GMT; HttpOnly" + ``` + +3. 重定向到登录页 + +![keycloak_login](https://gw.alicdn.com/imgextra/i4/O1CN01HLcl7r1boXwwnzGqA_!!6000000003512-0-tps-3840-2160.jpg) + +4. 用户输入用户名密码登录完成 + +5. 携带授权重定向到 Higress 并携带了 state 参数用于验证 csrf cookie ，授权码用于交换 token + + ```shell + curl --url "http://foo.bar.com/oauth2/callback" \ + --url-query "state=nT06xdCqn4IqemzBRV5hmO73U_hCjskrH_VupPqdcdw%3A%2Ffoo" \ + --url-query "code=0bdopoS2c2lx95u7iO0OH9kY1TvaEdJHo4lB6CT2_qVFm" + ``` + +6. 校验 csrf cookie 中加密存储的 state 值与 url 参数中的 state 值必须相同 + +7. 利用授权交换 id_token 和 access_token + + ```shell + curl -X POST \ + --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/oauth/token" \ + --data "grant_type=authorization_code" \ + --data "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ + --data "client_secret=ekqv5XoZuMFtYms1NszEqRx03qct6BPvGeJUeptNG4y09PrY16BKT9IWezTrrhJJ" \ + --data "redirect_uri=http%3A%2F%2Ffoo.bar.com%2Foauth2%2Fcallback" \ + --data "code=0bdopoS2c2lx95u7iO0OH9kY1TvaEdJHo4lB6CT2_qVFm" \ + ``` + + 返回的请求里包含了 id_token, access_token，refresh_token 用于后续刷新 token + + ```json + { + "access_token": "eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIiwiaXNzIjoiaHR0cHM6Ly9kZXYtbzQzeGIxbXo3eWE3YWNoNC51cy5hdXRoMC5jb20vIn0..WP_WRVM-y3fM1sN4.fAQqtKoKZNG9Wj0OhtrMgtsjTJ2J72M2klDRd9SvUKGbiYsZNPmIl_qJUf81D3VIjD59o9xrOOJIzXTgsfFVA2x15g-jBlNh68N7dyhXu9237Tbplweu1jA25IZDSnjitQ3pbf7xJVIfPnWcrzl6uT8G1EP-omFcl6AQprV2FoKFMCGFCgeafuttppKe1a8mpJDj7AFLPs-344tT9mvCWmI4DuoLFh0PiqMMJBByoijRSxcSdXLPxZng84j8JVF7H6mFa-dj-icP-KLy6yvzEaRKz_uwBzQCzgYK434LIpqw_PRuN3ClEsenwRgIsNdVjvKcoAysfoZhmRy9BQaE0I7qTohSBFNX6A.mgGGeeWgugfXcUcsX4T5dQ", + "refresh_token": "GrZ1f2JvzjAZQzSXmyr1ScWbv8aMFBvzAXHBUSiILcDEG", + "id_token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Imc1Z1ExSF9ZbTY0WUlvVkQwSVpXTCJ9.eyJlbWFpbCI6IjE2MDExNTYyNjhAcXEuY29tIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJpc3MiOiJodHRwczovL2Rldi1vNDN4YjFtejd5YTdhY2g0LnVzLmF1dGgwLmNvbS8iLCJhdWQiOiJZYWdGcVJEOXRmTklhYWM1QmFtamhzU2F0anJBbnNuWiIsImlhdCI6MTcxOTE5ODYzOCwiZXhwIjoxNzE5MjM0NjM4LCJzdWIiOiJhdXRoMHw2NjVkNzFlNzRjMTMxMTc3YmU2NmU2MDciLCJzaWQiOiJjdDJVOF9ZUS16VDdFOGkwRTNNeUstejc5ZGlWUWhhVSJ9.gfzXKJ0FeqzYqOUDLQHWcUG19IOLqkpLN09xTmIat0umrlGV5VNSumgWH3XJmmwnhdb8AThH3Jf-7kbRJzu4rM-BbGbFTRBTzNHeUajFOFrIgld5VENQ_M_sXHkTp0psWKSr9vF24kmilCfSbvC5lBKjt878ljZ7-xteWuaUYOMUdcJb4DSv0-zjX01sonJxYamTlhji3M4TAW7VwhwqyZt8dBhVSNaRw1wUKj-M1JrBDLyx65sroZtSqVA0udIrqMHEbWYb2de7JjzlqG003HRMzwOm7OXgEd5ZVFqgmBLosgixOU5DJ4A26nlqK92Sp6VqDMRvA-3ym8W_m-wJ_A", + "scope": "openid email offline_access", + "expires_in": 86400, + "token_type": "Bearer" + } + ``` + +8. 将获得的 id_token, access_token, refresh_token 加密存储在cookie _oauth2_proxy中 + +9. 重定向到用户访问的后端服务并设置 cookie，用于后续用户登录状态的验证，同时清除 cookie _oauth2_proxy_csrf + + ```json + "Set-Cookie": [ + "_oauth2_proxy_csrf=; Path=/; Expires=Mon, 24 Jun 2024 02:17:39 GMT; HttpOnly", + "_oauth2_proxy=8zM_Pcfpp_gesKFe4SMg08o5Iv0A8WAOQOmG1-vZBbQ56UggYVC0Cu-gFMEoxJZU5q1O5vqRlVBizlLetgVjRCksGVbttwl8tQ7h5YiyIubbbtvF1T4JzLh3QfzUUrwbB-VznOkh8qLbjAhddocecjBt4rMiDyceKXqMr4eO5TUEMx4vHtJYnTYalMeTYhGXk5MNSyrdZX9NnQnkdrCjiOQM13ggwob2nYwhGWaAlgzFSWkgkdtBy2Cl_YMWZ8_gKk9rDX289-JrJyGpr5k9O9RzRhZoY2iE3Mcr8-Q37RTji1Ga22QO-XkAcSaGqY1Qo7jLdmgZTYKC5JvtdLc4rj3vcbveYxU7R3Pt2vEribQjKTh4Sqb0aA03p4cxXyZN4SUfBW1NAOm4JLPUhKJy8frqC9_E0nVqPvpvnacaoQs8WkX2zp75xHoMa3SD6KZhQ5JUiPEiNkOaUsyafLvht6lLkNDhgzW3BP2czoe0DCDBLnsot0jH-qQpMZYkaGr-ZnRKI1OPl1vHls3mao5juOAW1VB2A9aughgc8SJ55IFZpMfFMdHdTDdMqPODkItX2PK44GX-pHeLxkOqrzp3GHtMInpL5QIQlTuux3erm3CG-ntlUE7JBtN2T9LEb8XfIFu58X9_vzMun4JQlje2Thi9_taI_z1DSaTtvNNb54wJfSPwYCCl4OsH-BacVmPQhH6TTZ6gP2Qsm5TR2o1U2D9fuVkSM-OPCG9l3tILambIQwC3vofMW6X8SIFSmhJUDvN7NbwxowBiZ6Y7GJRZlAk_GKDkpsdrdIvC67QqczZFphRVnm6qi-gPO41APCbcO6fgTwyOhbP3RrZZKWSIqWJYhNE3_Sfkf0565H7sC7Hc8XUUjJvP3WnjKS9x7KwzWa-dsUjV3-Q-VNl-rXTguVNAIirYK-qrMNMZGCRcJqcLnUF0V_J2lVmFyVsSlE3t0sDw2xmbkOwDptXFOjQL5Rb4esUMYdCBWFajBfvUtcZEFtYhD0kb6VcbjXO3NCVW5qKh_l9C9SRCc7TG1vcRAqUQlRXHacTGWfcWsuQkCJ3Mp_oWaDxs1GRDykQYxAn5sTICovThWEU2C6o75grWaNrkj5NU-0eHh3ryvxLmGLBOXZV9OQhtKShWmUgywSWMxOHOuZAqdAPULc8KheuGFjXYp-RnCbFYWePJmwzfQw89kSkj1KUZgMYwKEjSz62z2qc9KLczomv76ortQzvo4Hv9kaW6xVuQj5R5Oq6_WMBOqsmUMzcXpxCIOGjcdcZRBc0Fm09Uy9oV1PRqvAE4PGtfyrCaoqILBix8UIww63B07YGwzQ-hAXDysBK-Vca2x7GmGdXsNXXcTgu00bdsjtHZPDBBWGfL3g_rMAXr2vWyvK4CwNjcaPAmrlF3geHPwbIePT0hskBboX1v1bsuhzsai7rGM4r53pnb1ZEoTQDa1B-HyokFgo14XiwME0zE1ifpNzefjpkz1YY2krJlqfCydNwoKaTit4tD2yHlnxAeFF9iIrxzSKErNUFpmyLa7ge7V33vhEH-6k5oBTLE2Q2BrC6aAkLCcPwU9xv_SzBDQPRY0MEYv3kGF03Swo1crRbGh-aifYX9NiHDsmG6r1vAnx0MAOw2Jzuz2x6SSdfBrzlcoWBlrwiZzd9kAKq75n1Uy9uzZ8SRnkBrEZySHBwEbu196VklkRE0jqwC-e3wWNNuviSOfwkVeX-7QdOoO10yw9VK2sW52lFvIEf4chv_ta7bGfAZOWBjpktG6ZLD81SE6A88zpqG2SysSyNMp9hl-umG-5sFsjCn_c9E8bDvwkUOUVb9bNqhBDsZgR0BNPawiOZjmyfhzmwmWf-zgFzfFSV6BvOwNRi3sCOHTsWcuk9NBQ_YK8CpNkVl3WeIBSDfidimuC_QV9UWKs1GPk35ZRkM4zKtLY2JsBFWKaDy_P80TcOzcMBoP8gIBClXZ-WUqfE8s1yyc4jrq-qL1_wJ24ef1O9FktsbyZiDKXw2vnqsT8-g_hCeG-unrT1ZFscf8oNdqczARHX-K4vKH2k3uIqEx1M=|1719199056|2rsgdUIClHNEpxBLlHOVRYup6e4oKensQfljtmn4B80=; Path=/; Expires=Mon, 01 Jul 2024 03:17:36 GMT; HttpOnly" + ] + ``` + +10. 校验是否存在 cookie 存储了用户的 token 信息同时查看是否过期 + +11. 使用含有 Authorization 头部存储用户的 access_token 访问相应的 API + + ```shell + curl --url "foo.bar.com/headers" + --header "Authorization: Bearer eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIiwiaXNzIjoiaHR0cHM6Ly9kZXYtbzQzeGIxbXo3eWE3YWNoNC51cy5hdXRoMC5jb20vIn0..WP_WRVM-y3fM1sN4.fAQqtKoKZNG9Wj0OhtrMgtsjTJ2J72M2klDRd9SvUKGbiYsZNPmIl_qJUf81D3VIjD59o9xrOOJIzXTgsfFVA2x15g-jBlNh68N7dyhXu9237Tbplweu1jA25IZDSnjitQ3pbf7xJVIfPnWcrzl6uT8G1EP-omFcl6AQprV2FoKFMCGFCgeafuttppKe1a8mpJDj7AFLPs-344tT9mvCWmI4DuoLFh0PiqMMJBByoijRSxcSdXLPxZng84j8JVF7H6mFa-dj-icP-KLy6yvzEaRKz_uwBzQCzgYK434LIpqw_PRuN3ClEsenwRgIsNdVjvKcoAysfoZhmRy9BQaE0I7qTohSBFNX6A.mgGGeeWgugfXcUcsX4T5dQ" + ``` + +12. 后端服务根据 access_token 获取用户授权信息并返回对应的 HTTP 响应 + + ```json + { + "email": "******", + "email_verified": false, + "iss": "https://dev-o43xb1mz7ya7ach4.us.auth0.com/", + "aud": "YagFqRD9tfNIaac5BamjhsSatjrAnsnZ", + "iat": 1719198638, + "exp": 1719234638, + "sub": "auth0|665d71e74c131177be66e607", + "sid": "ct2U8_YQ-zT7E8i0E3MyK-z79diVQhaU" + } + ``` + +#### 用户令牌刷新 + +1. 模拟用户访问对应服务 API + +```shell +curl --url "foo.bar.com/headers" +``` + +2. 验证令牌的过期时间 +3. 如果在 cookie 中检测到存在 refresh_token，则可以访问相应的接口以交换新的 id_token 和 access_token + +```shell +curl -X POST \ + --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/oauth/token" \ + --data "grant_type=refresh_token" \ + --data "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ + --data "client_secret=ekqv5XoZuMFtYms1NszEqRx03qct6BPvGeJUeptNG4y09PrY16BKT9IWezTrrhJJ" \ + --data "refresh_token=GrZ1f2JvzjAZQzSXmyr1ScWbv8aMFBvzAXHBUSiILcDEG" +``` + +4. 携带 Authorization 的标头对应 access_token 访问对应 API +5. 后端服务根据 access_token 获取用户授权信息并返回对应的 HTTP 响应 + diff --git a/plugins/wasm-go/extensions/oidc/README_EN.md b/plugins/wasm-go/extensions/oidc/README_EN.md new file mode 100644 index 0000000000..3f37330103 --- /dev/null +++ b/plugins/wasm-go/extensions/oidc/README_EN.md @@ -0,0 +1,354 @@ +--- +title: OIDC Authentication +keywords: [higress, oidc] +description: OIDC Authentication Plugin Configuration Reference +--- +## Function Description +This plugin supports OpenID Connect (OIDC) identity authentication. Additionally, it enhances defenses against Cross-Site Request Forgery (CSRF) attacks and supports the Logout Endpoint and Refresh Token mechanism in the OpenID Connect protocol. Requests verified by the Wasm plugin after OIDC validation will carry the `Authorization` header, including the corresponding Access Token. + +## Running Attributes +Plugin execution phase: `authentication phase` + +Plugin execution priority: `350` + +## Configuration Fields +| Option | Type | Description | Default | +| ----------------------------- | ------------ | ------------------------------------------------------------ | ----------------- | +| cookie_name | string | The name of the cookie that the oauth_proxy creates. Should be changed to use a [cookie prefix](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#cookie_prefixes) (`__Host-` or `__Secure-`) if `--cookie-secure` is set. | `"_oauth2_proxy"` | +| cookie_secret | string | The seed string for secure cookies (optionally base64 encoded) | | +| cookie_domains | string\|list | Optional cookie domains to force cookies to (e.g. `.yourcompany.com`). The longest domain matching the request's host will be used (or the shortest cookie domain if there is no match). | | +| cookie_path | string | An optional cookie path to force cookies to (e.g. `/poc/`) | `"/"` | +| cookie_expire | duration | Expire timeframe for cookie. If set to 0, cookie becomes a session-cookie which will expire when the browser is closed. | 168h0m0s | +| cookie_refresh | duration | Refresh the cookie after this duration; `0` to disable | | +| cookie_secure | bool | Set [secure (HTTPS only) cookie flag](https://owasp.org/www-community/controls/SecureFlag) | true | +| cookie_httponly | bool | Set HttpOnly cookie flag | true | +| cookie_samesite | string | Set SameSite cookie attribute (`"lax"`, `"strict"`, `"none"`, or `""`). | `""` | +| cookie_csrf_per_request | bool | Enable having different CSRF cookies per request, making it possible to have parallel requests. | false | +| cookie_csrf_expire | duration | Expire timeframe for CSRF cookie | 15m | +| client_id | string | The OAuth Client ID | | +| client_secret | string | The OAuth Client Secret | | +| provider | string | OAuth provider | oidc | +| pass_authorization_header | bool | Pass OIDC IDToken to upstream via Authorization Bearer header | true | +| oidc_issuer_url | string | The OpenID Connect issuer URL, e.g. `"https://dev-o43xb1mz7ya7ach4.us.auth0.com"` | | +| oidc_verifier_request_timeout | uint32 | OIDC verifier discovery request timeout | 2000(ms) | +| scope | string | OAuth scope specification | | +| redirect_url | string | The OAuth Redirect URL, e.g. `"https://internalapp.yourcompany.com/oauth2/callback"` | | +| service_name | string | Registered name of the OIDC service, e.g. `auth.dns`, `keycloak.static` | | +| service_port | int64 | Service port of the OIDC service | | +| service_host | string | Host of the OIDC service when type is static IP | | +| match_type | string | Match type (`whitelist` or `blacklist`) | `"whitelist"` | +| match_list | rule\|list | A list of (match_rule_domain, match_rule_path, and match_rule_type). | | +| match_rule_domain | string | Match rule domain, support wildcard pattern such as `*.bar.com` | | +| match_rule_path | string | Match rule path such as `/headers` | | +| match_rule_type | string | Match rule type can be `exact` or `prefix` or `regex` | | + +## Usage +### Generate Cookie Secret +``` python +python -c 'import os, base64; print(base64.urlsafe_b64encode(os.urandom(32)).decode())' +``` + +Reference: [Oauth2-proxy Generating a Cookie Secret](https://oauth2-proxy.github.io/oauth2-proxy/configuration/overview#generating-a-cookie-secret) + +### Whitelist and Blacklist Mode +Supports whitelist and blacklist configuration, defaulting to whitelist mode, which is empty, meaning all requests need to be validated. Domain matching supports wildcard domains like `*.bar.com`, and matching rules support exact match `exact`, prefix match `prefix`, and regex match `regex`. + +* **Whitelist Mode** +```yaml +match_type: 'whitelist' +match_list: + - match_rule_domain: '*.bar.com' + match_rule_path: '/foo' + match_rule_type: 'prefix' +``` + +Requests matching the prefix `/foo` under the wildcard domain `*.bar.com` do not need validation. + +* **Blacklist Mode** +```yaml +match_type: 'blacklist' +match_list: + - match_rule_domain: '*.bar.com' + match_rule_path: '/headers' + match_rule_type: 'prefix' +``` + +Only requests matching the prefix `/headers` under the wildcard domain `*.bar.com` need validation. + +### Logout Users +To log out users, they must be redirected to the `/oauth2/sign_out` endpoint. This endpoint only removes cookies set by oauth2-proxy, meaning the user remains logged into the OIDC Provider and may be automatically logged back in upon re-accessing the application. Therefore, the `rd` query parameter must be used to redirect the user to the authentication provider's logout page, redirecting the user to a URL similar to the following (note URL encoding!): + +``` +/oauth2/sign_out?rd=https%3A%2F%2Fmy-oidc-provider.example.com%2Fsign_out_page +``` + +Alternatively, the redirect URL can be included in the `X-Auth-Request-Redirect` header: + +``` +GET /oauth2/sign_out HTTP/1.1 +X-Auth-Request-Redirect: https://my-oidc-provider.example.com/sign_out_page +... +``` + +The redirect URL can include the `post_logout_redirect_uri` parameter to specify the page to redirect to after OIDC Provider logout; for example, the backend service's logout page. If this parameter is not included, it will default to redirect to the OIDC Provider's logout page. For details, see the examples below for Auth0 and Keycloak (if the OIDC Provider supports session management and discovery, then "sign_out_page" should be obtained from the [metadata](https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfig) `end_session_endpoint`). + +### OIDC Service HTTPS Protocol +If the OIDC Provider uses the HTTPS protocol, refer to Higress's documentation on [Configuring Backend Service Protocol: HTTPS](https://higress.io/docs/latest/user/annotation-use-case/#%E9%85%8D%E7%BD%AE%E5%90%8E%E7%AB%AF%E6%9C%8D%E5%8A%A1%E5%8D%8F%E8%AE%AEhttps%E6%88%96grpc), which states that requests forwarded to the backend service should use the HTTPS protocol by using the annotation `higress.io/backend-protocol: "HTTPS"`, as shown in the Auth0 example Ingress configuration. + +## Configuration Example +### Auth0 Configuration Example +#### Step 1: Configure Auth0 Account +- Log in to the Developer Okta website [Developer Auth0 site](https://auth0.com/) +- Register a test web application +**Note**: You must fill in the Allowed Callback URLs, Allowed Logout URLs, Allowed Web Origins, etc., otherwise the OIDC Provider will consider the user's redirect URL or logout URL to be invalid. + +#### Step 2: Higress Configure Service Source +* Create an Auth0 DNS source in Higress service sources. +![auth0 create](https://gw.alicdn.com/imgextra/i1/O1CN01p9y0jF1tfzdXTzNYm_!!6000000005930-0-tps-3362-670.jpg) + +#### Step 3: OIDC Service HTTPS Configuration +```yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: auth0-ingress + annotations: + higress.io/destination: auth.dns + higress.io/backend-protocol: "HTTPS" + higress.io/ignore-path-case: "false" +spec: + ingressClassName: higress + rules: + - host: foo.bar.com + http: + paths: + - path: / + pathType: Prefix + backend: + resource: + apiGroup: networking.higress.io + kind: McpBridge + name: default +``` + +#### Step 4: Wasm Plugin Configuration +```yaml +redirect_url: 'http://foo.bar.com/oauth2/callback' +oidc_issuer_url: 'https://dev-o43xb1mz7ya7ach4.us.auth0.com/' +client_id: 'XXXXXXXXXXXXXXXX' +client_secret: 'XXXXXXXXXXXXXXXX' +scope: 'openid email offline_access' +cookie_secret: 'nqavJrGvRmQxWwGNptLdyUVKcBNZ2b18Guc1n_8DCfY=' +service_name: 'auth.dns' +service_port: 443 +match_type: 'whitelist' +match_list: + - match_rule_domain: '*.bar.com' + match_rule_path: '/foo' + match_rule_type: 'prefix' +``` + +**Note**: You must configure the service source first. The Wasm plugin needs to access the configured service to obtain OpenID configuration during initialization. + +#### Access Service Page; Redirect if Not Logged In +![auth0_login](https://gw.alicdn.com/imgextra/i3/O1CN01hVNk0C1gkUWLwuC0N_!!6000000004180-0-tps-3840-2160.jpg) + +#### Successful Login Redirects to Service Page +In the headers, you can see the `_oauth2_proxy` cookie carried for the next login access, and the Authorization corresponds to the IDToken used to get user information from the backend service. +![auth0 service](https://gw.alicdn.com/imgextra/i1/O1CN01vyrB6u1xPHep1RRqb_!!6000000006435-2-tps-3840-2160.png) + +#### Access Logout Redirects to Logout Page +``` +http://foo.bar.com/oauth2/sign_out?rd=https%3A%2F%2Fdev-o43xb1mz7ya7ach4.us.auth0.com%2Foidc%2Flogout +``` + +![auth0 logout](https://gw.alicdn.com/imgextra/i3/O1CN01UntF4x1UqC4StMqtT_!!6000000002568-0-tps-3840-2160.jpg) + +#### Access Logout Redirects to Logout Page (with post_logout_redirect_uri Parameter to Redirect to Specified URI) +``` +http://foo.bar.com/oauth2/sign_out?rd=https%3A%2F%2Fdev-o43xb1mz7ya7ach4.us.auth0.com%2Foidc%2Flogout%3Fpost_logout_redirect_uri%3Dhttp%3A%2F%2Ffoo.bar.com%2Ffoo +``` + +Note: The URI to which `post_logout_redirect_uri` redirects needs to be configured in the OIDC Provider Allowed URLs for a normal redirection. + +![auth0 logout redirect](https://gw.alicdn.com/imgextra/i1/O1CN01AtZ2cd1JlBxsgyCjG_!!6000000001068-0-tps-3840-2160.jpg) + +### Keycloak Configuration Example +#### Step 1: Get Started with Keycloak on Docker + +**Note**: You must fill in Valid redirect URIs, Valid post logout URIs, Web origins, otherwise the OIDC Provider will consider the user's redirect URL or logout URL to be invalid. + +#### Step 2: Higress Configure Service Source +* Create a Keycloak fixed address service in Higress service sources. +![keycloak create](https://gw.alicdn.com/imgextra/i1/O1CN01p9y0jF1tfzdXTzNYm_!!6000000005930-0-tps-3362-670.jpg) + +#### Step 3: Wasm Plugin Configuration +```yaml +redirect_url: 'http://foo.bar.com/oauth2/callback' +oidc_issuer_url: 'http://127.0.0.1:9090/realms/myrealm' +client_id: 'XXXXXXXXXXXXXXXX' +client_secret: 'XXXXXXXXXXXXXXXX' +scope: 'openid email' +cookie_secret: 'nqavJrGvRmQxWwGNptLdyUVKcBNZ2b18Guc1n_8DCfY=' +service_name: 'keycloak.static' +service_port: 80 +service_host: '127.0.0.1:9090' +match_type: 'blacklist' +match_list: + - match_rule_domain: '*.bar.com' + match_rule_path: '/headers' + match_rule_type: 'prefix' +``` + +#### Access Service Page; Redirect if Not Logged In +![keycloak_login](https://gw.alicdn.com/imgextra/i4/O1CN01HLcl7r1boXwwnzGqA_!!6000000003512-0-tps-3840-2160.jpg) + +#### Successful Login Redirects to Service Page +![keycloak service](https://gw.alicdn.com/imgextra/i1/O1CN01vyrB6u1xPHep1RRqb_!!6000000006435-2-tps-3840-2160.png) + +#### Access Logout Redirects to Logout Page +``` +http://foo.bar.com/oauth2/sign_out?rd=http%3A%2F%2F127.0.0.1:9090%2Frealms%2Fmyrealm%2Fprotocol%2Fopenid-connect%2Flogout +``` + +![keycloak logout](https://gw.alicdn.com/imgextra/i4/O1CN01kQwqB523OiroOWMgM_!!6000000007246-0-tps-3840-2160.jpg) + +#### Access Logout Redirects to Logout Page (with post_logout_redirect_uri Parameter to Redirect to Specified URI) +``` +http://foo.bar.com/oauth2/sign_out?rd=http%3A%2F%2F127.0.0.1:9090%2Frealms%2Fmyrealm%2Fprotocol%2Fopenid-connect%2Flogout%3Fpost_logout_redirect_uri%3Dhttp%3A%2F%2Ffoo.bar.com%2Ffoo +``` + +![keycloak logout redirect](https://gw.alicdn.com/imgextra/i1/O1CN01AtZ2cd1JlBxsgyCjG_!!6000000001068-0-tps-3840-2160.jpg) + +### Aliyun Configuration Example +#### Step 1: Configure Aliyun OAuth Application +Refer to the [Web Application Login to Alibaba Cloud](https://help.aliyun.com/zh/ram/user-guide/access-alibaba-cloud-apis-from-a-web-application) process to configure the OAuth application. + +#### Step 2: Higress Configure Service Source +* Create an Aliyun DNS service in Higress service sources. +![Aliyun service](https://gw.alicdn.com/imgextra/i3/O1CN01PMNGFS1mHXBtsEvEq_!!6000000004929-0-tps-3312-718.jpg) + +#### Step 3: Wasm Plugin Configuration +```yaml +redirect_url: 'http://foo.bar.com/oauth2/callback' +provider: aliyun +oidc_issuer_url: 'https://oauth.aliyun.com/' +client_id: 'XXXXXXXXXXXXXXXX' +client_secret: 'XXXXXXXXXXXXXXXX' +scope: 'openid' +cookie_secret: 'nqavJrGvRmQxWwGNptLdyUVKcBNZ2b18Guc1n_8DCfY=' +service_name: 'aliyun.dns' +service_port: 443 +match_type: whitelist +match_list: + - match_rule_domain: 'foo.bar.com' + match_rule_path: /foo + match_rule_type: prefix +``` + +#### Access Service Page; Redirect if Not Logged In +![aliyun_login_1](https://gw.alicdn.com/imgextra/i1/O1CN01L379Uk1b2umAraylT_!!6000000003408-0-tps-3840-2160.jpg) +Directly login using a RAM user or click the main account login. +![aliyun_login_2](https://gw.alicdn.com/imgextra/i1/O1CN01pfdA3l27Dy2TL83NA_!!6000000007764-0-tps-3840-2160.jpg) + +#### Successful Login Redirects to Service Page +![aliyun_result](https://gw.alicdn.com/imgextra/i3/O1CN015pGvi51eakt3pFS8Y_!!6000000003888-0-tps-3840-2160.jpg) + +### OIDC Flow Diagram +

+ oidc_process +

+ +### OIDC Flow Analysis +#### User Not Logged In +1. Simulate user accessing the corresponding service API + ```shell + curl --url "foo.bar.com/headers" + ``` +2. Higress redirects to the OIDC Provider login page while carrying the client_id, response_type, scope, and other OIDC authentication parameters, setting a CSRF cookie to defend against CSRF attacks. + ```shell + curl --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/authorize"\ + --url-query "approval_prompt=force" \ + --url-query "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ + --url-query "redirect_uri=http%3A%2F%2Ffoo.bar.com%2Foauth2%2Fcallback" \ + --url-query "response_type=code" \ + --url-query "scope=openid+email+offline_access" \ + --url-query "state=nT06xdCqn4IqemzBRV5hmO73U_hCjskrH_VupPqdcdw%3A%2Ffoo" \ + --header "Set-Cookie: _oauth2_proxy_csrf=LPruATEDgcdmelr8zScD_ObhsbP4zSzvcgmPlcNDcJpFJ0OvhxP2hFotsU-kZnYxd5KsIjzeIXGTOjf8TKcbTHbDIt-aQoZORXI_0id3qeY0Jt78223DPeJ1xBqa8VO0UiEOUFOR53FGxirJOdKFxaAvxDFb1Ok=|1718962455|V1QGWyjQ4hMNOQ4Jtf17HeQJdVqHdt5d65uraFduMIU=; Path=/; Expires=Fri, 21 Jun 2024 08:06:20 GMT; HttpOnly" + ``` +3. Redirect to the login page. +![keycloak_login](https://gw.alicdn.com/imgextra/i4/O1CN01HLcl7r1boXwwnzGqA_!!6000000003512-0-tps-3840-2160.jpg) +4. The user enters the username and password to log in. +5. A redirect back to Higress occurs with the authorization code and state parameter for CSRF cookie validation. + ```shell + curl --url "http://foo.bar.com/oauth2/callback" \ + --url-query "state=nT06xdCqn4IqemzBRV5hmO73U_hCjskrH_VupPqdcdw%3A%2Ffoo" \ + --url-query "code=0bdopoS2c2lx95u7iO0OH9kY1TvaEdJHo4lB6CT2_qVFm" + ``` +6. Verify that the encrypted state value stored in the CSRF cookie matches the state value in the URL parameters. +7. Use the authorization code to exchange for id_token and access_token. + ```shell + curl -X POST \ + --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/oauth/token" \ + --data "grant_type=authorization_code" \ + --data "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ + --data "client_secret=ekqv5XoZuMFtYms1NszEqRx03qct6BPvGeJUeptNG4y09PrY16BKT9IWezTrrhJJ" \ + --data "redirect_uri=http%3A%2F%2Ffoo.bar.com%2Foauth2%2Fcallback" \ + --data "code=0bdopoS2c2lx95u7iO0OH9kY1TvaEdJHo4lB6CT2_qVFm" \ + ``` + The response will include id_token, access_token, and refresh_token for future token refresh. + ```json + { + "access_token": "eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIiwiaXNzIjoiaHR0cHM6Ly9kZXYtbzQzeGIxbXo3eWE3YWNoNC51cy5hdXRoMC5jb20vIn0..WP_WRVM-y3fM1sN4.fAQqtKoKZNG9Wj0OhtrMgtsjTJ2J72M2klDRd9SvUKGbiYsZNPmIl_qJUf81D3VIjD59o9xrOOJIzXTgsfFVA2x15g-jBlNh68N7dyhXu9237Tbplweu1jA25IZDSnjitQ3pbf7xJVIfPnWcrzl6uT8G1EP-omFcl6AQprV2FoKFMCGFCgeafuttppKe1a8mpJDj7AFLPs-344tT9mvCWmI4DuoLFh0PiqMMJBByoijRSxcSdXLPxZng84j8JVF7H6mFa-dj-icP-KLy6yvzEaRKz_uwBzQCzgYK434LIpqw_PRuN3ClEsenwRgIsNdVjvKcoAysfoZhmRy9BQaE0I7qTohSBFNX6A.mgGGeeWgugfXcUcsX4T5dQ", + "refresh_token": "GrZ1f2JvzjAZQzSXmyr1ScWbv8aMFBvzAXHBUSiILcDEG", + "id_token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Imc1Z1ExSF9ZbTY0WUlvVkQwSVpXTCJ9.eyJlbWFpbCI6IjE2MDExNTYyNjhAcXEuY29tIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJpc3MiOiJodHRwczovL2Rldi1vNDN4YjFtejd5YTdhY2g0LnVzLmF1dGgwLmNvbS8iLCJhdWQiOiJZYWdGcVJEOXRmTklhYWM1QmFtamhzU2F0anJBbnNuWiIsImlhdCI6MTcxOTE5ODYzOCwiZXhwIjoxNzE5MjM0NjM4LCJzdWIiOiJhdXRoMHw2NjVkNzFlNzRjMTMxMTc3YmU2NmU2MDciLCJzaWQiOiJjdDJVOF9ZUS16VDdFOGkwRTNNeUstejc5ZGlWUWhhVSJ9.gfzXKJ0FeqzYqOUDLQHWcUG19IOLqkpLN09xTmIat0umrlGV5VNSumgWH3XJmmwnhdb8AThH3Jf-7kbRJzu4rM-BbGbFTRBTzNHeUajFOFrIgld5VENQ_M_sXHkTp0psWKSr9vF24kmilCfSbvC5lBKjt878ljZ7-xteWuaUYOMUdcJb4DSv0-zjX01sonJxYamTlhji3M4TAW7VwhwqyZt8dBhVSNaRw1wUKj-M1JrBDLyx65sroZtSqVA0udIrqMHEbWYb2de7JjzlqG003HRMzwOm7OXgEd5ZVFqgmBLosgixOU5DJ4A26nlqK92Sp6VqDMRvA-3ym8W_m-wJ_A", + "scope": "openid email offline_access", + "expires_in": 86400, + "token_type": "Bearer" + } + ``` +8. The obtained id_token, access_token, and refresh_token are encrypted and stored in the `_oauth2_proxy` cookie. +9. Redirect to the backend service accessed by the user and set cookies for subsequent user login state verification while clearing the `_oauth2_proxy_csrf` cookie. + ```json + "Set-Cookie": [ + "_oauth2_proxy_csrf=; Path=/; Expires=Mon, 24 Jun 2024 02:17:39 GMT; HttpOnly", + "_oauth2_proxy=8zM_Pcfpp_gesKFe4SMg08o5Iv0A8WAOQOmG1-vZBbQ56UggYVC0Cu-gFMEoxJZU5q1O5vqRlVBizlLetgVjRCksGVbttwl8tQ7h5YiyIubbbtvF1T4JzLh3QfzUUrwbB-VznOkh8qLbjAhddocecjBt4rMiDyceKXqMr4eO5TUEMx4vHtJYnTYalMeTYhGXk5MNSyrdZX9NnQnkdrCjiOQM13ggwob2nYwhGWaAlgzFSWkgkdtBy2Cl_YMWZ8_gKk9rDX289-JrJyGpr5k9O9RzRhZoY2iE3Mcr8-Q37RTji1Ga22QO-XkAcSaGqY1Qo7jLdmgZTYKC5JvtdLc4rj3vcbveYxU7R3Pt2vEribQjKTh4Sqb0aA03p4cxXyZN4SUfBW1NAOm4JLPUhKJy8frqC9_E0nVqPvpvnacaoQs8WkX2zp75xHoMa3SD6KZhQ5JUiPEiNkOaUsyafLvht6lLkNDhgzW3BP2czoe0DCDBLnsot0jH-qQpMZYkaGr-ZnRKI1OPl1vHls3mao5juOAW1VB2A9aughgc8SJ55IFZpMfFMdHdTDdMqPODkItX2PK44GX-pHeLxkOqrzp3GHtMInpL5QIQlTuux3erm3CG-ntlUE7JBtN2T9LEb8XfIFu58X9_vzMun4JQlje2Thi9_taI_z1DSaTtvNNb54wJfSPwYCCl4OsH-BacVmPQhH6TTZ6gP2Qsm5TR2o1U2D9fuVkSM-OPCG9l3tILambIQwC3vofMW6X8SIFSmhJUDvN7NbwxowBiZ6Y7GJRZlAk_GKDkpsdrdIvC67QqczZFphRVnm6qi-gPO41APCbcO6fgTwyOhbP3RrZZKWSIqWJYhNE3_Sfkf0565H7sC7Hc8XUUjJvP3WnjKS9x7KwzWa-dsUjV3-Q-VNl-rXTguVNAIirYK-qrMNMZGCRcJqcLnUF0V_J2lVmFyVsSlE3t0sDw2xmbkOwDptXFOjQL5Rb4esUMYdCBWFajBfvUtcZEFtYhD0kb6VcbjXO3NCVW5qKh_l9C9SRCc7TG1vcRAqUQlRXHacTGWfcWsuQkCJ3Mp_oWaDxs1GRDykQYxAn5sTICovThWEU2C6o75grWaNrkj5NU-0eHh3ryvxLmGLBOXZV9OQhtKShWmUgywSWMxOHOuZAqdAPULc8KheuGFjXYp-RnCbFYWePJmwzfQw89kSkj1KUZgMYwKEjSz62z2qc9KLczomv76ortQzvo4Hv9kaW6xVuQj5R5Oq6_WMBOqsmUMzcXpxCIOGjcdcZRBc0Fm09Uy9oV1PRqvAE4PGtfyrCaoqILBix8UIww63B07YGwzQ-hAXDysBK-Vca2x7GmGdXsNXXcTgu00bdsjtHZPDBBWGfL3g_rMAXr2vWyvK4CwNjcaPAmrlF3geHPwbIePT0hskBboX1v1bsuhzsai7rGM4r53pnb1ZEoTQDa1B-HyokFgo14XiwME0zE1ifpNzefjpkz1YY2krJlqfCydNwoKaTit4tD2yHlnxAeFF9iIrxzSKErNUFpmyLa7ge7V33vhEH-6k5oBTLE2Q2BrC6aAkLCcPwU9xv_SzBDQPRY0MEYv3kGF03Swo1crRbGh-aifYX9NiHDsmG6r1vAnx0MAOw2Jzuz2x6SSdfBrzlcoWBlrwiZzd9kAKq75n1Uy9uzZ8SRnkBrEZySHBwEbu196VklkRE0jqwC-e3wWNNuviSOfwkVeX-7QdOoO10yw9VK2sW52lFvIEf4chv_ta7bGfAZOWBjpktG6ZLD81SE6A88zpqG2SysSyNMp9hl-umG-5sFsjCn_c9E8bDvwkUOUVb9bNqhBDsZgR0BNPawiOZjmyfhzmwmWf-zgFzfFSV6BvOwNRi3sCOHTsWcuk9NBQ_YK8CpNkVl3WeIBSDfidimuC_QV9UWKs1GPk35ZRkM4zKtLY2JsBFWKaDy_P80TcOzcMBoP8gIBClXZ-WUqfE8s1yyc4jrq-qL1_wJ24ef1O9FktsbyZiDKXw2vnqsT8-g_hCeG-unrT1ZFscf8oNdqczARHX-K4vKH2k3uIqEx1M=|1719199056|2rsgdUIClHNEpxBLlHOVRYup6e4oKensQfljtmn4B80=; Path=/; Expires=Mon, 01 Jul 2024 03:17:36 GMT; HttpOnly" + ] + ``` +10. Verify the existence of the cookie storing user's token information and check if it has expired. +11. Use the access token with the Authorization header to access the corresponding API. + ```shell + curl --url "foo.bar.com/headers" + --header "Authorization: Bearer eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIiwiaXNzIjoiaHR0cHM6Ly9kZXYtbzQzeGIxbXo3eWE3YWNoNC51cy5hdXRoMC5jb20vIn0..WP_WRVM-y3fM1sN4.fAQqtKoKZNG9Wj0OhtrMgtsjTJ2J72M2klDRd9SvUKGbiYsZNPmIl_qJUf81D3VIjD59o9xrOOJIzXTgsfFVA2x15g-jBlNh68N7dyhXu9237Tbplweu1jA25IZDSnjitQ3pbf7xJVIfPnWcrzl6uT8G1EP-omFcl6AQprV2FoKFMCGFCgeafuttppKe1a8mpJDj7AFLPs-344tT9mvCWmI4DuoLFh0PiqMMJBByoijRSxcSdXLPxZng84j8JVF7H6mFa-dj-icP-KLy6yvzEaRKz_uwBzQCzgYK434LIpqw_PRuN3ClEsenwRgIsNdVjvKcoAysfoZhmRy9BQaE0I7qTohSBFNX6A.mgGGeeWgugfXcUcsX4T5dQ" + ``` +12. The backend service obtains user authorization information based on the access token and returns the corresponding HTTP response. + ```json + { + "email": "******", + "email_verified": false, + "iss": "https://dev-o43xb1mz7ya7ach4.us.auth0.com/", + "aud": "YagFqRD9tfNIaac5BamjhsSatjrAnsnZ", + "iat": 1719198638, + "exp": 1719234638, + "sub": "auth0|665d71e74c131177be66e607", + "sid": "ct2U8_YQ-zT7E8i0E3MyK-z79diVQhaU" + } + ``` + +#### User Token Refresh +1. Simulate user accessing the corresponding service API. +```shell +curl --url "foo.bar.com/headers" +``` +2. Verify the expiration time of the token. +3. If a refresh_token is detected in the cookie, access the corresponding interface to exchange a new id_token and access_token. +```shell +curl -X POST \ + --url "https://dev-o43xb1mz7ya7ach4.us.auth0.com/oauth/token" \ + --data "grant_type=refresh_token" \ + --data "client_id=YagFqRD9tfNIaac5BamjhsSatjrAnsnZ" \ + --data "client_secret=ekqv5XoZuMFtYms1NszEqRx03qct6BPvGeJUeptNG4y09PrY16BKT9IWezTrrhJJ" \ + --data "refresh_token=GrZ1f2JvzjAZQzSXmyr1ScWbv8aMFBvzAXHBUSiILcDEG" +``` +4. Access the corresponding API with the access token using the Authorization header. +5. The backend service obtains user authorization information based on the access token and returns the corresponding HTTP response. diff --git a/plugins/wasm-go/extensions/opa/README.md b/plugins/wasm-go/extensions/opa/README.md index 80428338ee..4c6e0c276a 100644 --- a/plugins/wasm-go/extensions/opa/README.md +++ b/plugins/wasm-go/extensions/opa/README.md @@ -1,10 +1,17 @@ -# 功能说明 +--- +title: OPA +keywords: [higress,opa] +description: OPA 策略控制插件配置参考 +--- + +## 功能说明 该插件实现了 `OPA` 策略控制 -# 该教程使用k8s，[k8s配置文件](../../../../test/e2e/conformance/tests/go-wasm-opa.yaml) +## 运行属性 -支持client `k8s,nacos,ip,route` 策略去访问 +插件执行阶段：`认证阶段` +插件执行优先级：`225` ## 配置字段 @@ -18,8 +25,6 @@ | servicePort | string | 非必填 | - | 服务端口（serviceSource为`k8s,nacos,ip`必填） | | namespace | string | 非必填 | - | 服务端口（serviceSource为`k8s,nacos`必填） | -这是一个用于OPA认证配置的表格，确保在提供所有必要的信息时遵循上述指导。 - ## 配置示例 ```yaml @@ -31,15 +36,15 @@ policy: example1 timeout: 5s ``` -# 在宿主机上执行OPA的流程 +## OPA 服务安装参考 -## 启动opa服务 +### 启动 OPA 服务 ```shell docker run -d --name opa -p 8181:8181 openpolicyagent/opa:0.35.0 run -s ``` -## 创建opa策略 +### 创建 OPA 策略 ```shell curl -X PUT '127.0.0.1:8181/v1/policies/example1' \ @@ -56,46 +61,10 @@ allow { }' ``` -## 查询策略 +### 查询策略 ```shell curl -X POST '127.0.0.1:8181/v1/data/example1/allow' \ -H 'Content-Type: application/json' \ -d '{"input":{"request":{"method":"GET"}}}' ``` - -# 测试插件 - -## 打包 WASM 插件 - -> 在 `wasm-go` 目录下把Dockerfile文件改成`PLUGIN_NAME=opa`，然后执行以下命令 - -```shell -docker build -t build-wasm-opa --build-arg GOPROXY=https://goproxy.cn,direct --platform=linux/amd64 . -``` - -## 拷贝插件 - -> 在当前的目录执行以下命令，将插件拷贝当前的目录 - -```shell -docker cp wasm-opa:/plugin.wasm . -``` - -## 运行插件 - -> 运行前修改envoy.yaml 这两个字段 `OPA_SERVER` `OPA_PORT` 替换宿主机上的IP和端口 - -```shell -docker compose up -``` - -## 使用curl测试插件 - -```shell -curl http://127.0.0.1:10000/get -X GET -v -``` - -```shell -curl http://127.0.0.1:10000/get -X POST -v -``` \ No newline at end of file diff --git a/plugins/wasm-go/extensions/opa/README_EN.md b/plugins/wasm-go/extensions/opa/README_EN.md new file mode 100644 index 0000000000..d5b28a5e21 --- /dev/null +++ b/plugins/wasm-go/extensions/opa/README_EN.md @@ -0,0 +1,58 @@ +--- +title: OPA +keywords: [higress,opa] +description: OPA policy control plugin configuration reference +--- +## Function Description +This plugin implements `OPA` policy control. + +## Running Attributes +Plugin Execution Phase: `Authentication Phase` +Plugin Execution Priority: `225` + +## Configuration Fields +| Field | Data Type | Required | Default Value | Description | +|------------------|-------------|----------|---------------|-----------------------------------------------| +| policy | string | Required | - | OPA Policy | +| timeout | string | Required | - | Timeout setting for access | +| serviceSource | string | Required | - | k8s, nacos, ip, route | +| host | string | Optional | - | Service host (required if serviceSource is `ip`) | +| serviceName | string | Optional | - | Service name (required if serviceSource is `k8s,nacos,ip`) | +| servicePort | string | Optional | - | Service port (required if serviceSource is `k8s,nacos,ip`) | +| namespace | string | Optional | - | Namespace (required if serviceSource is `k8s,nacos`) | + +## Configuration Example +```yaml +serviceSource: k8s +serviceName: opa +servicePort: 8181 +namespace: higress-backend +policy: example1 +timeout: 5s +``` + +## OPA Service Installation Reference +### Start OPA Service +```shell +docker run -d --name opa -p 8181:8181 openpolicyagent/opa:0.35.0 run -s +``` + +### Create OPA Policy +```shell +curl -X PUT '127.0.0.1:8181/v1/policies/example1' \ + -H 'Content-Type: text/plain' \ + -d 'package example1 +import input.request +default allow = false +allow { + # HTTP method must GET + request.method == "GET" +}' +``` + +### Query Policy +```shell +curl -X POST '127.0.0.1:8181/v1/data/example1/allow' \ + -H 'Content-Type: application/json' \ + -d '{"input":{"request":{"method":"GET"}}}' +``` diff --git a/plugins/wasm-go/extensions/request-validation/README.md b/plugins/wasm-go/extensions/request-validation/README.md index 25761ca6f3..eef6c785ce 100644 --- a/plugins/wasm-go/extensions/request-validation/README.md +++ b/plugins/wasm-go/extensions/request-validation/README.md @@ -1,7 +1,18 @@ -# 功能说明 +--- +title: 请求协议校验 +keywords: [higress,request validation] +description: 请求协议校验插件配置参考 +--- + +## 功能说明 `request-validation`插件用于提前验证向上游服务转发的请求。该插件使用`JSON Schema`机制进行数据验证，可以验证请求的body及header数据。 -# 配置字段 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`220` + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- |-----| -------- | @@ -14,9 +25,9 @@ **校验规则对header和body是一样的，下面以body为例说明** -# 配置示例 +## 配置示例 -## 枚举（Enum）验证 +### 枚举（Enum）验证 ```yaml body_schema: type: object @@ -31,7 +42,7 @@ body_schema: default: "enum_string_1" ``` -## 布尔（Boolean）验证 +### 布尔（Boolean）验证 ```yaml body_schema: type: object @@ -43,7 +54,7 @@ body_schema: default: true ``` -## 数字范围（Number or Integer）验证 +### 数字范围（Number or Integer）验证 ```yaml body_schema: type: object @@ -56,7 +67,7 @@ body_schema: maximum: 10 ``` -## 字符串长度（String）验证 +### 字符串长度（String）验证 ```yaml body_schema: type: object @@ -69,7 +80,7 @@ body_schema: maxLength: 10 ``` -## 正则表达式（Regex）验证 +### 正则表达式（Regex）验证 ```yaml body_schema: type: object @@ -83,7 +94,7 @@ body_schema: pattern: "^[a-zA-Z0-9_]+$" ``` -## 数组（Array）验证 +### 数组（Array）验证 ```yaml body_schema: type: object @@ -101,7 +112,7 @@ body_schema: default: [1, 2, 3] ``` -## 多字段组合（Combined）验证 +### 多字段组合（Combined）验证 ```yaml body_schema: type: object @@ -128,7 +139,7 @@ body_schema: pattern: "^[a-zA-Z0-9_]+$" ``` -## 自定义拒绝信息 +### 自定义拒绝信息 ```yaml body_schema: type: object @@ -141,6 +152,3 @@ rejected_code: 403 rejected_msg: "请求被拒绝" ``` -# 本地调试 - -参考[使用 GO 语言开发 WASM 插件](https://higress.io/zh-cn/docs/user/wasm-go#%E4%B8%89%E6%9C%AC%E5%9C%B0%E8%B0%83%E8%AF%95) \ No newline at end of file diff --git a/plugins/wasm-go/extensions/request-validation/README_EN.md b/plugins/wasm-go/extensions/request-validation/README_EN.md new file mode 100644 index 0000000000..5d898f850c --- /dev/null +++ b/plugins/wasm-go/extensions/request-validation/README_EN.md @@ -0,0 +1,149 @@ +--- +title: Request Protocol Validation +keywords: [higress,request validation] +description: Configuration reference for request protocol validation plugin +--- +## Function Description +The `request-validation` plugin is used to validate requests forwarded to upstream services in advance. This plugin utilizes the `JSON Schema` mechanism for data validation, capable of validating both the body and header data of requests. + +## Execution Attributes +Plugin Execution Phase: `Authentication Phase` +Plugin Execution Priority: `220` + +## Configuration Fields +| Name | Data Type | Requirements | Default Value | Description | +|-----------------|-----------|--------------|---------------|---------------------------------------------------| +| header_schema | object | Optional | - | Configuration for JSON Schema to validate request headers | +| body_schema | object | Optional | - | Configuration for JSON Schema to validate request body | +| rejected_code | number | Optional | 403 | HTTP status code returned when the request is rejected | +| rejected_msg | string | Optional | - | HTTP response body returned when the request is rejected | +| enable_swagger | bool | Optional | false | Configuration to enable Swagger documentation validation | +| enable_oas3 | bool | Optional | false | Configuration to enable OAS3 documentation validation | + +**Validation rules for header and body are the same, below is an example using body.** + +## Configuration Examples +### Enumeration (Enum) Validation +```yaml +body_schema: + type: object + required: + - enum_payload + properties: + enum_payload: + type: string + enum: + - "enum_string_1" + - "enum_string_2" + default: "enum_string_1" +``` + +### Boolean Validation +```yaml +body_schema: + type: object + required: + - boolean_payload + properties: + boolean_payload: + type: boolean + default: true +``` + +### Number Range (Number or Integer) Validation +```yaml +body_schema: + type: object + required: + - integer_payload + properties: + integer_payload: + type: integer + minimum: 1 + maximum: 10 +``` + +### String Length Validation +```yaml +body_schema: + type: object + required: + - string_payload + properties: + string_payload: + type: string + minLength: 1 + maxLength: 10 +``` + +### Regular Expression (Regex) Validation +```yaml +body_schema: + type: object + required: + - regex_payload + properties: + regex_payload: + type: string + minLength: 1 + maxLength: 10 + pattern: "^[a-zA-Z0-9_]+$" +``` + +### Array Validation +```yaml +body_schema: + type: object + required: + - array_payload + properties: + array_payload: + type: array + minItems: 1 + items: + type: integer + minimum: 1 + maximum: 10 + uniqueItems: true + default: [1, 2, 3] +``` + +### Combined Validation +```yaml +body_schema: + type: object + required: + - boolean_payload + - array_payload + - regex_payload + properties: + boolean_payload: + type: boolean + array_payload: + type: array + minItems: 1 + items: + type: integer + minimum: 1 + maximum: 10 + uniqueItems: true + default: [1, 2, 3] + regex_payload: + type: string + minLength: 1 + maxLength: 10 + pattern: "^[a-zA-Z0-9_]+$" +``` + +### Custom Rejection Message +```yaml +body_schema: + type: object + required: + - boolean_payload + properties: + boolean_payload: + type: boolean +rejected_code: 403 +rejected_msg: "Request rejected" +``` diff --git a/plugins/wasm-go/extensions/traffic-tag/README.md b/plugins/wasm-go/extensions/traffic-tag/README.md index 143d665023..cf0edc56ad 100644 --- a/plugins/wasm-go/extensions/traffic-tag/README.md +++ b/plugins/wasm-go/extensions/traffic-tag/README.md @@ -1,10 +1,19 @@ -# traffic-tag插件说明文档 +--- +title: 流量染色 +keywords: [higress,traffic tag] +description: 流量染色插件配置参考 +--- -本文档提供了 `traffic-tag` 插件的配置选项的详细信息。此配置用于管理基于灰度发布和测试目的定义的特定规则的流量标记。 ## 功能说明 -`traffic-tag` 插件允许根据权重或特定请求内容通过添加特定请求头的方式对请求流量进行标记。它支持复杂的逻辑来确定如何根据用户定义的标准标记流量。 +`traffic-tag` 插件允许根据权重或特定请求内容通过添加特定请求头的方式对请求流量进行染色。它支持复杂的逻辑来确定如何根据用户定义的标准染色流量。 + +## 运行属性 + +插件执行阶段：`默认阶段` +插件执行优先级：`400` + ## 配置字段 diff --git a/plugins/wasm-go/extensions/traffic-tag/README_EN.md b/plugins/wasm-go/extensions/traffic-tag/README_EN.md index 6d266b775f..238cd63d5d 100644 --- a/plugins/wasm-go/extensions/traffic-tag/README_EN.md +++ b/plugins/wasm-go/extensions/traffic-tag/README_EN.md @@ -1,79 +1,76 @@ -# Traffic-Tag Plugin Documentation - -This document provides detailed information about the configuration options for the `traffic-tag` plugin. This configuration is used to manage traffic tags based on rules defined for canary releases and testing purposes. - -## Feature Description +--- +title: Traffic Tagging +keywords: [higress, traffic tag] +description: Traffic tagging plugin configuration reference +--- +## Function Description +The `traffic-tag` plugin allows for tagging request traffic by adding specific request headers based on weight or specific request content. It supports complex logic to determine how to tag traffic according to user-defined standards. -The `traffic-tag` plugin allows for tagging request traffic by adding specific request headers based on weight or specific request content. It supports complex logic to determine how to tag traffic based on user-defined criteria. +## Running Attributes +Plugin execution phase: `Default Phase` +Plugin execution priority: `400` ## Configuration Fields - This section provides a detailed description of the configuration fields. -| Field Name | Type | Default | Required | Description | -|-------------------|------------------|---------|----------|-------------------------------------------------------------------| -| `conditionGroups` | array of objects | - | No | Defines groups of content-based tagging conditions, see **Condition Group Configuration**. | -| `weightGroups` | array of objects | - | No | Defines groups of weight-based tagging conditions, see **Weight Group Configuration**. | -| `defaultTagKey` | string | - | No | The default tag key name used when no conditions are matched. It only takes effect when **defaultTagValue** is also configured. | -| `defaultTagValue` | string | - | No | The default tag value used when no conditions are matched. It only takes effect when **defaultTagKey** is also configured. | +| Field Name | Type | Default Value | Required | Description | +|------------------|-------------------|---------------|----------|-------------------------------------------------------------------| +| `conditionGroups` | array of object | - | No | Defines content-based tagging condition groups, detailed structure in **Condition Group Configuration**. | +| `weightGroups` | array of object | - | No | Defines weight-based tagging condition groups, detailed structure in **Weight Group Configuration**. | +| `defaultTagKey` | string | - | No | Default tagging key name used when no conditions are matched. Only effective when **defaultTagVal** is also configured. | +| `defaultTagVal` | string | - | No | Default tagging value used when no conditions are matched. Only effective when **defaultTagKey** is also configured. | ### Condition Group Configuration - -Each item in `conditionGroups` has the following configuration fields: - -| Field Name | Type | Default | Required | Description | -|--------------|--------|---------|----------|------------------------------------------------------------------| -| `headerName` | string | - | Yes | The name of the HTTP header to add or modify. | -| `headerValue`| string | - | Yes | The value of the HTTP header. | -| `logic` | string | - | Yes | The logical relation within the condition group, supports `and`, `or`, all in lowercase. | -| `conditions` | array of objects | - | Yes | Describes specific tagging conditions, detailed structure below. | +The configuration fields for each item in `conditionGroups` are described as follows: + +| Field Name | Type | Default Value | Required | Description | +|------------------|--------|---------------|----------|-------------------------------------------------------------------| +| `headerName` | string | - | Yes | The HTTP header name to be added or modified. | +| `headerValue` | string | - | Yes | The value of the HTTP header. | +| `logic` | string | - | Yes | Logical relationship in the condition group, supports `and`, `or`, must be in lowercase. | +| `conditions` | array of object | - | Yes | Describes specific tagging conditions, detailed structure below. | --- +The configuration fields for each item in `conditions` are described as follows: -The configuration fields for each item in `conditions` are as follows: - -| Field Name | Type | Default | Required | Description | -|----------------|--------|---------|----------|---------------------------------------------------------------------| -| `conditionType`| string | - | Yes | The type of condition, supports `header`, `parameter`, `cookie`. | -| `key` | string | - | Yes | The keyword for the condition. | -| `operator` | string | - | Yes | The operator, supports `equal`, `not_equal`, `prefix`, `in`, `not_in`, `regex`, `percentage`. | -| `value` | array of strings | - | Yes | The value for the condition, **only** supports multiple values for `in` and `not_in`. | +| Field Name | Type | Default Value | Required | Description | +|-------------------|-------------------|---------------|----------|-------------------------------------------------------------------| +| `conditionType` | string | - | Yes | Condition type, supports `header`, `parameter`, `cookie`. | +| `key` | string | - | Yes | The key of the condition. | +| `operator` | string | - | Yes | Operator, supports `equal`, `not_equal`, `prefix`, `in`, `not_in`, `regex`, `percentage`. | +| `value` | array of string | - | Yes | The value of the condition. **Only when** the operator is `in` and `not_in` multiple values are supported. | -> **Note: When `operator` is `regex`, the regex engine used is [RE2](https://github.com/google/re2). For more details, see the [RE2 official documentation](https://github.com/google/re2/wiki/Syntax).** +> **Note: When the `operator` is `regex`, the regular expression engine used is [RE2](https://github.com/google/re2). For details, please refer to the [RE2 Official Documentation](https://github.com/google/re2/wiki/Syntax).** ### Weight Group Configuration - -Each item in `weightGroups` has the following configuration fields: - -| Field Name | Type | Default | Required | Description | -|--------------|---------|---------|----------|--------------------------------------------------------| -| `headerName` | string | - | Yes | The name of the HTTP header to add or modify. | -| `headerValue`| string | - | Yes | The value of the HTTP header. | -| `weight` | integer | - | Yes | The percentage of traffic weight. | - -### Operator Descriptions - -| Operator | Description | -|--------------|---------------------------------------------------------| -| `equal` | Exact match, the value must be completely equal. | -| `not_equal` | Not equal match, condition is met when values are not equal. | -| `prefix` | Prefix match, condition is met when the specified value is a prefix of the actual value. | -| `in` | Inclusion match, actual value needs to be in the specified list. | -| `not_in` | Exclusion match, condition is met when the actual value is not in the specified list. | -| `regex` | Regex match, matches according to regex rules. | -| `percentage` | Percentage match, the principle: `hash(get(key)) % 100 < value` satisfies the condition. | - -> **Tip: Differences between `percentage` and `weight`** +The configuration fields for each item in `weightGroups` are described as follows: + +| Field Name | Type | Default Value | Required | Description | +|------------------|-------------------|---------------|----------|-------------------------------------------------------------------| +| `headerName` | string | - | Yes | The HTTP header name to be added or modified. | +| `headerValue` | string | - | Yes | The value of the HTTP header. | +| `weight` | integer | - | Yes | Traffic weight percentage. | + +### Operator Description +| Operator | Description | +|---------------|----------------------------------------------------| +| `equal` | Exact match, values must be identical. | +| `not_equal` | Not equal match, condition met when values are different. | +| `prefix` | Prefix match, condition met when the specified value is a prefix of the actual value. | +| `in` | Inclusion match, actual value must be in the specified list. | +| `not_in` | Exclusion match, condition met when actual value is not in the specified list. | +| `regex` | Regular expression match, matched according to regex rules. | +| `percentage` | Percentage match, principle: `hash(get(key)) % 100 < value`, condition met when true. | + +> **Tip: About the difference between `percentage` and `weight`** > -> - **`percentage` operator**: Used in condition expressions, based on a specified percentage and specified key-value pair to determine whether to perform a certain action. For the same key-value pair, the result of multiple matches is idempotent, i.e., if it hits the condition this time, it will also hit next time. -> - **`weight` field**: Used to define the traffic weight of different treatment paths. In weight-based traffic marking, `weight` determines the proportion of traffic that a path should receive. Unlike `percentage`, which is based on a fixed comparison basis, `weight` involves a random distribution of traffic weight, so the same request may match multiple outcomes in multiple matches. +> - **`percentage` operator**: Used in conditional expressions, it determines whether to perform a certain operation based on specified percentages and key-value pairs. For the same key-value pair, the result of multiple matches is idempotent, meaning if a condition is hit this time, it will hit it next time as well. +> - **`weight` field**: Used to define traffic weights for different processing paths. In weight-based traffic tagging, `weight` determines the proportion of traffic a particular path should receive. Unlike `percentage`, since there is no fixed comparison basis and it is based on random weight distribution, multiple matches for the same request may yield multiple results. > -> When using `percentage` for condition matching, each request is evaluated to see if it meets a specific percentage condition; whereas `weight` statically randomly allocates the proportion of overall traffic. +> When using `percentage` for conditional matching, it assesses whether each request meets specific percentage conditions; while `weight` is a static random allocation of overall traffic distribution. ## Configuration Example - -**Example 1: Content-Based Matching** - -According to the following configuration, requests that meet both the condition of the request header `role` being one of `user`, `viewer`, or `editor` and the query parameter `foo=bar` will have the request header `x-mse-tag: gray` added. Since `defaultTagKey` and `defaultTagVal` are configured, when no conditions are matched, the request header `x-mse-tag: base` will be added to the request. +**Example 1: Content-based Matching** +According to the configuration below, requests where the request header `role` has a value of `user`, `viewer`, or `editor` and contain query parameter `foo=bar` will have the request header `x-mse-tag: gray` added. Since `defaultTagKey` and `defaultTagVal` are configured, when no conditions are matched, the request will have the request header `x-mse-tag: base` added. ```yaml defaultTagKey: x-mse-tag @@ -94,15 +91,14 @@ conditionGroups: key: foo operator: equal value: - - bar + - bar ``` -**Example 2: Weight-Based Matching** - -According to the following configuration, requests have a 30% chance of having the request header `x-mse-tag: gray` added, a 30% chance of having `x-mse-tag: blue` added, and a 40% chance of not having any header added. +**Example 2: Weight-based Matching** +According to the configuration below, there is a 30% chance that the request will have the request header `x-mse-tag: gray` added, a 30% chance it will have `x-mse-tag: blue` added, and a 40% chance that no header will be added. ```yaml -# The total weight sums to 100, the unconfigured 40 weight will not add a header +# The total weight is 100; the 40 weight not configured in the example will not add a header. weightGroups: - headerName: x-mse-tag headerValue: gray diff --git a/plugins/wasm-go/extensions/transformer/README.md b/plugins/wasm-go/extensions/transformer/README.md index 5f1d709e5e..29ae3f962a 100644 --- a/plugins/wasm-go/extensions/transformer/README.md +++ b/plugins/wasm-go/extensions/transformer/README.md @@ -1,8 +1,19 @@ -# 功能说明 +--- +title: 请求响应转换 +keywords: [higress,transformer] +description: 请求响应转换插件配置参考 +--- + + +## 功能说明 `transformer` 插件可以对请求/响应头、请求查询参数、请求/响应体参数进行转换，支持的转换操作类型包括删除、重命名、更新、添加、追加、映射、去重。 +## 运行属性 + +插件执行阶段：`认证阶段` +插件执行优先级：`410` -# 配置字段 +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | :----: | :----: | :----: | :----: | -------- | @@ -52,7 +63,7 @@ -# 转换操作类型 +## 转换操作类型 | 操作类型 | key 字段含义 | value 字段含义 | 描述 | | :----: | :----: | :----: | ------------------------------------------------------------ | @@ -67,11 +78,97 @@ -# 配置示例 +## 配置示例 + +### 实现基于Body参数路由 + +配置示例： + +```yaml +reqRules: +- operate: map + headers: + - fromKey: userId + toKey: x-user-id + mapSource: body +``` + +此规则将请求body中的`userId`解析出后，设置到请求Header`x-user-id`中，这样就可以基于Higress请求Header匹配路由的能力来实现基于Body参数的路由了。 + +此配置同时支持`application/json`和`application/x-www-form-urlencoded`两种类型的请求Body。 + +举例来说： + +**对于application/json类型的body** + +```bash +curl localhost -d '{"userId":12, "userName":"johnlanni"}' -H 'content-type:application/json' +``` + +将从json中提取出`userId`字段的值，设置到`x-user-id`中，后端服务收到的请求头将增加:`x-usr-id: 12`。 + +因为在插件新增这个Header后，网关将重新计算路由，所以可以实现网关路由配置根据这个请求头来匹配路由到特定的目标服务。 + + +**对于application/x-www-form-urlencoded类型的body** + +```bash +curl localhost -d 'userId=12&userName=johnlanni' +``` + +将从`k1=v1&k2=v2`这样的表单格式中提取出`userId`字段的值，设置到`x-user-id`中，后端服务收到的请求头将增加:`x-usr-id: 12`。 + +因为在插件新增这个Header后，网关将重新计算路由，所以可以实现网关路由配置根据这个请求头来匹配路由到特定的目标服务。 + +#### json path 支持 + +可以根据 [GJSON Path 语法](https://github.com/tidwall/gjson/blob/master/SYNTAX.md)，从复杂的 json 中提取出字段。 + +比较常用的操作举例，对于以下 json: + +```json +{ + "name": {"first": "Tom", "last": "Anderson"}, + "age":37, + "children": ["Sara","Alex","Jack"], + "fav.movie": "Deer Hunter", + "friends": [ + {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]}, + {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]}, + {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]} + ] +} +``` + +可以实现这样的提取: + +```text +name.last "Anderson" +name.first "Tom" +age 37 +children ["Sara","Alex","Jack"] +children.0 "Sara" +children.1 "Alex" +friends.1 {"first": "Roger", "last": "Craig", "age": 68} +friends.1.first "Roger" +``` + +现在如果想从上面这个 json 格式的 body 中提取出 friends 中第二项的 first 字段，来设置到 Header `x-first-name` 中，同时抽取 last 字段，来设置到 Header `x-last-name` 中，则可以使用这份插件配置: + +```yaml +reqRules: +- operate: map + headers: + - fromKey: friends.1.first + toKey: x-first-name + - fromKey: friends.1.last + toKey: x-last-name + mapSource: body +``` -## Request Transformer +### Request Transformer -### 转换请求头部 +#### 转换请求头部 ```yaml reqRules: @@ -139,7 +236,7 @@ $ curl -v console.higress.io/get -H 'host: foo.bar.com' \ } ``` -### 转换请求查询参数 +#### 转换请求查询参数 ```yaml reqRules: @@ -193,7 +290,7 @@ $ curl -v "console.higress.io/get?k1=v11&k1=v12&k2=v2" } ``` -### 转换请求体 +#### 转换请求体 ```yaml reqRules: @@ -313,11 +410,11 @@ $ curl -v -X POST console.higress.io/post -H 'host: foo.bar.com' \ } ``` -## Response Transformer +### Response Transformer 与 Request Transformer 类似，在此仅说明转换 JSON 形式的请求/响应体时的注意事项： -### key 嵌套 `.` +#### key 嵌套 `.` 1.通常情况下，指定的 key 中含有 `.` 表示嵌套含义，如下： @@ -365,7 +462,7 @@ $ curl -v console.higress.io/get } ``` -### 访问数组元素 `.index` +#### 访问数组元素 `.index` 可以通过数组下标 `array.index 访问数组元素，下标从 0 开始： @@ -449,7 +546,7 @@ $ curl -v -X POST console.higress.io/post \ } ``` -### 遍历数组元素 `.#` +#### 遍历数组元素 `.#` 可以使用 `array.#` 对数组进行遍历操作： @@ -502,95 +599,3 @@ $ curl -v -X POST console.higress.io/post \ ... } ``` -# 特殊用法：实现基于Body参数路由 - -**Note** - -> 需要数据面的proxy wasm版本大于等于0.2.100 - -> 编译时，需要带上版本的tag，例如：`tinygo build -o main.wasm -scheduler=none -target=wasi -gc=custom -tags="custommalloc nottinygc_finalizer proxy_wasm_version_0_2_100" ./` - - -配置示例： - -```yaml -reqRules: -- operate: map - headers: - - fromKey: userId - toKey: x-user-id - mapSource: body -``` - -此规则将请求body中的`userId`解析出后，设置到请求Header`x-user-id`中，这样就可以基于Higress请求Header匹配路由的能力来实现基于Body参数的路由了。 - -此配置同时支持`application/json`和`application/x-www-form-urlencoded`两种类型的请求Body。 - -举例来说： - -**对于application/json类型的body** - -```bash -curl localhost -d '{"userId":12, "userName":"johnlanni"}' -H 'content-type:application/json' -``` - -将从json中提取出`userId`字段的值，设置到`x-user-id`中，后端服务收到的请求头将增加:`x-usr-id: 12`。 - -因为在插件新增这个Header后，网关将重新计算路由，所以可以实现网关路由配置根据这个请求头来匹配路由到特定的目标服务。 - - -**对于application/x-www-form-urlencoded类型的body** - -```bash -curl localhost -d 'userId=12&userName=johnlanni' -``` - -将从`k1=v1&k2=v2`这样的表单格式中提取出`userId`字段的值，设置到`x-user-id`中，后端服务收到的请求头将增加:`x-usr-id: 12`。 - -因为在插件新增这个Header后，网关将重新计算路由，所以可以实现网关路由配置根据这个请求头来匹配路由到特定的目标服务。 - -## json path 支持 - -可以根据 [GJSON Path 语法](https://github.com/tidwall/gjson/blob/master/SYNTAX.md)，从复杂的 json 中提取出字段。 - -比较常用的操作举例，对于以下 json: - -```json -{ - "name": {"first": "Tom", "last": "Anderson"}, - "age":37, - "children": ["Sara","Alex","Jack"], - "fav.movie": "Deer Hunter", - "friends": [ - {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]}, - {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]}, - {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]} - ] -} -``` - -可以实现这样的提取: - -```text -name.last "Anderson" -name.first "Tom" -age 37 -children ["Sara","Alex","Jack"] -children.0 "Sara" -children.1 "Alex" -friends.1 {"first": "Roger", "last": "Craig", "age": 68} -friends.1.first "Roger" -``` - -现在如果想从上面这个 json 格式的 body 中提取出 friends 中第二项的 first 字段，来设置到 Header `x-first-name` 中，同时抽取 last 字段，来设置到 Header `x-last-name` 中，则可以使用这份插件配置: - -```yaml -reqRules: -- operate: map - headers: - - fromKey: friends.1.first - toKey: x-first-name - - fromKey: friends.1.last - toKey: x-last-name - mapSource: body -``` diff --git a/plugins/wasm-go/extensions/transformer/README_EN.md b/plugins/wasm-go/extensions/transformer/README_EN.md new file mode 100644 index 0000000000..86dd98c87d --- /dev/null +++ b/plugins/wasm-go/extensions/transformer/README_EN.md @@ -0,0 +1,522 @@ +--- +title: Request Response Transformation +keywords: [higress,transformer] +description: Request response transformation plugin configuration reference +--- +## Function Description +The `transformer` plugin can transform request/response headers, request query parameters, and request/response body parameters. Supported transformation operation types include deletion, renaming, updating, adding, appending, mapping, and deduplication. + +## Execution Attributes +Plugin execution phase: `authentication phase` +Plugin execution priority: `410` + +## Configuration Fields +| Name | Data Type | Fill Requirement | Default Value | Description | +| :----: | :----: | :----: | :----: | -------- | +| reqRules | string | Optional, at least one of reqRules or respRules must be filled | - | Request transformer configuration, specifying the transformation operation type and rules for transforming request headers, request query parameters, and request body | +| respRules | string | Optional, at least one of reqRules or respRules must be filled | - | Response transformer configuration, specifying the transformation operation type and rules for transforming response headers and response body | + +The configuration fields for each item in `reqRules` and `respRules` are as follows: + +| Name | Data Type | Fill Requirement | Default Value | Description | +| :----: | :----: | :----: | :----: | -------- | +| operate | string | Required, optional values are `remove`, `rename`, `replace`, `add`, `append`, `map`, `dedupe` | - | Specifies the transformation operation type. Supported operation types include remove (remove), rename (rename), replace (replace), add (add), append (append), map (map), dedupe (dedupe). When there are multiple transformation rules of different types, they are executed in the order of the above operation types. | +| mapSource | string | Optional, optional values are `headers`, `querys`, `body` | - | Valid only when operate is `map`. Specifies the mapping source. If this field is not filled, the default mapping source is itself. | +| headers | array of object | Optional | - | Specifies transformation rules for request/response headers. | +| querys | array of object | Optional | - | Specifies transformation rules for request query parameters. | +| body | array of object | Optional | - | Specifies transformation rules for request/response body parameters. Request body transformations allow content-types of `application/json`, `application/x-www-form-urlencoded`, and `multipart/form-data` while response body transformations only allow content-type of `application/json`. | + +The configuration fields for each item in `headers`, `querys`, `body` are as follows: + +| Name | Data Type | Fill Requirement | Default Value | Description | +| :----: | :----: | :----: | -------- | --------------------------------------------------- | +| key | string | Optional | - | Used when operate is `remove`, see [Transformation Operation Types](#转换操作类型) for details. | +| oldKey | string | Optional | - | Used when operate is `rename`, see [Transformation Operation Types](#转换操作类型) for details. | +| newKey | string | Optional | - | Used when operate is `rename`, see [Transformation Operation Types](#转换操作类型) for details. | +| key | string | Optional | - | Used when operate is `replace`, see [Transformation Operation Types](#转换操作类型) for details. | +| newValue | string | Optional | - | Used when operate is `replace`, see [Transformation Operation Types](#转换操作类型) for details. | +| key | string | Optional | - | Used when operate is `add`, see [Transformation Operation Types](#转换操作类型) for details. | +| value | string | Optional | - | Used when operate is `add`, see [Transformation Operation Types](#转换操作类型) for details. | +| key | string | Optional | - | Used when operate is `append`, see [Transformation Operation Types](#转换操作类型) for details. | +| appendValue | string | Optional | - | Used when operate is `append`, see [Transformation Operation Types](#转换操作类型) for details. | +| fromKey | string | Optional | - | Used when operate is `map`, see [Transformation Operation Types](#转换操作类型) for details. | +| toKey | string | Optional | - | Used when operate is `map`, see [Transformation Operation Types](#转换操作类型) for details. | +| key | string | Optional | - | Used when operate is `dedupe`, see [Transformation Operation Types](#转换操作类型) for details. | +| strategy | string | Optional | - | Used when operate is `dedupe`, see [Transformation Operation Types](#转换操作类型) for details. | +| value_type | string | Optional, optional values are `object`, `boolean`, `number`, `string` | string | When `content-type: application/json`, this field specifies the value type of request/response body parameters. | +| host_pattern | string | Optional | - | Specifies the request hostname matching rule. Valid when transformation operation type is `replace`, `add`, `append`. | +| path_pattern | string | Optional | - | Specifies the request path matching rule. Valid when transformation operation type is `replace`, `add`, `append`. | + +Note: +* `request transformer` supports the following transformation objects: request headers, request query parameters, request body (application/json, application/x-www-form-urlencoded, multipart/form-data). +* `response transformer` supports the following transformation objects: response headers, response body (application/json). +* The plugin supports bidirectional conversion capability, meaning that a single plugin can perform transformations on both requests and responses. +* The execution order of transformation operation types is the order written in the configuration file, e.g., remove → rename → replace → add → append → map → dedupe or dedupe → map → append → add → replace → rename → remove. +* When the transformation object is headers, `key` is case-insensitive. When headers are operated and are `rename` or `map`, `value` is also case-insensitive (as this field has a key meaning). However, `key` and `value` fields in querys and body are case-sensitive. +* `value_type` is only effective for content type application/json for request/response bodies. +* `host_pattern` and `path_pattern` support [RE2 syntax](https://pkg.go.dev/regexp/syntax), valid only for `replace`, `add`, `append` operations. In a transformation rule, only one of the two can be optionally filled. If both are filled, then `host_pattern` takes effect while `path_pattern` becomes ineffective. + +## Transformation Operation Types +| Operation Type | Key Field Meaning | Value Field Meaning | Description | +| :----: | :----: | :----: | ------------------------------------------------------------ | +| Remove remove | Target key | Not required | If the specified `key` exists, delete it; otherwise, no operation | +| Rename rename | Target oldKey | New key name newKey | If the specified `oldKey:value` exists, rename its key to `newKey`, resulting in `newKey:value`; otherwise, no operation | +| Replace replace | Target key | New value newValue | If the specified `key:value` exists, update its value to `newValue`, resulting in `key:newValue`; otherwise, no operation | +| Add add | Added key | Added value | If the specified `key:value` does not exist, add it; otherwise, no operation | +| Append append | Target key | Appending value appendValue | If the specified `key:value` exists, append appendValue to get `key:[value..., appendValue]`; otherwise, it is equivalent to performing add operation, resulting in `key:appendValue`. | +| Map map | Mapping source fromKey | Mapping target toKey | If the specified `fromKey:fromValue` exists, map its value fromValue to the value of toKey, resulting in `toKey:fromValue`, while retaining `fromKey:fromValue` (note: if toKey already exists, its value will be overwritten); otherwise, no operation. | +| Deduplicate dedupe | Target key | Specified deduplication strategy strategy | `strategy` optional values include:
`RETAIN_UNIQUE`: Retain all unique values in order, e.g., `k1:[v1,v2,v3,v3,v2,v1]`, deduplication results in `k1:[v1,v2,v3]`.
`RETAIN_LAST`: Retain the last value, e.g., `k1:[v1,v2,v3]`, deduplication results in `k1:v3`.
`RETAIN_FIRST` (default): Retain the first value, e.g., `k1:[v1,v2,v3]`, deduplication results in `k1:v1`.
(Note: When deduplication results in only one element v1, the key-value pair becomes `k1:v1`, not `k1:[v1]`.) | + +## Configuration Example + +### Implement Routing Based on Body Parameters +Configuration example: +```yaml +reqRules: +- operate: map + headers: + - fromKey: userId + toKey: x-user-id + mapSource: body +``` +This rule extracts the `userId` from the request body and sets it in the request header `x-user-id`. This allows routing based on body parameters using Higress's ability to match on request headers. + +This configuration supports both `application/json` and `application/x-www-form-urlencoded` types of request bodies. + +For example: +**For application/json type body** +```bash +curl localhost -d '{"userId":12, "userName":"johnlanni"}' -H 'content-type:application/json' +``` +The value of the `userId` field will be extracted from the JSON and set to `x-user-id`. The backend service will receive a request header with: `x-user-id: 12`. + +After the plugin adds this header, the gateway will recalculate the routes, allowing the routing configuration to match the specific target service based on this request header. + +**For application/x-www-form-urlencoded type body** +```bash +curl localhost -d 'userId=12&userName=johnlanni' +``` +The value of the `userId` field will be extracted from the form format `k1=v1&k2=v2` and set to `x-user-id`. The backend service will receive a request header with: `x-user-id: 12`. + +After the plugin adds this header, the gateway will recalculate the routes, allowing the routing configuration to match the specific target service based on this request header. + +#### JSON Path Support +You can extract fields from complex JSON according to [GJSON Path syntax](https://github.com/tidwall/gjson/blob/master/SYNTAX.md). + +Common operations include, for the following JSON: +```json +{ + "name": {"first": "Tom", "last": "Anderson"}, + "age": 37, + "children": ["Sara","Alex","Jack"], + "fav.movie": "Deer Hunter", + "friends": [ + {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]}, + {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]}, + {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]} + ] +} +``` +You can achieve such extractions: +```text +name.last "Anderson" +name.first "Tom" +age 37 +children ["Sara","Alex","Jack"] +children.0 "Sara" +children.1 "Alex" +friends.1 {"first": "Roger", "last": "Craig", "age": 68} +friends.1.first "Roger" +``` +Now, if you want to extract the `first` field from the second item in `friends` from the above JSON formatted body and set it to the header `x-first-name`, while also extracting the `last` field to set it to the header `x-last-name`, you can use this plugin configuration: +```yaml +reqRules: +- operate: map + headers: + - fromKey: friends.1.first + toKey: x-first-name + - fromKey: friends.1.last + toKey: x-last-name + mapSource: body +``` + +### Request Transformer +#### Transforming Request Headers +```yaml +reqRules: +- operate: remove + headers: + - key: X-remove +- operate: rename + headers: + - oldKey: X-not-renamed + newKey: X-renamed +- operate: replace + headers: + - key: X-replace + newValue: replaced +- operate: add + headers: + - key: X-add-append + value: host-\$1 + host_pattern: ^(.*)\.com$ +- operate: append + headers: + - key: X-add-append + appendValue: path-\$1 + path_pattern: ^.*?\/(\w+)[\?]{0,1}.*$ +- operate: map + headers: + - fromKey: X-add-append + toKey: X-map +- operate: dedupe + headers: + - key: X-dedupe-first + strategy: RETAIN_FIRST + - key: X-dedupe-last + strategy: RETAIN_LAST + - key: X-dedupe-unique + strategy: RETAIN_UNIQUE +``` +Send Request +```bash +$ curl -v console.higress.io/get -H 'host: foo.bar.com' \ +-H 'X-remove: exist' -H 'X-not-renamed:test' -H 'X-replace:not-replaced' \ +-H 'X-dedupe-first:1' -H 'X-dedupe-first:2' -H 'X-dedupe-first:3' \ +-H 'X-dedupe-last:a' -H 'X-dedupe-last:b' -H 'X-dedupe-last:c' \ +-H 'X-dedupe-unique:1' -H 'X-dedupe-unique:2' -H 'X-dedupe-unique:3' \ +-H 'X-dedupe-unique:3' -H 'X-dedupe-unique:2' -H 'X-dedupe-unique:1' +# httpbin response result +{ + "args": {}, + "headers": { + ... + "X-Add-Append": "host-foo.bar,path-get", + ... + "X-Dedupe-First": "1", + "X-Dedupe-Last": "c", + "X-Dedupe-Unique": "1,2,3", + ... + "X-Map": "host-foo.bar,path-get", + "X-Renamed": "test", + "X-Replace": "replaced" + }, + ... +} +``` +#### Transforming Request Query Parameters +```yaml +reqRules: +- operate: remove + querys: + - key: k1 +- operate: rename + querys: + - oldKey: k2 + newKey: k2-new +- operate: replace + querys: + - key: k2-new + newValue: v2-new +- operate: add + querys: + - key: k3 + value: v31-\$1 + path_pattern: ^.*?\/(\w+)[\?]{0,1}.*$ +- operate: append + querys: + - key: k3 + appendValue: v32 +- operate: map + querys: + - fromKey: k3 + toKey: k4 +- operate: dedupe + querys: + - key: k4 + strategy: RETAIN_FIRST +``` +Send Request +```bash +$ curl -v "console.higress.io/get?k1=v11&k1=v12&k2=v2" +# httpbin response result +{ + "args": { + "k2-new": "v2-new", + "k3": [ + "v31-get", + "v32" + ], + "k4": "v31-get" + }, + ... + "url": "http://foo.bar.com/get?k2-new=v2-new&k3=v31-get&k3=v32&k4=v31-get" +} +``` +#### Transforming Request Body +```yaml +reqRules: +- operate: remove + body: + - key: a1 +- operate: rename + body: + - oldKey: a2 + newKey: a2-new +- operate: replace + body: + - key: a3 + newValue: t3-new + value_type: string +- operate: add + body: + - key: a1-new + value: t1-new + value_type: string +- operate: append + body: + - key: a1-new + appendValue: t1-\$1-append + value_type: string + host_pattern: ^(.*)\.com$ +- operate: map + body: + - fromKey: a1-new + toKey: a4 +- operate: dedupe + body: + - key: a4 + strategy: RETAIN_FIRST +``` +Send Requests: +**1. Content-Type: application/json** +```bash +$ curl -v -X POST console.higress.io/post -H 'host: foo.bar.com' \ +-H 'Content-Type: application/json' -d '{"a1":"t1","a2":"t2","a3":"t3"}' +# httpbin response result +{ + ... + "headers": { + ... + "Content-Type": "application/json", + ... + }, + "json": { + "a1-new": [ + "t1-new", + "t1-foo.bar-append" + ], + "a2-new": "t2", + "a3": "t3-new", + "a4": "t1-new" + }, + ... +} +``` +**2. Content-Type: application/x-www-form-urlencoded** +```bash +$ curl -v -X POST console.higress.io/post -H 'host: foo.bar.com' \ +-d 'a1=t1&a2=t2&a3=t3' +# httpbin response result +{ + ... + "form": { + "a1-new": [ + "t1-new", + "t1-foo.bar-append" + ], + "a2-new": "t2", + "a3": "t3-new", + "a4": "t1-new" + }, + "headers": { + ... + "Content-Type": "application/x-www-form-urlencoded", + ... + }, + ... +} +``` +**3. Content-Type: multipart/form-data** +```bash +$ curl -v -X POST console.higress.io/post -H 'host: foo.bar.com' \ +-F a1=t1 -F a2=t2 -F a3=t3 +# httpbin response result +{ + ... + "form": { + "a1-new": [ + "t1-new", + "t1-foo.bar-append" + ], + "a2-new": "t2", + "a3": "t3-new", + "a4": "t1-new" + }, + "headers": { + ... + "Content-Type": "multipart/form-data; boundary=------------------------1118b3fab5afbc4e", + ... + }, + ... +} +``` +### Response Transformer +Similar to Request Transformer, this only describes the precautions for transforming JSON-formatted request/response bodies: + +#### Key Nesting `.` +1. In general, a key containing `.` indicates a nested meaning, as follows: +```yaml +respRules: +- operate: add + body: + - key: foo.bar + value: value +``` +```bash +$ curl -v console.higress.io/get +# httpbin response result +{ + ... + "foo": { + "bar": "value" + }, + ... +} +``` +2. When using `\.` to escape `.` in the key, it indicates a non-nested meaning, as follows: +> When enclosing a string with double quotes, use `\\.` for escaping +```yaml +respRules: +- operate: add + body: + - key: foo\.bar + value: value +``` +```bash +$ curl -v console.higress.io/get +# httpbin response result +{ + ... + "foo.bar": "value", + ... +} +``` +#### Accessing Array Elements `.index` +You can access array elements by their index `array.index`, where the index starts from 0: +```json +{ + "users": [ + { + "123": { "name": "zhangsan", "age": 18 } + }, + { + "456": { "name": "lisi", "age": 19 } + } + ] +} +``` +1. Remove the first element of `user`: +```yaml +reqRules: +- operate: remove + body: + - key: users.0 +``` +```bash +$ curl -v -X POST console.higress.io/post \ +-H 'Content-Type: application/json' \ +-d '{"users":[{"123":{"name":"zhangsan"}},{"456":{"name":"lisi"}}]}' +# httpbin response result +{ + ... + "json": { + "users": [ + { + "456": { + "name": "lisi" + } + } + ] + }, + ... +} +``` +2. Rename the key `123` of the first element of `users` to `msg`: +```yaml +reqRules: +- operate: rename + body: + - oldKey: users.0.123 + newKey: users.0.first +``` +```bash +$ curl -v -X POST console.higress.io/post \ +-H 'Content-Type: application/json' \ +-d '{"users":[{"123":{"name":"zhangsan"}},{"456":{"name":"lisi"}}]}' +# httpbin response result +{ + ... + "json": { + "users": [ + { + "msg": { + "name": "zhangsan" + } + }, + { + "456": { + "name": "lisi" + } + } + ] + }, + ... +} +``` +#### Iterating Array Elements `.#` +You can use `array.#` to iterate over an array: +> ❗️This operation can only be used in replace, do not attempt this operation in other transformations to avoid unpredictable results +```json +{ + "users": [ + { + "name": "zhangsan", + "age": 18 + }, + { + "name": "lisi", + "age": 19 + } + ] +} +``` +```yaml +reqRules: +- operate: replace + body: + - key: users.#.age + newValue: 20 +``` +```bash +$ curl -v -X POST console.higress.io/post \ +-H 'Content-Type: application/json' \ +-d '{"users":[{"name":"zhangsan","age":18},{"name":"lisi","age":19}]}' +# httpbin response result +{ + ... + "json": { + "users": [ + { + "age": "20", + "name": "zhangsan" + }, + { + "age": "20", + "name": "lisi" + } + ] + }, + ... +} +``` diff --git a/plugins/wasm-go/extensions/waf/README.md b/plugins/wasm-go/extensions/waf/README.md index 109b2a9535..92b2cf2b53 100644 --- a/plugins/wasm-go/extensions/waf/README.md +++ b/plugins/wasm-go/extensions/waf/README.md @@ -1,13 +1,25 @@ -# 功能说明 +--- +title: WAF +keywords: [higress,waf] +description: WAF 插件配置参考 +--- + +## 功能说明 waf插件实现了基于ModSecurity的规则防护引擎，可以根据用户配置的规则屏蔽可疑请求，并支持OWASP CRS，为站点提供基础的防护功能。 -# 配置字段 +## 运行属性 + +插件执行阶段：`授权阶段` +插件执行优先级：`330` + + +## 配置字段 | 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | | -------- | -------- | -------- | -------- | -------- | | useCRS | bool | 选填 | false | 是否开启OWASP CRS，详情可参考[coreruleset](https://github.com/coreruleset/coreruleset/tree/v3.3.2) | | secRules | array of string | 选填 | - | 用户自定义的waf防护规则，语法规则可参考[ModSecurity中文手册](http://www.modsecurity.cn/chm/) | -# 配置示例 +## 配置示例 ```yaml useCRS: true secRules: @@ -23,30 +35,3 @@ secRules: curl http://example.com/admin curl http://example.com -d "maliciouspayload" ``` - -# 对特定路由或域名开启 -```yaml -useCRS: true -secRules: - - "SecDebugLogLevel 3" - - "SecRuleEngine On" - - "SecAction \"id:100,phase:1,pass\"" - - "SecRule REQUEST_URI \"@streq /admin\" \"id:101,phase:1,t:lowercase,deny\"" - - "SecRule REQUEST_BODY \"@rx maliciouspayload\" \"id:102,phase:2,t:lowercase,deny\"" -_rules_: -- _match_route_: - - "route-1" - secRules: - - "SecDebugLogLevel 3" - - "SecRuleEngine On" - - "SecAction \"id:102,phase:1,deny\"" -- _match_domain_: - - "*.example.com" - - test.com - secRules: - - "SecDebugLogLevel 3" - - "SecRuleEngine On" - - "SecAction \"id:102,phase:1,pass\"" -``` - -此例 `_match_route_` 中指定的 `route-1` 即在创建网关路由时填写的路由名称，当匹配到这两个路由时，将使用此段配置； 此例 `_match_domain_` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名，当发现域名匹配时，将使用此段配置； 配置的匹配生效顺序，将按照 `_rules_` 下规则的排列顺序，匹配第一个规则后生效对应配置，后续规则将被忽略。 diff --git a/plugins/wasm-go/extensions/waf/README_EN.md b/plugins/wasm-go/extensions/waf/README_EN.md new file mode 100644 index 0000000000..d34603a7b6 --- /dev/null +++ b/plugins/wasm-go/extensions/waf/README_EN.md @@ -0,0 +1,39 @@ +--- +title: WAF +keywords: [higress,waf] +description: WAF plugin configuration reference +--- +## Function Description + +The waf plugin implements a ModSecurity-based rule protection engine, which can block suspicious requests based on user-defined rules, and supports OWASP CRS, providing basic protection features for the site. + +## Running Attributes + +Plugin execution phase: `authorization phase` +Plugin execution priority: `330` + +## Configuration Fields + +| Name | Data Type | Filling Requirements | Default Value | Description | +|----------|--------------------|----------------------|---------------|-----------------------------------------------------------------------------| +| useCRS | bool | Optional | false | Whether to enable OWASP CRS, for details refer to [coreruleset](https://github.com/coreruleset/coreruleset/tree/v3.3.2) | +| secRules | array of string | Optional | - | User-defined WAF protection rules, syntax rules can refer to [ModSecurity Chinese Manual](http://www.modsecurity.cn/chm/) | + +## Configuration Example + +```yaml +useCRS: true +secRules: + - "SecDebugLogLevel 3" + - "SecRuleEngine On" + - "SecAction \"id:100,phase:1,pass\"" + - "SecRule REQUEST_URI \"@streq /admin\" \"id:101,phase:1,t:lowercase,deny\"" + - "SecRule REQUEST_BODY \"@rx maliciouspayload\" \"id:102,phase:2,t:lowercase,deny\"" +``` + +Based on this configuration, the following requests will be prohibited from access: + +```bash +curl http://example.com/admin +curl http://example.com -d "maliciouspayload" +```