[ai-json-resp] Extract JSON from LLM, Validate with Schema, Ensure Valid JSON, Auto-Retry #1236
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
johnlanni
requested changes
Aug 21, 2024
johnlanni
reviewed
Aug 21, 2024
johnlanni
requested changes
Aug 21, 2024
johnlanni
reviewed
Aug 21, 2024
johnlanni
reviewed
Aug 21, 2024
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1236 +/- ##
==========================================
+ Coverage 35.91% 44.31% +8.40%
==========================================
Files 69 75 +6
Lines 11576 9823 -1753
==========================================
+ Hits 4157 4353 +196
+ Misses 7104 5142 -1962
- Partials 315 328 +13 |
CH3CHO
reviewed
Aug 23, 2024
CH3CHO
reviewed
Aug 26, 2024
CH3CHO
reviewed
Aug 27, 2024
|
|
CH3CHO
reviewed
Aug 27, 2024
| ReqBody: body, | ||
| } | ||
|
|
||
| config.serviceClient.Post(bufferRB.Path, bufferRB.ReqHeader, bufferRB.ReqBody, |
There was a problem hiding this comment.
这里会不会失败，失败会怎么样？要不要continue？
There was a problem hiding this comment.
这里有点没明白，您是指Post失败吗？那还会进入回调函数吗？如果会进入那应该已经写了处理？如果不会进入，那好像没法子处理r？
johnlanni
requested changes
Aug 31, 2024
| | enableSwagger | bool | optional | false | 是否启用 Swagger 协议进行验证 | | ||
| | enableOas3 | bool | optional | true | 是否启用 Oas3 协议进行验证 | | ||
| | jsonSchemaMaxDepth | int | optional | 5 | 由于插件性能限制，为防止递归耗尽资源，需指定支持的 JSON Schema 最大深度，超过该深度的 Schema 不会用于验证响应| | ||
| | rejectOnDepthExceeded | bool | optional | false | 若为 true，当 JSON Schema 的深度超过 maxJsonSchemaDepth 时，插件将直接返回错误；若为 false，则将仍将 Json Schema 用于LLM提示构造并继续执行 | |
There was a problem hiding this comment.
jsonSchemaMaxDepth 和 rejectOnDepthExceeded 这两个配置我建议去掉，最大深度在代码里有个默认值就行，文档里提示一下即可，默认行为就是超过最大深度，不用进行校验，但会生成提示词
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ⅰ. Describe what this PR did
[ai-json-resp] LLM响应结构化插件，用于根据默认或用户配置的Json Schema对AI的响应进行结构化。和ai proxy插件协作，除自定义的json schema和网关服务外无需额外配置参数。
Ⅱ. Does this pull request fix one issue?
fixes #1214