Add documentation for parameter substitution when provisioning workflows #6547

dbwiddis · 2024-02-29T19:58:30Z

Description

opensearch-project/flow-framework#525 added the ability to substitute expressions with API parameters during workflow provisioning.

This PR updates the documentation for this new feature.

This will be part of the 2.13 release.

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

hdhalter · 2024-03-01T01:12:59Z

Thanks, @dbwiddis! Is this ready for doc review?

dbwiddis · 2024-03-01T05:15:42Z

Yes, but there will be other 2.13 features. Would you prefer to wait and have all at once?

dbwiddis · 2024-03-04T20:24:30Z

Flipping this to draft as we may be implementing a different feature which will slightly change this.

amitgalitz · 2024-03-20T20:40:50Z

Hey @hdhalter since this feature has expanded a little more I will have a good amount more added to this PR for the issue we have here: opensearch-project/flow-framework#496.

We talked offline about this but I might only have more of the edits later in tonight (11:00PM PST) as I am merging a bug fix for the RC at 4pm. Is it okay to be adding extra documentation to here by then? Should I make a new PR?

hdhalter · 2024-03-20T20:44:41Z

Hey @hdhalter since this feature has expanded a little more I will have a good amount more added to this PR for the issue we have here: opensearch-project/flow-framework#496.

We talked offline about this but I might only have more of the edits later in tonight (11:00PM PST) as I am merging a bug fix for the RC at 4pm. Is it okay to be adding extra documentation to here by then? Should I make a new PR?

No problem, @amitgalitz. Feel free to add to this PR, and we'll review it when it's ready. Thanks!

Signed-off-by: Daniel Widdis <[email protected]>

dbwiddis · 2024-03-22T17:27:30Z

@hdhalter I understand @amitgalitz will be submitting a separate PR, so this one should be good to review/merge as-is.

hdhalter · 2024-03-22T17:33:54Z

@hdhalter I understand @amitgalitz will be submitting a separate PR, so this one should be good to review/merge as-is.

Yes, he submitted one yesterday. I'll put this forward for doc review. Thanks, @dbwiddis !

kolchfa-aws · 2024-03-22T17:36:36Z

Thank you, @dbwiddis! This PR is my queue for review.

kolchfa-aws

Thank you for providing this documentation, @dbwiddis! A couple of style suggestions. Please review, and then we'll be ready to move the PR to editorial review. Thanks!

_automating-configurations/api/create-workflow.md

kolchfa-aws · 2024-03-22T18:56:16Z

_automating-configurations/api/create-workflow.md


 | Parameter | Data type | Description |
 | :--- | :--- | :--- |
 | `provision` | Boolean | Whether to provision the workflow as part of the request. Default is `false`. |
 | `validation` | String | Whether to validate the workflow. Valid values are `all` (validate the template) and `none` (do not validate the template). Default is `all`. |
+| User-provided | String | Parameters matching substitutions in the template. Optional. |


Instead of "user-provided", can we say something like "User-provided parameters that vary depending on the step type" or something like that?

Some of that is in the description field (and it only varies based on the user template, not step type). I made an attempt in a918a6b that I hope works.

We also may wish to link to the provision-workflow.md file "Query Paremeters" section, which should probably include the identical text here.... just added that as fda9311

@dbwiddis Done. Please take a look.

_automating-configurations/api/provision-workflow.md

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Daniel Widdis <[email protected]>

Signed-off-by: Daniel Widdis <[email protected]>

kolchfa-aws · 2024-03-22T19:40:51Z

_automating-configurations/api/provision-workflow.md

+
+| Parameter | Data type | Description |
+| :--- | :--- | :--- |
+| User-provided | String | Parameters that match substitutions in the template. Optional. |


I was already making a similar edit in fda9311 :)

The wording that you chose is perfect, thank you 😄 Let me add links

Signed-off-by: Daniel Widdis <[email protected]>

Signed-off-by: Fanit Kolchina <[email protected]>

natebower

@dbwiddis @kolchfa-aws Just one minor comment. Thanks!

natebower · 2024-03-22T20:21:06Z

_automating-configurations/api/provision-workflow.md

-#### Example request
+## Query parameters
+
+If you have included a substitution expression in the template, you may pass it as a query parameter or as a string value of a request body field. For example, if you specified a credential field in a template as `openAI_key: '${{ openai_key }}'`, then you can include the `openai_key` parameter as a query parameter or body field so it can be substituted during provisioning.


Should this end in a colon?

Added an intro sentence to the example.

_automating-configurations/api/provision-workflow.md

Signed-off-by: kolchfa-aws <[email protected]>

dbwiddis · 2024-03-25T15:48:39Z

Anything else needed on this? Looks like all comments have been addressed.

kolchfa-aws · 2024-03-25T15:56:33Z

@dbwiddis If you're OK with this commit c594fa1, we're ready to merge.

dbwiddis · 2024-03-25T16:45:39Z

@dbwiddis If you're OK with this commit c594fa1, we're ready to merge.

That change looks good!

dbwiddis requested review from hdhalter, kolchfa-aws, Naarcha-AWS, vagimeli, AMoo-Miki, natebower, dlvenable and stephen-crawford as code owners February 29, 2024 19:58

dbwiddis added the v2.13.0 label Feb 29, 2024

dbwiddis marked this pull request as draft March 4, 2024 20:24

kolchfa-aws self-assigned this Mar 4, 2024

dbwiddis mentioned this pull request Mar 6, 2024

[FEATURE] 2.13 Launch Configurable Automation templates for OpenSearch AI Use Cases opensearch-project/flow-framework#541

Closed

hdhalter added the 2 - In progress Issue/PR: The issue or PR is in progress. label Mar 12, 2024

dbwiddis force-pushed the param-substitution branch from 091e104 to c9d9860 Compare March 20, 2024 16:53

dbwiddis marked this pull request as ready for review March 20, 2024 16:53

dbwiddis force-pushed the param-substitution branch from c9d9860 to 134be1b Compare March 20, 2024 18:42

dbwiddis added 2 commits March 22, 2024 10:26

Add documentation for parameter substitution when provisioning workflows

72c7c1b

Signed-off-by: Daniel Widdis <[email protected]>

fix typo

c94a04d

Signed-off-by: Daniel Widdis <[email protected]>

dbwiddis force-pushed the param-substitution branch from 134be1b to c94a04d Compare March 22, 2024 17:26

hdhalter added 4 - Doc review PR: Doc review in progress and removed 2 - In progress Issue/PR: The issue or PR is in progress. labels Mar 22, 2024

Merge branch 'main' into param-substitution

e8703b8

kolchfa-aws approved these changes Mar 22, 2024

View reviewed changes

dbwiddis and others added 6 commits March 22, 2024 12:30

Update _automating-configurations/api/create-workflow.md

0a83ad2

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Daniel Widdis <[email protected]>

Update _automating-configurations/api/provision-workflow.md

4358e8e

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Daniel Widdis <[email protected]>

Update _automating-configurations/api/provision-workflow.md

ee9afa1

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Daniel Widdis <[email protected]>

Update _automating-configurations/api/provision-workflow.md

98f8ae1

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Daniel Widdis <[email protected]>

Update _automating-configurations/api/provision-workflow.md

c7aa470

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Daniel Widdis <[email protected]>

Clarify user-provided parameters in create workflow API

a918a6b

Signed-off-by: Daniel Widdis <[email protected]>

kolchfa-aws reviewed Mar 22, 2024

View reviewed changes

dbwiddis and others added 2 commits March 22, 2024 12:42

Clarify user-provided parameters in provision workflow API

fda9311

Signed-off-by: Daniel Widdis <[email protected]>

Add link to provision workflow from create workflow

c594fa1

Signed-off-by: Fanit Kolchina <[email protected]>

natebower reviewed Mar 22, 2024

View reviewed changes

kolchfa-aws reviewed Mar 22, 2024

View reviewed changes

_automating-configurations/api/provision-workflow.md Outdated Show resolved Hide resolved

Update _automating-configurations/api/provision-workflow.md

77c325e

Signed-off-by: kolchfa-aws <[email protected]>

kolchfa-aws merged commit b191690 into opensearch-project:main Mar 25, 2024
3 checks passed

hdhalter added 3 - Done Issue is done/complete release-notes PR: Include this PR in the automated release notes and removed 4 - Doc review PR: Doc review in progress labels Mar 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add documentation for parameter substitution when provisioning workflows #6547

Add documentation for parameter substitution when provisioning workflows #6547

dbwiddis commented Feb 29, 2024

hdhalter commented Mar 1, 2024

dbwiddis commented Mar 1, 2024

dbwiddis commented Mar 4, 2024

amitgalitz commented Mar 20, 2024

hdhalter commented Mar 20, 2024

dbwiddis commented Mar 22, 2024

hdhalter commented Mar 22, 2024

kolchfa-aws commented Mar 22, 2024

kolchfa-aws left a comment

kolchfa-aws Mar 22, 2024

dbwiddis Mar 22, 2024

kolchfa-aws Mar 22, 2024

kolchfa-aws Mar 22, 2024

dbwiddis Mar 22, 2024

kolchfa-aws Mar 22, 2024

natebower left a comment

natebower Mar 22, 2024

kolchfa-aws Mar 22, 2024

dbwiddis commented Mar 25, 2024

kolchfa-aws commented Mar 25, 2024

dbwiddis commented Mar 25, 2024

	\| User-provided \| String \| Parameters that match substitutions in the template. Optional. \|
	\| User-provided substitution expressions \| String \| Parameters that match substitutions in the template. Optional. \|

Add documentation for parameter substitution when provisioning workflows #6547

Add documentation for parameter substitution when provisioning workflows #6547

Conversation

dbwiddis commented Feb 29, 2024

Description

Checklist

hdhalter commented Mar 1, 2024

dbwiddis commented Mar 1, 2024

dbwiddis commented Mar 4, 2024

amitgalitz commented Mar 20, 2024

hdhalter commented Mar 20, 2024

dbwiddis commented Mar 22, 2024

hdhalter commented Mar 22, 2024

kolchfa-aws commented Mar 22, 2024

kolchfa-aws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

natebower left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dbwiddis commented Mar 25, 2024

kolchfa-aws commented Mar 25, 2024

dbwiddis commented Mar 25, 2024