Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more clarity around how to generate the bearer token #100

Closed
wants to merge 9 commits into from

Conversation

pwoznic
Copy link
Contributor

@pwoznic pwoznic commented Sep 29, 2023

Updated the topic with an updated screenshot, steps to create the app client, and examples (curl and nodejs) about how to generate the bearer token.

Updated the topic with an updated screenshot, steps to create the app client, and examples (curl and nodejs) about how to generate the bearer token.
@netlify
Copy link

netlify bot commented Sep 29, 2023

Deploy Preview for luxury-shortbread-acee05 ready!

Name Link
🔨 Latest commit 342a200
🔍 Latest deploy log https://app.netlify.com/sites/luxury-shortbread-acee05/deploys/6529b8d656d1b400085b62a9
😎 Deploy Preview https://deploy-preview-100--luxury-shortbread-acee05.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link
Contributor

@ofermend ofermend left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Minor updates
Added an intro to each topic that required OAuth with links to example credential grants. I also updated the topics that can use both methods.
Updated text across topics
Copy link
Collaborator

@cjcenizal cjcenizal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this Paul! I read through the docs from start to finish and was still left feeling confused. Here are some of the points that confused me:

  • Why link to client credentials grant? That page doesn't contain any additional information and we don't explain what that grant does or means compared to other options.
  • Why mention username/password auth? AFAIK we don't support API requests with this auth mechanism.
  • The information under the "How to obtain a JWT token" is verbose and makes references to things that don't seem to be mentioned in the document. For example: "the redirect URI must match the redirect URL configured for the client. Note the peculiarities of the HTTP authorization header: this is per the OAuth 2.0 standard." Neither redirect URI or HTTP authorization header are mentioned in the doc.
  • The ordering and structuring of this document confuses me. There's a "How to obtain a JWT token" and also a "Generate the JWT token" heading, which sound the same to me. What's the difference? There's a lot of emphasis put on creating an app client, which is a very straightforward process. There's a lot of emphasis put on generating a token, but a lot of the info seems irrelevant or duplicated. There's almost no emphasis put on how to use the token.

Here's how I propose we structure and edit down this page to address these issues.


Vectara uses "Application clients" to support authentication with OAuth 2.0. These application clients enable you to generate JWT tokens which, like API keys, are used by Vectara to authenticate API requests.

Always keep your OAuth tokens private. Do not share them through email, Slack, Discord, forums, or other public channels because it can lead to unauthorized access. Treat these tokens with the same confidentiality as your personal credentials.

OAuth 2.0 vs API keys

OAuth 2.0 takes more work to set up than API keys but offers many advantages over API keys:

  • OAuth 2.0 has built in revocation flows in case a token is compromised.
  • The JWT token will automatically expire , so if a JWT token ever does get posted to a public place, it's less likely to be valid by the time an attacker discovers it.
  • OAuth 2.0 is inherently more tightly scoped than API keys.
  • JWT tokens are detected by many security scanning tools, allowing them to more easily be flagged in the case of accidental publication.

Authenticate with OAuth 2.0

OAuth 2.0 authentication consists of three steps:

  1. Create an application client
  2. Generate a JWT token
  3. Use the JWT token in an API request

1. Create an application client

  1. In Console, visit the “API access” page and open the “App clients” tab.
  2. Click the “Create app client” button and configure it with the name, description, and roles you want.

This page provides three pieces of information you’ll use to generate a JWT token:

Authentication URL

Access your authentication URL by clicking the copy button for the “OAuth 2.0 authentication URL”.

(screenshot)

Client ID

Access the client_id by clicking the copy button next to your app client’s ID.

(screenshot)

Client secret

Access the client_secret by clicking the drop-down button to the right of your app client and selecting “Copy secret”.

(screenshot)

2. Generate a JWT token

Use the information from the previous step to send a request to generate a JWT token.

Here’s how you can generate a JWT token in JavaScript. This is how you can authenticate Vectara API requests in a JavaScript application.

const { access_token } = await axios.post(
  <your authentication URL goes here>,
  {
    grant_type: "client_credentials",
    client_id: <your client ID goes here>,
    client_secret: <“your client secret goes here>
  }
 );

Here’s how you can generate one from the command line. This is useful if you want to try out requests in our API Playground.

curl -XPOST -H "Content-type: application/x-www-form-urlencoded" -d 
"grant_type=client_credentials&client_id=<your client ID goes here>&client_secret=<your client secret goes here> <your authentication URL goes here> 

3. Use the JWT token in an API request

To use a JWT token in an API request, pass it using the Authorization header configuration.

(Use a code example here similar to https://docs.vectara.com/docs/common-use-cases/app-authn-authz/api-keys#use-an-api-key)

If you’re using the API Playground, use the JWT token to fill in the Bearer Token field.

(Screenshot)

@@ -13,36 +16,61 @@ Fundamentally, you provide an OAuth 2.0 authentication provider with a
"client ID" which is *like* a username and a "client secret" which is *like*
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: remove italics because they don't convey additional info


API Keys can be used for querying and indexing operations, but cannot at this
time be used for administrative operations such as creating or deleting
corpora. These operations authenticate with a Bearer Token via the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this referred to as Bearer Token in the API playground and JWT token in the OAuth docs. I think JWT token is more common, so I suggest we consistently refer to it using this term, unless in the context of the API playground.

@cjcenizal
Copy link
Collaborator

BTW here's the correct way to make a request to generate a JWT token in JavaScript:

const {
  data: { access_token: jwt }
} = await axios({
  method: "POST",
  headers: { "content-type": "application/x-www-form-urlencoded" },
  data: qs.stringify({
    grant_type: "client_credentials",
    client_id: "<your client ID goes here>",
    client_secret: "<your client secret goes here>"
  }),
  url: "<your authentication URL goes here>"
});

Incorporated feedback into a new version of the topic
Copy link
Collaborator

@cjcenizal cjcenizal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I had a number of questions and suggestions, but I'm OK with addressing them in a subsequent PR if you'd like to merge this now.

In terms of images, I think we can help folks zero in on the bits of the UI that matter and also make our images more future-proof by cropping out the parts of the screen that aren't relevant. For example:

image

image

image

Vectara uses "Application clients" to support authentication with OAuth 2.0.
These application clients enable you to generate JWT tokens which are used by
Vectara to authenticate API requests. Think of OAuth 2.0 has a more secure way
to send API calls, similar to an API key or username/password combination
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why mention username/password auth? AFAIK we don't support API requests with this auth mechanism.


OAuth 2.0 authentication consists of three steps:
1. [Create an application client](/docs/api-reference/auth-apis/oauth-2#create-an-application-client)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to use anchors you can remove the path and just use the anchor tag, e.g. #create-an-application-client in this case. This will be more robust against changes to the file organization.

But TBH I think we should remove these links. As a user looking at them, it's not clear that they link to anchors. I actually thought they linked out to other pages and had to hover the link to figure out where they want. Since the anchors are just a little bit father down (and the user will almost certainly scroll to them) I don't think the links provide much value.

### Generate a JWT Token

Use the information from the previous step to send a request to generate a JWT
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a "request to generate a JWT token" different from a "client credentials request"? If they're the same, can we use consistent terminology?

@@ -22,6 +22,11 @@ import TabItem from "@theme/TabItem";

## CreateCorpus

This operation authenticates with an OAuth 2.0 Bearer
Token (also known as a JWT token) via the client credentials grant.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "via the client credentials grant" an important detail? I'm genuinely curious. We mention it in many places.

Now that you have values for the authentication URL, `client_id`, and `client_secret`,
you can generate the JWT token with a `client-credentials` grant. We provide [client
credentials grant examples](/docs/getting-started-samples/JWTFetcher.cs) in different programming languages.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This page seems like it would be better titled "JWT token". This seems to be the term we're using everywhere, and "client credentials grant" seems like it's just an implementation detail when fetching the token. Why are we elevating that term, when folks will be looking for "JWT token"?

This paragraph also seems like it belongs in the "Generate a JWT Token" section.

* Add MMR reranker content

Updated Reranking topic with info about MMR

* Update reranking.md

Updated based on comments
* Create Learn section

Created the new Learn section and began moving topics

* Additional updates

Additional updates

* Fix links

Fixed links from deploy error

* Update IA for new Learn section

Continuing to move, merge, rename, etc WIP

* Fix links

Fixed broken links from deploy log

* Fix more links

More links updated

* Continue updating Learn section

More WIP changes

* Update topics in learn section

Updated topics WIP

* Fix links

Updated links WIP

* Update OAuth section

I moved the OAuth topic to the Authentication section of Learn (this PR is still draft as I continue to move and rename topics). In the meantime issue 96 was opened to help clarify how to generate a Bearer Token. I will create a separate branch and PR to get that addressed asap as this Learn section PR is still in draft.

* Update Learn section

Updated several topics and modified the sidebar - WIP

* Update Learn IA

More updates - WIP

* Update links

Fixed links

* More updates

Additional WIP updates

* Update redirects

* More link fixes

* Update _redirects

* Create overview.md

* Fixed a couple links

---------

Signed-off-by: Paul Wozniczka <[email protected]>
I discussed this PR with Shane for more context. The username/pw is just an analogy because many readers might not be familiar with oauth. At the moment we only support the client credentials grant as the OAuth flow vs other methods.
@pwoznic pwoznic closed this Nov 3, 2023
@pwoznic pwoznic deleted the generate-token branch November 3, 2023 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants