Below are example code snippets for a few use cases. For additional information about Azure AI Inference SDK, see full documentation and samples.

1. Create a personal access token

Go to personal access tokens

You do not need to give any permissions to the token. Note that the token will be sent to a Microsoft service.

To use the code snippets below, create an environment variable to set your token as the key for the client code.

If you're using bash:

export GITHUB_TOKEN="<your-github-token-goes-here>"

If you're in powershell:

$Env:GITHUB_TOKEN="<your-github-token-goes-here>"

If you're using Windows command prompt:

set GITHUB_TOKEN=<your-github-token-goes-here>

2. Install dependencies

Install Node.js.
Copy the following lines of text and save them as a file package.json inside your folder.

{
  "type": "module",
  "dependencies": {
    "@azure-rest/ai-inference": "latest",
    "@azure/core-auth": "latest",
    "@azure/core-sse": "latest"
  }
}

Note: @azure/core-sse is only needed when you stream the chat completions response.

Open a terminal window in this folder and run npm install.
For each of the code snippets below, copy the content into a file sample.js and run with node sample.js.

3. Run a basic code sample

This sample demonstrates a call to embeddings API. It is leveraging the GitHub AI model inference endpoint and your GitHub token. The call is synchronous.

import ModelClient from "@azure-rest/ai-inference";
import { isUnexpected } from "@azure-rest/ai-inference";
import { AzureKeyCredential } from "@azure/core-auth";

const token = process.env["GITHUB_TOKEN"];
const endpoint = "https://models.inference.ai.azure.com";
const modelName = "text-embedding-3-small";

export async function main() {

  const client = new ModelClient(endpoint, new AzureKeyCredential(token));

  const response = await client.path("/embeddings").post({
    body: {
      input: ["first phrase", "second phrase", "third phrase"],
      model: modelName
    }
  });

  if (isUnexpected(response)) {
    throw response.body.error;
  }

  for (const item of response.body.data) {
    let length = item.embedding.length;
    console.log(
	  `data[${item.index}]: length=${length}, ` +
	  `[${item.embedding[0]}, ${item.embedding[1]}, ` +
	  `..., ${item.embedding[length - 2]}, ${item.embedding[length -1]}]`);
  }
  console.log(response.body.usage);
}

main().catch((err) => {
  console.error("The sample encountered an error:", err);
});

4. Going beyond rate limits

The rate limits for the playground and free API usage are intended to help you experiment with models and prototype your AI application. For use beyond those limits, and to bring your application to scale, you must provision resources from an Azure account, and authenticate from there instead of your GitHub personal access token. You don't need to change anything else in your code. Use this link to discover how to go beyond the free tier limits in Azure AI.

Model navigation navigation

Tags

Languages

About

Tags

Languages