Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions docs/intelligentapps/images/models/models.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
113 changes: 59 additions & 54 deletions docs/intelligentapps/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,38 @@ ContentId: 52ad40fe-f352-4e16-a075-7a9606c5df3b
DateApproved: 03/12/2026
MetaDescription: Find a popular generative AI model by publisher and source. Bring your own model that is hosted with a URL, or select an Ollama model.
---
# Explore models in Foundry Toolkit
# Explore and manage AI models in Foundry Toolkit

Foundry Toolkit provides comprehensive support for a wide variety of generative AI models.
Foundry Toolkit provides comprehensive support for exploring, deploying, and testing a wide variety of generative AI models.

With Foundry Toolkit, you can:

- Explore remote and local models from Microsoft Foundry, third-party providers, and local runtimes.
- Deploy models to Microsoft Foundry, connect to provider-hosted or custom endpoints, or download supported local models.
- Test text and image models in the [**Playground**](/docs/intelligentapps/playground.md).

Within the model catalog, you can explore and utilize models from multiple hosting sources:

- Models hosted on GitHub, such as Llama3, Phi-3, and Mistral, including pay-as-you-go options.
- Models provided directly by publishers, including OpenAI's ChatGPT, Anthropic's Claude, and Google's Gemini.
- Models hosted on Microsoft Foundry.
- Models downloaded locally from repositories like Foundry Local, Ollama, and ONNX.
- Custom self-hosted or externally deployed models accessible via Bring-Your-Own-Model (BYOM) integration.
- Models provided directly by publishers, including OpenAI's ChatGPT, Anthropic's Claude, Google's Gemini, and NVIDIA NIM models.
- Local models from Microsoft Foundry on Windows, Foundry Local, Ollama, and ONNX.
- Custom self-hosted or externally deployed models accessible through Bring Your Own Model (BYOM).

Deploy models directly to Foundry from within the model catalog, streamlining your workflow.

> [!NOTE]
> Use Microsoft Foundry, Foundry Local, and GitHub models added to Foundry Toolkit with GitHub Copilot. For more information, check out [Changing the model for chat](/docs/agent-customization/language-models.md#change-the-model-for-chat).
> Region availability, quota, pricing, authentication, deployment eligibility, and safety limits are service-specific. Use Microsoft Learn and Microsoft Foundry documentation as the source of truth for those requirements. For more information, see [Azure AI Foundry model region availability](https://learn.microsoft.com/azure/ai-foundry/concepts/models-featured#region-availability) and [Azure AI Foundry quota](https://learn.microsoft.com/azure/ai-foundry/how-to/quota).

## Supported model sources

| Source | Use it to | Notes |
| ------ | --------- | ----- |
| Microsoft Foundry | Browse catalog models, deploy models to a selected Foundry project, and use deployed project models. | Availability depends on your project, region, quota, and service eligibility. |
| Microsoft Foundry on Windows / Foundry Local | Download and run supported local models. | Foundry Toolkit shows **Microsoft Foundry on Windows** on Windows and **Foundry Local** on macOS. |
| Ollama | Add local Ollama models or connect to a custom Ollama endpoint. | Install Ollama and download the model in Ollama before adding it to Foundry Toolkit. |
| ONNX / ONNX (Converted) | Use local ONNX models in Foundry Toolkit. | Convert models to the Foundry Toolkit model format before adding them. |
| OpenAI, Anthropic, Google, and NVIDIA NIM | Use supported publisher-hosted models. | Authentication, pricing, and feature availability depend on the provider. |
| Custom / BYOM | Add a self-hosted or externally deployed OpenAI-compatible endpoint. | Provide the endpoint URL, model name, and authentication details. |

![Foundry Toolkit model catalog displaying various generative AI models](./images/models/models.png)

Expand All @@ -30,45 +46,18 @@ To find a model in the model catalog:
1. Select **Developer Tools** > **Discover** > **Model Catalog** to open the model catalog
1. Use the filters to reduce the list of available models.

- **Hosted by**: Foundry Toolkit supports Microsoft Foundry, Foundry Local, GitHub, ONNX, OpenAI, Ollama, Anthropic, Google, NVIDIA NIM, MiniMax, Kimi, GLM and Windows AI API as model hosting sources.
- **Hosted by**: Foundry Toolkit supports Microsoft Foundry, Microsoft Foundry on Windows, Foundry Local, ONNX, OpenAI, Ollama, Anthropic, Google, NVIDIA NIM, Custom, and other model hosting sources.
- **Publisher**: The publisher for AI models, such as Microsoft, Meta, Google, OpenAI, Anthropic, Mistral AI, and more.
- **Feature**: Supported features of the model, such as `Text Attachment`, `Image Attachment`, `Web Search`, `Structured Outputs`, and more.
- **Model type**: Filter models that can run remotely or locally on CPU, GPU, or NPU. This filter depends on the local availability.
- **Fine-tuning Support**: Show models that can be used to run fine-tuning.
1. Browse the models in different categories, such as:
- **Popular Models** is a curated list of widely used models across various tasks and domains.
- **Microsoft Foundry Hosted Models** provide easy access to popular models hosted on Microsoft Foundry.
- **GitHub Models** provide easy access to popular models hosted on GitHub. It's best for fast prototyping and experimentation.
- **Local Models** to select models hosted on Microsoft Foundry on Windows or Ollama, supporting CPU via GGUF quantization.
- **Local Models** lets you select local models from Microsoft Foundry on Windows, Foundry Local, Ollama, or ONNX, depending on your platform and installed runtimes.
- **Add Custom Models** lets you add self-hosted or externally deployed OpenAI-compatible endpoints.
1. Alternatively, use the search box to find a specific model by name or description

## Add a model

Depending on your starting point, there are different flows for adding a model to work with in Foundry Toolkit.

### Add a model from the catalog

To add a model from the model catalog:

1. Locate the model you want to add in the model catalog
1. The flow for adding models is slightly different based on the providers:

- **Microsoft Foundry**: Deploy a model in your Microsoft Foundry project. Refer to [Deploy a model to Microsoft Foundry](#deploy-a-model-to-microsoft-foundry) for detailed instructions.

- **Custom Model** that is hosted remotely, requires an API key, and has an OpenAI chat completion compatible endpoint URL. Refer to the [Add a custom model](#add-a-custom-model) for detailed instructions.

- **Foundry Local**: Foundry Local downloads and runs the model, which might take a few minutes depending on your internet speed. The model is available on a localhost page and added to Foundry Toolkit. Learn more in [What is Foundry Local?](https://learn.microsoft.com/azure/ai-foundry/foundry-local/what-is-foundry-local?view=foundry-classic&preserve-view=true).

- **Ollama**: The model is downloaded from Ollama and added to Foundry Toolkit. Refer to the [Add Ollama models](#add-ollama-models) for detailed instructions.

- **GitHub**: Foundry Toolkit asks for your GitHub credentials to access the model repository. Once authenticated, the model is added directly into Foundry Toolkit.
> [!NOTE]
> Foundry Toolkit now [supports GitHub pay-as-you-go models](/docs/intelligentapps/playground.md#github-pay-as-you-go-model-support), so you can keep working after passing free tier limits.

- **ONNX**: To add an ONNX model, first convert it to the Foundry Toolkit model format using the [model conversion tool](/docs/intelligentapps/modelconversion.md). After conversion, add the model to Foundry Toolkit.

Once a model is added, the model appears under **MY RESOURCES/Models** in the tree view, and you can use it in the [**Playground**](/docs/intelligentapps/playground.md) or [**Agent Builder**](/docs/intelligentapps/agentbuilder.md).

## Deploy a model to Microsoft Foundry

Deploy a model to Microsoft Foundry directly from Foundry Toolkit. Run the model in the cloud and access it via an endpoint.
Expand All @@ -84,9 +73,24 @@ Deploy a model to Microsoft Foundry directly from Foundry Toolkit. Run the model

1. Select **Deploy to Microsoft Foundry** to start the deployment process.
1. Confirm the deployment by reviewing the details and selecting **Deploy** to proceed.
1. Once the deployment is complete, the model is available in the **MY RESOURCES** > **Your project name** > **Models** section of Foundry Toolkit, and you can use it in the playground or agent builder.
1. Once the deployment is complete, the model is available in **My Resources** > **Your project name** > **Models**, and you can use it in the playground or agent builder.

## Add or connect a model

Depending on your starting point and model source, there are different flows for adding or connecting a model to work with in Foundry Toolkit.

1. Locate the model you want to add in the model catalog.
1. Choose the action for the model source:

- **Microsoft Foundry**: Deploy a model in your Microsoft Foundry project. For detailed instructions, see [Deploy a model to Microsoft Foundry](#deploy-a-model-to-microsoft-foundry).
- **Custom Model**: Connect to a remotely hosted model that uses an OpenAI compatible endpoint. For detailed instructions, see [Connect a custom model](#connect-a-custom-model).
- **Microsoft Foundry on Windows / Foundry Local**: Download and run the model locally, which might take a few minutes depending on your internet speed. Learn more in [What is Foundry Local?](https://learn.microsoft.com/azure/ai-foundry/foundry-local/what-is-foundry-local?view=foundry-classic&preserve-view=true).
- **Ollama**: Download the model from Ollama and add it to Foundry Toolkit. For detailed instructions, see [Add Ollama models](#add-ollama-models).
- **ONNX**: Convert the ONNX model to the Foundry Toolkit model format using the [model conversion tool](/docs/intelligentapps/modelconversion.md), and then add it to Foundry Toolkit.

Once a model is added, you can use it in the [**Playground**](/docs/intelligentapps/playground.md) or [**Agent Builder**](/docs/intelligentapps/agentbuilder.md). Microsoft Foundry models appear under **My Resources** > **Your project name** > **Models**. Local models appear under **My Resources** > **Local Resources** > **Models**.

### Add a custom model
### Connect a custom model

For self-hosted or deployed models accessible from the internet with an OpenAI compatible endpoint, add it to Foundry Toolkit for use in the playground.

Expand All @@ -108,13 +112,13 @@ For self-hosted or deployed models accessible from the internet with an OpenAI c

Ollama enables many popular genAI models to run locally with CPU via GGUF quantization. If Ollama is installed on your local machine with downloaded Ollama models, add them to Foundry Toolkit for use in the model playground.

Prerequisites for using Ollama models in the Microsoft Foundry Toolkit for {% data variables.product.prodname_vscode %} extension is [Ollama](https://ollama.com/download) (Tested on Ollama v0.4.1).
To use Ollama models in Foundry Toolkit, install [Ollama](https://ollama.com/download) and download the models you want to add.

To add local Ollama into the Foundry Toolkit
To add local Ollama models to Foundry Toolkit:

1. From one of the entrypoints mentioned previously, select **Add Ollama Model**.

- **MY RESOURCES** > **Local Resources**, select the **+** button next to **Models**.
- In **My Resources** > **Local Resources**, select the **+** button next to **Models**.

![Screenshot of the plus button next to models in local resources.](./images/models/my-resources-local-resources-models.png)

Expand All @@ -128,7 +132,7 @@ To add local Ollama into the Foundry Toolkit

1. Select **Continue** after reading the acknowledgement that Ollama is a third-party model provider.

![SCreenshot of Ollama acknowledgement.](./images/models/ollama-acknowledgement.png)
![Screenshot of Ollama acknowledgement.](./images/models/ollama-acknowledgement.png)

1. Next, select **Select models from Ollama library**. This displays the models you have installed in Ollama. Use the checkboxes to select the ones you want to use with Foundry Toolkit.

Expand All @@ -155,16 +159,17 @@ Use the actions on the model card in the model catalog:

## Manage models

You can manage your models in the **MY RESOURCES/Models** section of the Foundry Toolkit sidebar:
You can manage your models from the **My Resources** web view in Foundry Toolkit:

- View the list of models added to Foundry Toolkit.
- Right-click on a model to access options such as:
- For Microsoft Foundry models, go to **My Resources**. Foundry models are grouped by project in the web view.
- For local models, go to **My Resources** > **Local Resources** > **Models**.
- Right-click a model to access options such as:
- **Load in Playground**: Load the model in the [Playground](/docs/intelligentapps/playground.md) for testing.
- **Copy Model Name**: Copy the model name to the clipboard for use in other contexts, such as your code integration.
- **Refresh**: Refresh the model configuration to ensure you have the latest settings.
- **Edit**: Modify the model settings, such as the API key or endpoint.
- **Delete**: Remove the model from Foundry Toolkit.
- **About this Model**: View detailed information about the model, including its publisher, source, and supported features.
- **Refresh**: Refresh the model configuration to ensure you have the latest settings.
- **Edit**: Modify the model settings, such as the API key or endpoint.
- **Delete**: Remove the model from Foundry Toolkit.
- **About this Model**: View detailed information about the model, including its publisher, source, and supported features.

- Right-click on `ONNX` section title to access options such as:
- **Start Server**: Start the ONNX server to run ONNX models locally.
Expand All @@ -173,20 +178,20 @@ You can manage your models in the **MY RESOURCES/Models** section of the Foundry

## License and sign-in

Some models require a publisher or hosting-service license and account to sign-in. In that case, before you can run the model in the [model playground](/docs/intelligentapps/playground.md), you're prompted to provide this information.
Some models require a publisher or hosting-service license and account sign-in. In that case, before you can run the model in the [model playground](/docs/intelligentapps/playground.md), you're prompted to provide this information.

## What you learned

In this article, you learned how to:

- Explore and manage generative AI models in Foundry Toolkit.
- Find models from various sources, including Microsoft Foundry, Foundry Local, GitHub, ONNX, OpenAI, Anthropic, Google, Ollama, and custom endpoints.
- Find models from various sources, including Microsoft Foundry, Microsoft Foundry on Windows, Foundry Local, ONNX, OpenAI, Anthropic, Google, NVIDIA NIM, Ollama, and custom endpoints.
- Add models to your toolkit and deploy them to Microsoft Foundry.
- Add custom models, including Ollama and OpenAI compatible models, and test them in the playground or agent builder.
- Use the model catalog to view available models and select the best fit for your AI application needs.
- Use filters and search to find models quickly.
- Browse models by category, such as Popular, GitHub, ONNX, and Ollama.
- Browse models by category, such as Popular, Microsoft Foundry, Local, Custom, and More.
- Convert and add custom ONNX models using the model conversion tool.
- Manage models in MY RESOURCES/Models, including editing, deleting, refreshing, and viewing details.
- Manage models from **My Resources**, including editing, deleting, refreshing, and viewing details.
- Start and stop the ONNX server and copy endpoints for local models.
- Handle license and sign-in requirements for some models before testing them.