This repository is for the AI Tour workshop: Interacting with Multimodal Models in Azure AI Foundry
This workshop is designed to give you a hands-on introduction to the core concepts and best practices for interacting with OpenAI models in Azure AI Foundry portal. If you have been provided with a Skillable Environment, you'll be using the VM and pre-provisioned Azure resources provided to you to complete the lab. If you are running this workshop on your own, you will need to have an Azure subscription and provision the resources yourself by deploying the resources to Azure.
Innovate with Azure OpenAI's GPT-4o multimodal model in this hands-on experience in Azure AI Foundry. Learn the core concepts and best practices to effectively generate with text, sound, and images using GPT-4o-mini, DALL-E and GPT-4o-realtime. Create AI assistants that enhance user experiences and drive innovation.
75 Minutes
- Understand how Large Language Models work, including what tokens are.
- Explore Prompt Engineering techniques and best practices
- Understand how models apply existing knowledge
- Get started with building an Azure AI assistant
- Implementing function calling in LLM applications
- Azure AI Foundry portal
The step by step workshop instructions can be found as follows:
- Skillable Workshop Instructions
- Workshop Instructions if you are deploying directly on Azure - for Field Stops
| Resources | Links | Description |
|---|---|---|
| Session Slides | View | Review the slides presented during the workshop at your own pace |
| Intro to Azure OpenAI Service | Microsoft Learn module | Learn more about Azure OpenAI Service |
| Azure OpenAI Service documentation | Azure OpenAI Service documentation | Learn more about Azure OpenAI Service |
| Azure OpenAI Service pricing | Pricing details | Learn more about Azure OpenAI Service pricing |
| Transparency Note for Azure OpenAI Service | Transparency Note | Learn more about Azure OpenAI Service use cases, capabilities and limitations |
 Gustavo Cordido 📢 |
 Bethany Jepchumba 📢 |
Microsoft is committed to helping our customers use our AI products responsibly, sharing our learnings, and building trust-based partnerships through tools like Transparency Notes and Impact Assessments. Many of these resources can be found at https://aka.ms/RAI. Microsoft’s approach to responsible AI is grounded in our AI principles of fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability.
Large-scale natural language, image, and speech models - like the ones used in this sample - can potentially behave in ways that are unfair, unreliable, or offensive, in turn causing harms. Please consult the Azure OpenAI service Transparency note to be informed about risks and limitations.
The recommended approach to mitigating these risks is to include a safety system in your architecture that can detect and prevent harmful behavior. Azure AI Content Safety provides an independent layer of protection, able to detect harmful user-generated and AI-generated content in applications and services. Azure AI Content Safety includes text and image APIs that allow you to detect material that is harmful. Within Azure AI Foundry portal, the Content Safety service allows you to view, explore and try out sample code for detecting harmful content across different modalities. The following quickstart documentation guides you through making requests to the service.
Another aspect to take into account is the overall application performance. With multi-modal and multi-models applications, we consider performance to mean that the system performs as you and your users expect, including not generating harmful outputs. It's important to assess the performance of your overall application using Performance and Quality and Risk and Safety evaluators. You also have the ability to create and evaluate with custom evaluators.
You can evaluate your AI application in your development environment using the Azure AI Evaluation SDK. Given either a test dataset or a target, your generative AI application generations are quantitatively measured with built-in evaluators or custom evaluators of your choice. To get started with the azure ai evaluation sdk to evaluate your system, you can follow the quickstart guide. Once you execute an evaluation run, you can visualize the results in Azure AI Foundry portal .