日本語

SLM Innovator Lab

Unlock the full potential of your AI projects with the SLM Innovator Lab, powered by the Azure AI/ML Platform. Our lab is tailored for customers who excel in fine-tuning and deploying multiple SLM models on Azure, as well as those aiming to optimize base model performance through fine-tuning to create RAG applications. With the advanced capabilities of AI Studio, you can establish efficient and scalable LLMOps.

This hands-on lab is suitable for the following purposes:

1. 1-day workshop (4-7 hours depending on customer) / 2-day workshop with LLMOps hands-on
2. Hackathon starter code
3. Reference guide for SLM fine-tuning&serving PoC/Prototype

For detailed workshop information, please see the hands-on guide at this link.

Hands-on guide | Requirements | Get started

New content (25-Oct-2024)

🔥 LLMOps with Promptflow Python SDK
In this hands-on, you will learn how to create a new flow, define the chat flow structure, and integrate the fine-tuned model endpoint using Python SDK. You will also learn how to compare and evaluate the model's performance using the flows. This is in addition to the hands-on that was previously available based on the Azure AI Studio UI.
Go to notebook

🔥 Microsoft Olive model optimization
Microsoft Olive is a hardware-aware AI model optimization toolchain developed by Microsoft to streamline the deployment of AI models. Olive simplifies the process of preparing AI models for deployment by making them faster and more efficient, particularly for use on edge devices, cloud, and various hardware configurations. This hands-on considers on-device or hybrid deployment scenarios.
Go to notebook

🔥 Content Safety with Python SDK
In this hands-on, you will be able to: manage text blocklist, analyze text and images for sexual content, violence, hate, and self-harm with multi-severity levels. You will also learn how to integrate with Azure Open AI Service: Use the Azure Open AI Service to rewrite the content for harmful content.
Go to notebook

Requirements

Before starting, you should meet the following requirements:

• Access to Azure OpenAI Service

• Azure ML getting started: Connect to Azure ML workspace and get your <WORKSPACE_NAME>, <RESOURCE_GROUP> and <SUBSCRIPTION_ID>.

• Azure AI Studio getting started: Create a project

• Azure AI Document Intelligence (v4.0 - 2024-02-29 preview)

• [Compute instance - for code development] A low-end instance without GPU is recommended: Standard_E2as_v4 (AMD 2 cores, 16GB RAM, 32GB storage) or Standard_DS11_v2 (Intel 2 cores, 14GB RAM, 28GB storage, No GPUs)

• [Compute cluster - for SLM/LLM fine-tuning] A single NVIDIA A100 GPU node (Standard_NC24ads_A100_v4) is recommended. If you do not have a dedicated quota or are on a tight budget, choose Low-priority VM.

• [SLM/LLM deployment] Two NVIDIA V100 GPUs (Standard_NC6s_v3) or two NVIDIA A100 GPUs (Standard_NC24ads_A100_v4) are recommended.

Note For managed online endpoints, [Azure ML reserves 20% of the quota for the deployment].¹ If you request a given number of instances for those VM SKUs in a deployment, you must have a quota for ceil(1.2 × number of instances requested for deployment) × number of cores for the VM SKU available to avoid getting an error. For example, if you request 1 instances of a Standard_NC6s_v3 VM (that comes with six cores) in a deployment, you should have a quota for 12 cores (ceil(1.2 × 1 instances) = 2, 2 × 6 cores) available.

In case you do not have any of the above requirements ready yet, please go to Lab preparation first.

Lab 0. Lab preparation

Please do not forget to modify the .env file to match your account. Rename .env.sample to .env or copy and use it

Cautions

This workshop assumes that you are configuring in a public environment and you have access to the internet. If you are configuring in a private environment, you may need to set up a private network to access the services. The following are some common issues you may encounter when you configure in a private environment:

• If you set up the Azure ML workspace and Azure AI Studio in private network, you may need to set up a VPN or a private link to access the services.
• If you are using a low-priority VM, you may need to wait for the VM to be available. The availability of the VMs may vary by region.
• If you have blob storage, you can use it to store the data and models. However, you may need to set up the connection to the blob storage in the Azure ML workspace.
• If you have a quota issue, you may need to request a quota increase for the VMs or GPUs.
• Once you configure the network in Azure ML workspace, you can not change it. You may need to create a new workspace if you want to change the network.
• If you are using a compute instance which is not in the same region as the Azure ML workspace, you may need to set up a VPN or a private link to access the services.
• If you are using a compute instance which created in Azure AI Studio, you can't execute training jobs in the compute instance. You may need to create a new compute instance in Azure ML workspace.
• If you run into an PermissionMismatch error when you download the artifacts, you may need to asign the correct permission to the Azure ML workspace.

Get started

1. Create your compute instance in Azure ML. For code development, we recommend Standard_DS11_v2 (2 cores, 14GB RAM, 28GB storage, No GPUs).
2. Open the terminal of the CI and run:

   git clone https://github.com/Azure/slm-innovator-lab.git
   cd slm-innovator-lab && conda activate azureml_py310_sdkv2
   pip install -r requirements.txt

Hands-on Labs

Lab 1. Data preparation

Lab 2. LLM fine-tuning and serving

Lab 3. LLMOps

References

Expand

Data preparation

• Evolve-Instruct
• GLAN (Generalized Instruction Tuning)
• Auto Evolve-Instruct
• Azure Machine Learning examples

SLM fine-tuning

Phi-3/Phi-3.5

• Finetune Small Language Model (SLM) Phi-3 using Azure ML
• microsoft/Phi-3-mini-4k-instruct: This is Microsoft's official Phi-3-mini-4k-instruct model.
• microsoft/Phi-3-mini-128k-instruct: This is Microsoft's official Phi-3-mini-128k-instruct model.
• microsoft/Phi-3.5-mini-instruct: This is Microsoft's official Phi-3.5-mini-instruct model.
• microsoft/Phi-3.5-MoE-instruct: This is Microsoft's official Phi-3.5-MoE-instruct model.
• Korean language proficiency evaluation for LLM/SLM models using KMMLU, CLIcK, and HAE-RAE dataset
• daekeun-ml/Phi-3-medium-4k-instruct-ko-poc-v0.1

Florence-2

• Fine-tuning Florence-2 for VQA (Visual Question Answering) using the Azure ML Python SDK and MLflow
• Hugging Face Blog - Finetune Florence-2 on DoCVQA

LLMOps

• LLMOps with Prompt flow (Supports both AI Studio and Azure Machine Learning

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

License Summary

This sample code is provided under the MIT-0 license. See the LICENSE file.

Footnotes

1. This extra quota is reserved for system-initiated operations such as OS upgrades and VM recovery, and it won't incur cost unless such operations run. ↩