- Description
- Prerequisites
- Azure Setup
- Local machine setup
- Review and update the configuration files
- Azure functions deployment
- Cleaning (Optional)
- Credits
- References
The current repo serves as a playground to demonstrate how to switch an Azure Service Bus to RabbitMq in the context of an Azure Function and local development environment. It is trying to solve the lack of a local emulator for Azure Service Bus. The issue is mentioned by Jimmy Bogard in this article. It provides a local, completly isolated dev environment for Azure Functions inside local Kubernetes infrastructure.
The proposed solution is using:
- a MassTransit wrapper for Azure Function RabbitMq trigger
- a Kubernetes in Docker infrastructure
- KEDA for event driven scaling of Azure Functions instances
- Azure Application Insights (optional) for monitoring
The switching procedure is based on a custom configuration (named LocalDev) that is conditionally processed in the Azure Functions csproj files.
Note: Because RabbitMq does not have an equivalent for the ASB topic, the topics will be simulated as queues (work in progress to switch to exchange and keys)
├───docker-files
├───docs
├───k8s
└───src
├───AzureFunction.MassTransit.Demo.Core
│ └───Consumers
├───AzureFunction.MassTransit.Dual.DemoQueue
│ └───Properties
├───AzureFunction.MassTransit.Dual.DemoTopic
│ └───Properties
├───EndpointCreator.MassTransit.Rmq.Demo
│ └───Properties
├───HeroDomain.Contracts
├───Publisher.MassTransit.Demo.Core
└───Publisher.MassTransit.Dual.Demo
└───Properties
The demo is using https://www.myget.org/F/matei-tm/api/v2 feed for the prerelease nugets of MassTransit.WebJobs.Extensions.RabbitMQ.
See the Nuget.Config file.
The Azure functions project files are containing package references for both Asb and RabbitMq, but depending on the configuration value, at build time, they will generate a clean assembly.
The conditional compilation is done by using #if/#else/#endif preprocessor directives
The conditional compilation is done by using #if/#else/#endif preprocessor directives
For the sake of simplicity, the installation tasks will be performed with:
- Chocolatey for Windows OS
- Homebrew for MacOS/Linux OS
Mandatory: An Azure subscription
The current demo can be completed with a minimal subscription. It can be:
- a student version with 100$ credit
- a dev essentials version with 200$ credit
- a pay-as-you-go (depending on the speed of progress, it will charge less than 20$)
Check here for installation instructions.
# open an administrative Power Shell console
choco install azure-cli
brew install azure-cliIf you are using the local installation of the Azure CLI and you are managing several tenants and subscriptions, run the az login command first and add your subscription. See here different authentication methods.
# accessing from localhost
az loginCheck here for installation instructions.
# open an administrative Power Shell console
choco install kubernetes-cli
brew install kubernetes-cliCheck here for installation instructions.
# open an administrative Power Shell console
choco install kubernetes-helm
brew install kubernetes-helmIn order to be able to build the custom images containing disks with Iso files, the docker CLI is needed. Install Docker Desktop on your localbox. Depending on your OS use the proper installation guide:
- Mac https://docs.docker.com/docker-for-mac/install/
- Windows https://docs.docker.com/docker-for-windows/install/
If you want to play with Azure Service Bus, a preliminary setup is required. The following actions are using the Azure CLI. As well you can use the Azure portal to complete them. Finally we will need:
- A service bus namespace
- A queue named OrdersQueue
- A topic named OrdersTopic
- A subscription OrdersSubscription to the OrdersTopic
- A connection string to the service bus
All the resources will be created under a single resource group named k8s. Having everything in one basket will permit to purge all resources in a single step and cut all the subsequent costs. The following command is using westeurope as location. If it's the case, change it according to your own needs.
az group create --location westeurope -n k8s 
Run the following command to create a Service Bus messaging namespace. In order to avoid naming conflicts generated by another player of this demo, replace DemoSb04 with your own name of the Service Bus namespace.
az servicebus namespace create --resource-group k8s --name DemoSb04 --location westeurope
Run the following command to create a queue in the namespace you created in the previous step.
Replace DemoSb04 with your own name of the Service Bus namespace.
az servicebus queue create --resource-group k8s --namespace-name DemoSb04 --name OrdersQueue
Run the following command to create a topic in the namespace you created in the previous step.
Replace DemoSb04 with your own name of the Service Bus namespace.
az servicebus topic create --resource-group k8s --namespace-name DemoSb04 --name OrdersTopic
Run the following command to create a subscription to the topic you created in the previous step.
Replace DemoSb04 with your own name of the Service Bus namespace.
az servicebus topic subscription create --resource-group k8s --namespace-name DemoSb04 --topic-name OrdersTopic --name OrdersSubscription
Run the following command to get the primary connection string for the namespace. You will use this connection string to connect to the queue/topic and send and receive messages. Replace DemoSb04 with your own name of the Service Bus namespace.
az servicebus namespace authorization-rule keys list --resource-group k8s --namespace-name DemoSb04 --name RootManageSharedAccessKey --query primaryConnectionString --output tsv
Note down the connection string and the queue name. You use them to send and receive messages.
If you want to monitor the applications create an App Insights resource and provide the connection information to the configuration files.
az extension add -n application-insights
az monitor app-insights component create --app demoApp --location westeurope --kind web --resource-group k8s --application-type web
# bash/zsh
az monitor app-insights component show --app demoApp --resource-group k8s | grep connection
# bash/zsh
az monitor app-insights component show --app demoApp --resource-group k8s | grep instrumentationKeyClone the repo and switch to that folder
git clone https://github.com/matei-tm/AzureFunction.Demo
cd AzureFunction.Demo
Assure that current kubectl context is pointing to docker desktop
kubectl config use-context docker-desktop
Setting Kubernetes-based Event Driven Autoscaler
helm repo add kedacore https://kedacore.github.io/charts
helm repo update
kubectl create namespace keda
helm install keda kedacore/keda --namespace keda
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml
If the port 8001 is captured by another app, get the app name, verify it and close/kill it.
# Powershell
$theCulpritPort="8001"
Get-NetTCPConnection -LocalPort $theCulpritPort `
| Select-Object -Property "OwningProcess", @{'Name' = 'ProcessName';'Expression'={(Get-Process -Id $_.OwningProcess).Name}} `
| Get-Uniquekubectl proxy
kubectl apply -f k8s/adminuser.yaml
kubectl apply -f k8s/clusterrolebinding.yaml
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
Access to the dashboard is expiring after a period of inactivity. In order to generate a new token, execute the following command in bash terminal
kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"
We need a local docker registry to host the images. Create a local registry
docker run -d -p 5000:5000 --name registry registry:2For the RabbitMq service, a Docker container will be used
docker run -d --hostname my-rabbit --name some-rabbit -p 8080:15672 -p 5672:5672 rabbitmq:3-management
Connection string for Azure Functions will use host.docker.internal (amqp://guest:guest@host.docker.internal:5672)
Connection string for Publisher will use localhost 127.0.0.1 ip (amqp://guest:guest@127.0.0.1:5672)
To access the RabbitMq Dashboard open http://127.0.0.1:8080/ with username:guest and password:guest
Replace the connection strings for ASB/RabbitMq/AppInsights in the following files. Use the values provided in previous steps
.
├── docker-files
├── docs
├── k8s
└── src
├── AzureFunction.MassTransit.Demo.Core
│  └── Consumers
├── AzureFunction.MassTransit.Dual.DemoQueue
│  ├── Properties
│  ├── host.json
│  └── local.settings.json
├── AzureFunction.MassTransit.Dual.DemoTopic
│  ├── Properties
│  ├── host.json
│  └── local.settings.json
├── EndpointCreator.MassTransit.Rmq.Demo
│  ├── Properties
│  ├── appsettings.Development.json
│  ├── appsettings.json
│  └── host.json
├── HeroDomain.Contracts
├── Publisher.MassTransit.Asb.Demo
│  ├── Properties
│  ├── appsettings.Development.json
│  └── appsettings.json
├── Publisher.MassTransit.Demo.Core
└── Publisher.MassTransit.Dual.Demo
├── Properties
├── appsettings.Development.json
├── appsettings.json
└── host.json
Optional, the secrets for publishers can be provided through development safe storage feature.
| Key | Description | Retrieve section |
|---|---|---|
| APPINSIGHTS_INSTRUMENTATIONKEY | App Insights instrumentation key | view |
| APPLICATIONINSIGHTS_CONNECTION_STRING | App Insights connection string | view |
| AzureWebJobsServiceBus | the ASB endpoint for the Azure Function trigger | view |
| ServiceBus | the ASB endpoint for the consumers configured to handle the message received by the trigger. The key name is the default value requested by MassTransit | view |
| Key | Description | Retrieve section |
|---|---|---|
| ApplicationInsights.InstrumentationKey | App Insights instrumentation key | view |
| AppConfig-ServiceBus.ServiceBusConnectionString | the ASB endpoint to publish/send messages | view |
Other keys can stay with the current values
From repo root folder, execute the following commands
# Azure function with ASB endpoint and topic trigger
docker build -t af-masstransit-asb-topic -f docker-files/af-masstransit-asb-topic.Dockerfile .# Azure function with ASB endpoint and queue trigger
docker build -t af-masstransit-asb-queue -f docker-files/af-masstransit-asb-queue.Dockerfile .# Azure function with RabbitMq endpoint and topic trigger
docker build -t af-masstransit-rmq-topic -f docker-files/af-masstransit-rmq-topic.Dockerfile .# Azure function with RabbitMq endpoint and queue trigger
docker build -t af-masstransit-rmq-queue -f docker-files/af-masstransit-rmq-queue.Dockerfile .torq=( topic queue )
asborrmq=( asb rmq )
for i in {0..1}; do
for j in {0..1}; do
docker build -t localhost:5000/af-masstransit-dual-demo/af-masstransit-"${asborrmq[$j]}"-"${torq[$i]}" -f docker-files/af-masstransit-"${asborrmq[$j]}"-"${torq[$i]}".Dockerfile . ;
done;
done;docker image ls -a | grep af-masstransitThe builded images will be pushed to the local registry. Assuming that you built all the images
torq=( topic queue )
asborrmq=( asb rmq )
for i in {0..1}; do
for j in {0..1}; do
docker push localhost:5000/af-masstransit-dual-demo/af-masstransit-"${asborrmq[$j]}"-"${torq[$i]}" ;
done;
done;docker pull localhost:5000/af-masstransit-dual-demo/af-masstransit-rmq-queue
torq=( topic queue )
asborrmq=( asb rmq )
for i in {0..1}; do
for j in {0..1}; do
func kubernetes deploy --name af-masstransit-"${asborrmq[$j]}"-"${torq[$i]}" --image-name host.docker.internal:5000/af-masstransit-dual-demo/af-masstransit-"${asborrmq[$j]}"-"${torq[$i]}" --namespace rabbit ;
done;
done;Depending on your needs, you can use only a single deployment. In that case use one of the following commands
# Use one command at a time
func kubernetes deploy --name af-masstransit-asb-topic --image-name host.docker.internal:5000/af-masstransit-dual-demo/af-masstransit-asb-topic --namespace rabbit
func kubernetes deploy --name af-masstransit-rmq-topic --image-name host.docker.internal:5000/af-masstransit-dual-demo/af-masstransit-rmq-topic --namespace rabbit
func kubernetes deploy --name af-masstransit-asb-queue --image-name host.docker.internal:5000/af-masstransit-dual-demo/af-masstransit-asb-queue --namespace rabbit
func kubernetes deploy --name af-masstransit-rmq-queue --image-name host.docker.internal:5000/af-masstransit-dual-demo/af-masstransit-rmq-queue --namespace rabbit
Being a local, private registry the host.docker.internal must be added to insecure-registries section in Docker config file.
The keys contained by local.settings.json are deployed as secrets into k8s.
Check the stored values to be in sync with the collected values from az cli output. Check File local.settings.json section for more info
Go to http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/secret/rabbit/af-masstransit-asb-topic?namespace=rabbit and verify the following keys:
- APPINSIGHTS_INSTRUMENTATIONKEY
- APPLICATIONINSIGHTS_CONNECTION_STRING
- AzureWebJobsServiceBus - the ASB endpoint for the Azure Function trigger
- ServiceBus - the ASB endpoint for the consumers configured by MassTransit
Go to http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/secret/rabbit/af-masstransit-asb-queue?namespace=rabbit and verify the following keys:
- APPINSIGHTS_INSTRUMENTATIONKEY
- APPLICATIONINSIGHTS_CONNECTION_STRING
- AzureWebJobsServiceBus - the ASB endpoint for the Azure Function trigger
- ServiceBus - the ASB endpoint for the consumers configured by MassTransit
Go to http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/secret/rabbit/af-masstransit-rmq-queue?namespace=rabbit and verify the following keys:
- APPINSIGHTS_INSTRUMENTATIONKEY
- APPLICATIONINSIGHTS_CONNECTION_STRING
Go to http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/secret/rabbit/af-masstransit-rmq-topic?namespace=rabbit and verify the following keys:
- APPINSIGHTS_INSTRUMENTATIONKEY
- APPLICATIONINSIGHTS_CONNECTION_STRING
On the publisher side an environment variable named RABBITMQ_ENABLED is controlling the behaviour
The RABBITMQ_ENABLED environment variable can take two values, true or false. If not set the bus will be configured with Azure Service Bus.
-
With Powershell
# Powershell $env:RABBITMQ_ENABLED = "true"
-
With DOS Command line
rem DOS Command line SET RABBITMQ_ENABLED=true -
With Bash/Zsh
# bash/zsh export RABBITMQ_ENABLED=true
You can build any of Debug/Release/LocalDev configuration
dotnet build src/AzureFunction.Demo.sln --configuration DebugRun the following application in order to create the queues and exchanges needed by the demo.
./src/EndpointCreator.MassTransit.Rmq.Demo/bin/Debug/net6.0/EndpointCreator.MassTransit.Rmq.Demo.exe
Open http://127.0.0.1:8080/#/queues (user:guest, pwd: guest) and verify the queues to be as in the following figure
In a new terminal run the following commands
# bash/zsh (for other terminals change the following line with the proper command from Setting the publisher environment section )
export RABBITMQ_ENABLED=false
cd src/Publisher.MassTransit.Dual.Demo/bin/Debug/net6.0
./Publisher.MassTransit.Dual.Demo.exeSending a message will use the OrdersQueue queue
Check in k8s dashboard on Pods section if a replica af-masstransit-asb-queue-* is launched (the launcher can have a 30 sec delay). Open the pod and check the logs.
Publishing a message will use the OrdersTopic topic
Check in k8s dashboard on Pods section if a replica af-masstransit-asb-topic-* is launched (the launcher can have a 30 sec delay). Open the pod and check the logs.
In a new dos_command/powershell terminal run the following commands
# bash/zsh (for other terminals change the following line with the proper command from Setting the publisher environment section )
export RABBITMQ_ENABLED=true
cd src/Publisher.MassTransit.Dual.Demo/bin/Debug/net6.0
./Publisher.MassTransit.Dual.Demo.exeSending a message will use the OrdersQueue
Check in k8s dashboard on Pods section if a replica af-masstransit-rmq-queue-* is launched (the launcher can have a 30 sec delay). Open the pod and check the logs.
Publishing a message will use the OrdersTopic queue (no topic equivalent in RabbitMq)
Check in k8s dashboard on Pods section if a replica af-masstransit-rmq-topicp is launched (the launcher can have a 30 sec delay). Open the pod and check the logs.
The demo is complete. Use the following hints to clean the local and cloud environments.
If you need to clean the local kubernetes, use the following command
torq=( topic queue )
asborrmq=( asb rmq )
for i in {0..1}; do
for j in {0..1}; do
kubectl delete deploy af-masstransit-"${asborrmq[$j]}"-"${torq[$i]}" --namespace rabbit ;
kubectl delete ScaledObject af-masstransit-"${asborrmq[$j]}"-"${torq[$i]}" --namespace rabbit ;
kubectl delete secret af-masstransit-"${asborrmq[$j]}"-"${torq[$i]}" --namespace rabbit ;
done;
done;If you need to purge all Azure resources, delete the resource group
az group delete -n k8s
Inspired by Chris Patterson's (@phatboyg) Azure Function with MassTransit sample https://github.com/MassTransit/Sample-AzureFunction/tree/master/src/Sample.AzureFunction. Lots of the basic structure of the code is thanks to this. The current work is extending the sample with an implementation for RabbitMq.