Table of contents

• SAP HANA VM Deployments
• Deployment Framework
• HANA VM Sizes and Storage Configurations
• Prerequesites for SAP HANA Deployments
• Deploy the landing zone
• Deploy SAP HANA VMs
• HANA Cloud Measurement Test Results
• Quality checks
• SAP App VM Deployment
• Troubleshooting
• FAQ
• Disclaimer

SAP HANA VM Deployments

**This repository is used to deploy SAP HANA Databases 2.0 with Azure DevOps including the following options: **

• SLES 12 & 15
• RHEL 7 & 8
• VM sizes from 128GB to 12TB
• Optional double disk encryption at rest (platform and customer managed keys)
• Deployment wih Azure or own custom images
• OS Preparation with required patches and configurations according to relevant SAP notes
• HANA 2.0 Installation
• Backup Integration into an Azure Recovery Service Vault including optional execution of initial OS & HANA backups
• Selective disk backup, which excludes hana log & data disks from the OS backups
• Setup Azure Monitoring Extension for SAP
• Execution of HANA Cloud Measurement Tool (HCMT)
• Execution quality checks (WIP)
• Removal of the complete deployment

Deployment Framework

The DevOps Pipeline is used as a GUI to simplify deployments. It fetches the pipeline from the GitHub repository. The GitHub repository itself can be most easily adapted to your landing zone specifics with Visual Studio Code on your local PC. In the grey rectangle we see the Ubuntu VM which acts as deployment agent and the require Azure ressources like VNET, Recovery Service Vault, Storage, etc.

HANA VM Sizes and Storage Configurations

Size	HANA VM	HANA VM Storage (EXE + DATA + LOG + SHARE + BACKUP)
128_GB	E16ds_v4	1xP6(64GB) + 3xP6(64GB) + 3xP10(128GB) + 1xP20(512GB) + 1xP20(512GB)
160_GB	E20ds_v4	1xP6(64GB) + 4xP6(64GB) + 3xP10(128GB) + 1xP20(512GB) + 1xP20(512GB)
192_GB	M32ts	1xP6(64GB) + 4xP6(64GB) + 3xP10(128GB) + 1xP20(512GB) + 1xP20(512GB)
256_GB	M32ls	1xP6(64GB) + 4xP6(64GB) + 3xP10(128GB) + 1xP20(512GB) + 1xP20(512GB)
384_GB	E48ds_v4	1xP6(64GB) + 3xP15(256GB) + 3xP10(128GB) + 1xP20(512GB) + 1xP20(512GB)
512_GB	M64ls	1xP6(64GB) + 4xP10(128GB) + 3xP10(128GB) + 1xP20(512GB) + 1xP20(512GB)
875_GB	M64ls	1xP6(64GB) + 4xP15(256GB) + 3xP10(128GB) + 1xP20(512GB) + 1xP20(512GB)
1.000_GB	M64ds_v2	1xP6(64GB) + 4xP15(256GB) + 3xP15(256GB) + 1xP30(1TB) + 1xP30(1TB)
1.792_GB	M64dms_v2	1xP6(64GB) + 4xP20(512GB) + 3xP15(256GB) + 1xP30(1TB) + 1xP30(1TB)
2.000_GB	M128ds_v2	1xP10(128GB) + 4xP20(512GB) + 3xP15(256GB) + 1xP30(1TB) + 1xP30(1TB)
2.850_GB	M208s_v2	1xP10(128GB) + 4xP30(1024GB) + 3xP15(256GB) + 1xP30(1TB) + 1xP30(1TB)
3.892_GB	M128dms_v2	1xP10(128GB) + 5xP30(1024GB) + 3xP15(256GB) + 1xP30(1TB) + 1xP30(1TB)
5.700_GB	M208ms_v2	1xP10(128GB) + 4xP40(2048GB) + 3xP15(256GB) + 1xP30(1TB) + 1xP30(1TB)
11.400_GB	M416ms_v2	1xP10(128GB) + 4xP50(4096GB) + 3xP15(256GB) + 1xP30(1TB) + 1xP30(1TB)

Note: Eds_v4 Series use premium disk without write accellerations, therefore this is recommended for Non-PRD envrionments only

Deploy only a HANA VM and Storage via ARM

Use this button for VM and storage deployment only option via an ARM template. For the full capabitilities of this repository continue below path for the DevOps deployments.

Note: Required target Subnet ID can be retrieved in cloud shell via:
az network vnet subnet list -g [ResourceGroup] --vnet-name [Name] --query [].id

Example: az network vnet subnet list -g saponazuregermanywestcentral --vnet-name vnet-sap-germanywestcentral-004 --query [].id

Prerequesites for SAP HANA Deployments

1. Azure Subscription
2. A service principle ID including the secret with contributor rights on the subscription
3. Azure DevOps and Github account
4. SAP User for the Software Downloads
5. An existing landing zone with basic resource or alterntively deploy a landing zone with the included pipeline and ARM templates
6. An ssh public and private key pair. 'ssh-keygen -f mykeypair -t rsa -b 4096'

Deploy the landing zone

1. Create a Project in Azure DevOps

2. Import this Github repository https://github.com/mimergel/sap-hana-vm.git

   

   

3. Add following extensions to your DevOps Project

   • Ansible Extension
   • Post Build Cleanup
     
   
   

4. Create the Pipeline for the landing zone

   • In the DevOps Pipeline Area
   • Create a "New Pipeline"
   • Where is your code? => "Azure Repos Git"
   • Select a repository => "sap-hana-vm"
   • Configure your pipeline => "Existing Azure Pipeline YAML file"
   • Branch "Main" (or Beta)
   • Path "/DevOpsPipeline/sap-landing-zone.yaml"
   • Continue and Click on the right side of the Run button to "Save"
   • Optionally change the name in the Pipeline overview
   
   

   The landing zone includes following resources:

   • VNET + Subnets + NSGs
   • Recovery Service Vault with policies for HANA & OS backups
   • Storage accounts (For SAP binaries, Scripts & Boot Diagnostics)
   • Bastion Host
   • An ubuntu VM that will act as DevOps deployment agent
   • Windows Admin VM (For HANA Studio, SAPGui, Easy SAPBits Upload to storage account, etc.)
   • Keyvault
   • Disk encryption set
   
   

5. Create the required variable group

   In the Pipeline section under Library create the following variable group SAP-deployments

   Variables:

   * adminuser                  azureadm
   * advice.detachedHead        false
   * Agent                      [Agent Pool Name]
   * ARM_CLIENT_ID              [SPN ID]
   * ARM_CLIENT_SECRET          [SPN secret]
   * ARM_SUBSCRIPTION_ID        [subscription id]
   * ARM_TENANT_ID              [tenant id]
   * AZURE_CONNECTION_NAME      [azure connection name as defined in devops service connections]]
   * diagnosticsstorageaccount  [name of diagnostics storage account]
   * hana-pw                    [password for the hana db]
   * privatednszone             [e.g. sap.contoso.net]
   * rsv                        [recovery service vault, e.g. rsv-sap-germanywestcentral-004]
   * S-Username                 [S-Username]
   * S-Password                 [S-User password]
   * sap-pw                     [password for sap login, not yet used]
   * pubsshkey                  [public key]
   * skipComponentGovernanceDetection true
   * url-disk-cfg               [url to diskconfig.sh script]
   * 
   

   Example:
   

   
   

   Add pipeline permissions:
   

   
   
   

   
   

   Tip: In case you plan to deploy into differen landing zones / regions / subscription 
   you might want to create a separate variable group with variables specific to the landing zone
   

   Example: Variable group SAP-deployments-germany
   

   
   

6. Deploy the landing zone

   • Press "Run Pipeline", enter required parameters and "Run"

   

   
   

7. Finalize the Deployment Agent Setup

   • Login with your ssh user to the linux vm which will serve as deployment agent, then

     cd devopsagent ; ./config.sh

   • Follow the prompts and enter required information, have the PAT (personal access token) from DevOps ready.

     See here where to retrieve the PAT

   • Script responses as follows:

     

   • Ensure the deployment agent software is automatically started as a service after each reboot:
     

     sudo ./svc.sh install ; sudo ./svc.sh start

   • Save your private ssh-key in ~/.ssh/id_rsa (ensure 600 file permission). This ensures possible login from the deployment agent to the HANA VM which is required for Ansible activities.

Deploy SAP HANA VMs

1. In the DevOps Pipeline Area

   • Create a "New Pipeline"
   • Where is your code? => "Azure Repos Git"
   • Select a repository => "sap-hana-vm"
   • Configure your pipeline => "Existing Azure Pipeline YAML file"
   • Branch "Main" (or Beta)
   • Path "/DevOpsPipeline/sap-hana-vm-arm.yaml"
   • Continue and Click on the right side of the Run button to "Save"
   • Optionally change the name in the Pipeline overview

2. In case the target networks don't have access to the internet

   • Upload diskConfig.sh in the storage container and adapt variables url-disk-cfg in the pipeline variables
   • Upload msawb-plugin-config-com-sap-hana.sh to the container and adapt variable url_msawb_plugin in Ansible/vars/defaults.yml

3. Adapt VNET, Subnet and other parameters in the pipeline to match your landing zone situation

4. Create an azure resource manager service connection with the service principal in project settings

5. Run the Pipeline

   Now you're ready to deploy the SAP HANA VM including subsequent tasks.
   

   • Run the pipeline
     
     

   • Provide inputs as required and press "Run"
     
     

HANA Cloud Measurement Test Results

• The tests run a couple of hours. Once the execution is completed it will create a file here: [hanavm]:/hana/shared/install/setup/hcmtresult-<timestamp>.zip

• You need to upload the results file on a SAP web site to check if the systems meet the configuration and performance requirements. Upload link: https://hotui-supportportal.dispatcher.hana.ondemand.com/index.html

• More information on HCMT in this blog
  

  Example
  

Quality checks

• Note: implementation WIP

• Once the execution is completed it will create a html file with all results in the Inventory folder: quality-checks.html

  Example
  

SAP App VM Deployment

Use this ARM template to deploy the SAP Application VMs. Automated SAP Installation and deployment via an Azure DevOps Pipeline functionality will be added soon.

Azure DevOps Agent

Use this ARM template to deploy the Azure DevOps Agent only.

Troubleshooting

• ARM deployment fails because the URL to the diskConfig.sh Script is not reachable from the deployed VM. In this case login to the VM and try with wget to download the script. Use your own container in your storage account and ensure it's reachable from VMs in the target subnet
• During Stage "Prepare_OS" ssh connection must work from the deployment agent to the HANA VM. In case of troubles try to connect from the agent maually via ssh and solve the issue. Connection must work without interactive ssh prompts. You might need to set StrictHostKeyChecking no in ~/.ssh/config when deploying VMs with different names to the same IP
• HANA Installation fails when using forbidden SID: ADD, ALL, AMD, AND, ANY, ARE, ASC, AUX, AVG, BIT, CDC, COM, CON, DBA, END, EPS, FOR, GET, GID, IBM, INT, KEY, LOG, LPT, MAP, MAX, MIN, MON, NIX, NOT, NUL, OFF, OLD, OMS, OUT, PAD, PRN, RAW, REF, ROW, SAP, SET, SGA, SHG, SID, SQL, SUM, SYS, TMP, TOP, UID, USE, USR, VAR
• The pipeline fails in step "Prepare OS
  • ~/.ssh/id_rsa file is missing or has wrong permissions.
  • Add host_key_checking = False to the ansible configuration file /etc/ansible/ansible.cfg. This will prevent ssh prompts during first logins.
• Ansible stages do not run
  • Try to run the Ansible command on the deployment agent manually
  • Therefore clone the code on the deployment agent:
    • git clone https://github.com/<your-git-user-id>/sap-hana-vm.git
    • Add the fqdn of the VM into the file /etc/ansible/hosts
    • cd sap-hana-vm
    • Execute: ansible-playbook -vvvv Ansible/os-settings_playbook.yml
    • Analyse the now more detailed debugging information due to option "-vvvv"
• Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user
  • On the deployment agent set allow_world_readable_tmpfiles = True in /etc/ansible/ansible.cfg Make sure to have folling content in /etc/ansible/ansible.cfg

   [defaults]
   allow_world_readable_tmpfiles = True
   host_key_checking = False
  

• Backup or SAP Monitoring scripts fails
  • Perform az login on the ubuntu deployment agent
• Deployment of the devopsdeployer VM fails in the last step: message: "VM has reported a failure when processing extension 'CustomScript' ..."
  • The VM might not have access to the internet and therefore cannot download the custom script that handles to download of ansible, azure cli, etc.
  • Solution: Adapt the FW and grant temporarily (e.g. for 5 minutes) internet access for the VM. Run the custom script manually. Steps:
    • wget https://raw.githubusercontent.com/mimergel/sap-hana-vm/main/Scripts/setup-deployment-agent.sh
    • chmod 755 setup-deployment-agent.sh
    • sudo ./setup-deployment-agent.sh
• The Self-hosted DevOps deployment agent is not able to connect to Azure DevOps
  • Make sure the deployment agent VM is able to connect to Azure DevOps, most likely a FW rule is required. See here for details.
• The value of parameter linuxConfiguration.ssh.publicKeys.keyData is invalid
  • The key is incorrect. Make sure the pubsshkey variable is correct:
    
• Failed to connect to the host via ssh: key_load_public: invalid format or other ssh connectivity issues
  • Use the same adminuser name on deployer and HANA VM
• fatal: [10.10.10.4]: UNREACHABLE!
  • add your ssh private key to ~/.ssh/id_rsa and ensure correct file permission 600.
• fatal: [10.10.10.4]: FAILED! => {"msg": "Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user ....
  • WORLD_READABLE_TMPFILES not set to True in ansible.cfg file on deployer or higher ansible used where this flag is not supported any more
• InvalidParameter: Destination path for SSH public keys is currently limited to its default value /home/$(adminuser)/.ssh/authorized_keys due to a known issue in Linux provisioning agent.
  • Make sure the adminuser variable is set and corresponds to the adminuser during initial deployment of the HANA VM
• Deployment template validation failed: 'The value for the template parameter 'adminPasswordOrKey' at line '43' and column '29' is not provided.
  • Please provide your own public ssh key that has been created upfront. The ARM template doesn't handle an option "Generate new key pair"

FAQ

• Where is the HCMT result?
  • [hanavm]:/hana/shared/install/setup/hcmtresult-<timestamp>.zip
  • Upload the test results here
  • More information on HCMT here
• How do I create the service principle?
  • Via CLI: https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli
  • Via Portal: https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal

Disclaimer

THIS REPOSITORY AND ALL IT'S CONTENT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.