Deploy in Azure VNet

This page provides detailed information on how to deploy the Azure stacks into your private network using Azure Virtual Network (VNet). By following the steps outlined in this guide, you will be able to ensure that all of your Azure stack resources are securely deployed in a private network environment.

Introduction

To ensure the highest level of security for your resources, we have implemented several measures tailored to different components of your system:

• Function App: We have closed all inbound traffic and enabled VNet integration for outbound traffic. This means that the Function App can only communicate with resources that are part of your virtual network, preventing unauthorized access from external sources.
• Service Bus and Storage Account: To further enhance security, we will close most of the public endpoints and create Private Endpoints for Service Bus and Storage Account. Private Endpoints allow access from your private network only, and prevent any external access to your resources.
• Application Insights and Log Analytics workspace: Our deployment includes a parameter that lets you control public network access to the Application Insights and Log Analytics workspaces. If you disable public network access, we strongly recommend adding those resources to Azure Monitor Private Link Scopes (AMPLS). This ensures that logs are only ingested and queried from authorized sources, protecting your data from unauthorized access.

It's important to note that stacks with VNet cannot be updated to stacks without VNet, and vice versa. This means that if you deploy a stack with VNet, you cannot remove the VNet configuration in a subsequent update. Similarly, if you deploy a stack without VNet, you cannot add VNet configuration in a subsequent update. To change whether to use VNet in the stack, you need to delete the stack and deploy it again.

Prerequisites

Before you can deploy the Azure Stacks system in your private network using Azure Virtual Network (VNet), you must first complete several prerequisite steps. These steps are essential to ensuring that your Azure Stacks resources are deployed securely in your private network environment.

Supported Azure Regions

Our service is currently supported in several Azure regions, including the regions where the resources such as VNet, Private Link, and Private DNS Zone are supported, as indicated in Azure's Products available by region documentation.

VNet & Subnets

Before deploying the Azure Stacks, you need to create a VNet with the following four subnets:

1. Scanner Function App Subnet
2. Blob Listener Function App Subnet
3. Post Scan Action Tag Function App Subnet
4. Private Endpoints Subnet

The first three subnets use the Azure Function App feature - VNet integration, which allows the Azure Stacks functions to integrate with the VNet. To accommodate the expected scale-out worker number, each subnet requires a sufficient number of available IP addresses. For instance, each Function App worker requires 2 IP addresses, so the available IP number should be set to twice the maximum scale-out number. Moreover, according to Integrating your app with an Azure virtual network, it is recommended to reserve at least a /28 subnet, which provides a minimum of 16 available IP addresses.

For the Private Endpoints Subnet, we recommend a minimum of 9 available IP addresses at a basic all-in-one stack. If you have multiple storage stacks, each storage stack requires an additional 5 IP addresses. Therefore, the available IP number needs to be greater than 9, plus the number of additional IP addresses required for any additional storage stacks.

Additionally, Azure automatically reserves the first and last four IP addresses in every subnet, leaving a total of five reserved IP addresses. Therefore, you need to allocate an extra five IP addresses for each subnet. To learn more about this, please refer to Azure Virtual Network frequently asked questions (FAQ).

Please follow the table below to set up the subnets:

The resources in the Azure Stacks need to communicate with each other and the protected storage account, so make sure that the subnets you create are able to access each other.

Azure restricts Function App integration only to the VNet deployed in the same region. Therefore, ensure that the Azure Stacks are deployed within the same region as the VNet.

To create the subnets for function app via the Azure CLI, use the following command:

az network vnet subnet create \
  --name <function-app-subnet-name> \
  --resource-group <application-resource-group-name> \
  --vnet-name <vnet-name> \
  --address-prefixes <address-prefixes> \
  --delegations Microsoft.Web/serverFarms

To create the subnet for private endpoint via the Azure CLI, use the following command:

az network vnet subnet create \
  --name <private-endpoint-subnet-name> \
  --resource-group <application-resource-group-name> \
  --vnet-name <vnet-name> \
  --address-prefixes <address-prefixes>

Protected Storage Account

To enable the Azure Stack to access a storage account that only allows private network access, you must allow access from the stack's network through either a Private Endpoint or a Service Endpoint.

Private Endpoint is a more secure option that connects your virtual network to a private link service using a network interface. This allows the storage account to be accessed through an IP address within your virtual network. On the other hand, Service Endpoint allows traffic from a specific subnet to access the storage account without going through the public internet.

We recommend using Private Endpoint for enhanced security, especially if your storage account only accepts private connections or if there is a firewall blocking public IPs. Our stacks will access the protected storage account from Scanner Function App Subnet, Blob Listener Function App Subnet, and Post Scan Action Tag Function App Subnet mentioned in the previous section. Please ensure these subnets can access the protected storage account, or the scan will fail.

The event grid system topic does not support VNet. To subscribe to blob events, you will need to bypass traffic for trusted Azure services using the following Azure CLI command:

az storage account update \
  --resource-group <resource_group_name> \
  --name <protected_storage_account_name> \
  --bypass AzureServices

Private DNS Zones

To ensure that the private endpoint is able to connect to the correct resources, the DNS settings must be configured for the private DNS zone correctly.

To prepare the private DNS zone, you need to create the zone with the specific name and link to the target VNet. The records that map the resource endpoint to the private IP address of the corresponding private endpoint are automatically created during the Azure Stack deployment process. You can refer to Azure Private Endpoint DNS configuration for more information on how to configure private DNS zones.

The following Private DNS Zones are required for the protected resources:

• Storage account blob: privatelink.blob.core.windows.net
• Storage account file: privatelink.file.core.windows.net
• Storage account table: privatelink.table.core.windows.net
• Service bus: privatelink.servicebus.windows.net

To create a Private DNS Zone via the Azure CLI, use the following command:

az network private-dns zone create \
  --resource-group <platform-resource-group-name> \
  --name privatelink.blob.core.windows.net

az network private-dns zone create \
  --resource-group <platform-resource-group-name> \
  --name privatelink.file.core.windows.net

az network private-dns zone create \
  --resource-group <platform-resource-group-name> \
  --name privatelink.table.core.windows.net

az network private-dns zone create \
  --resource-group <platform-resource-group-name> \
  --name privatelink.servicebus.windows.net

To link the Private DNS Zone with VNet via the Azure CLI, use the following command:

az network private-dns link vnet create \
  --registration-enabled false \
  --name <link-name> \
  --resource-group <platform-resource-group-name> \
  --virtual-network <target-vnet-resource-id> \
  --zone-name privatelink.blob.core.windows.net

az network private-dns link vnet create \
  --registration-enabled false \
  --name <link-name> \
  --resource-group <platform-resource-group-name> \
  --virtual-network <target-vnet-resource-id> \
  --zone-name privatelink.file.core.windows.net

az network private-dns link vnet create \
  --registration-enabled false \
  --name <link-name> \
  --resource-group <platform-resource-group-name> \
  --virtual-network <target-vnet-resource-id> \
  --zone-name privatelink.table.core.windows.net

az network private-dns link vnet create \
  --registration-enabled false \
  --name <link-name> \
  --resource-group <platform-resource-group-name> \
  --virtual-network <target-vnet-resource-id> \
  --zone-name privatelink.servicebus.windows.net

Azure Monitor Private Link Scopes (optional)

Azure Monitor Private Link Scopes (AMPLS) is a service that enables you to securely access Azure Monitor logs and metrics from within your private network using a private endpoint. This means that your data is protected and doesn't traverse the public internet.

To access the Azure Stacks' application insights logs data from your private network, you need to create an Azure Monitor Private Link Scope and its corresponding private endpoint to connect to the target VNet. You can refer to the article Configure your private link for detailed instructions on setting up the AMPLS. Once the deployment of the Azure Stacks is completed, you can add the application insights resources to the AMPLS, which ensures that the log data is only accessible from within the private network.

If your VNet is linked to an Azure Monitor Private Link Scope, it is important to add the corresponding Application Insights resources to the scope to avoid losing logs. Failure to add the necessary resources to the scope can result in loss of logs due to Azure's network situation.

Deploy the Azure Stacks

To deploy the Azure Stacks into VNet, you need to fulfill the template parameters with the resources created in the previous section. Here are the nine parameters required:

Ensure you have the values for each parameter before you proceed with the deployment.

Configure the Azure Stacks

Once you have deployed the Azure Stacks, you need to configure them before they can start protecting your data. There are two key tasks that are required:

Add Azure Stacks's Application Insight to Azure Monitor Private Link Scopes

If your log includes the public network, means the deployment parameter Restricted Access For Application Insight is false, please ignore this section.

To secure access to Application Insights, you can add it to Azure Monitor Private Link Scopes. This restricts access to the Application Insights logs to only those who have access to the Private Link Scopes. To add Application Insights to the Private Link Scopes:

1. In the Azure portal, navigate to your Azure Monitor Private Link Scopes.
2. In the "Configure" section, click "Azure Monitor Resources".
3. Click "Add".
4. Choose the Stacks Application Insights.
5. Click "Apply".

Alternatively, you can use the following command:

az monitor private-link-scope scoped-resource create \
  --linked-resource <TARGET_APPLICATION_INSIGHT_RESOURCE_ID> \
  --name <SCOPED_RESOURCE_NAME> \
  --resource-group <AMPLS_RESOURCE_GROUP> \
  --scope-name <AMPLS_NAME>

To ensure that the stacks' logs are accessed from the private network, you need to add at least 3 Application Insights resources to the Azure Monitor Private Link Scope. For the scanner stack, you can find the Application Insights resource ID on the deployment output as ScannerApplicationInsightResourceID. For the storage stack, you can find the Application Insights resource IDs from the deployment output as BlobListenerApplicationInsightsResourceID and ActionTagApplicationInsightsResourceID. You can refer to the Connect Azure Monitor resources document for more information on how to do this.

Before you can remove the Azure Stacks, you must first remove the association between Azure Monitor Private Link Scope (AMPLS) and Application Insights. Failing to do so can lead to errors during the removal process. To remove the association, you can delete the Application Insights resource from the scope either through the Azure portal or the Azure CLI.

Firewall (optional)

Typically, the VNet traffic is protected by the Azure Firewall or Network Security Group (NSG). We recommend using Azure Firewall to control the traffic since NSG does not support for FQDN filtering. It's important to ensure that appropriate firewall settings are in place to ensure that Azure Stack runs smoothly.

Controlling traffic by Fully Qualified Domain Names (FQDNs) is supported by two types of firewall rules: Network Rules and Application Rules. If you plan to use Network Rules to achieve this, you need to enable DNS proxy first. However, Azure recommends using Application Rules for HTTP/S traffic, as they are more suitable for this purpose. To learn more about the differences between Application Rules and Network Rules for FQDN filtering, refer to this resource on Azure's website: Use FQDN filtering in network rules.

Communication with a third party service

To allow communication with third party service, you need to add the following application rules to the firewall's allow list:

Set up for a Scanner Stack's KeyVaults

In the scanner stack, there are two KeyVaults that remain in the public network. If the scanner function app cannot access these KeyVaults, the scan will fail. Therefore, you need to add the KeyVaults to a whitelist.

You can set up the whitelist target FQDNs from application rule, which you can find in the scanner deployment output ScannerLicenseKeyVaultURI and ScannerConfigKeyVaultURI.

Alternatively, you can set up the whitelist service tag from network rule, which is a simpler approach but allows traffic to all target KeyVaults.

Set up for Application Insight

If you are not using Azure Monitor Private Link Services (AMPLS), you will need to add the following firewall rule for Application Insights. The destination FQDN values can be found in the Azure Private Endpoint DNS configuration documentation and IP addresses used by Azure Monitor.

We strongly recommend enabling firewall logs to help troubleshoot any network issues that may arise. Please refer to the Monitor Azure Firewall logs and metrics documentation for the firewall settings.

It is important to ensure that all subnets for the Azure Stacks can communicate with each other.

Conclusion

We have implemented several security measures to enhance the protection of your resources. We have closed the public endpoints for your Service Account and Service Bus, and created Private Endpoints to allow access from your private network only. For the Function App, we have closed all inbound traffic and enabled VNet integration for outbound traffic. Additionally, for Application Insights, we have integrated with Azure Monitor Private Link Scopes (AMPLS) to ensure that logs can only be ingested and queried from authorized sources.