Deploy Azure OpenAI using Terraform with Private Endpoint

With more looking into using ChatGPT for use within the enterprise, many are looking towards deploying Azure OpenAI since it provides access to a self-hosted API instance, within a specified region and the ability to manage the network communication to be using private network access. Meaning that access to GPT is only via a managed network and not publicly available, as it is with ChatGPT if you are using those APIs.

This blog post will cover how to deploy Azure OpenAI with Terraform using private endpoints and therefore locking down access only to a managed network in Azure. This means that even if someone manages to get a hold of your API key for GPT in Azure, they will not be able to use it unless they have access to your network as well.

It should be noted that if you plan to deploy Azure OpenAI services, there are still some limitations on where the service is available, as seen in the screenshot below. You can find the updated list of LLMs and regions available here –> Azure OpenAI Service models – Azure OpenAI | Microsoft Learn

So let us assume that we would want to deploy a self-hosted Azure OpenAI instance in a separate VNET with a Private Endpoint configuration the Terraform code would look something like this, using a simplified setup with everything contained within the same RG. This first section would deploy the Azure OpenAI instance with an LLM deployment with GPT-35-turbo. The model number versions can be found in the same article mentioned above. Also, the model numbers will change when OpenAI releases new models and Microsoft adds them to the service.

NOTE: This section also sets public network access enabled = false, meaning that you will only be able to reach this service via the private endpoint and not the public address of the service. Hence the deployment needs to be run from a self-hosted or dedicated runner that has access to the internal network.
NOTE2: Remember to add a custom subdomain if you want this to work. Change it to something else.

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "West Europe"
}

resource "azurerm_cognitive_account" "example" {
  name                = "example-ca"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  kind                = "OpenAI"
  sku_name            = "S0"
  public_network_access_enabled = false
  custom_subdomain_name = "example-openai2"
}

resource "azurerm_cognitive_deployment" "example" {
  name                 = "example-cd"
  cognitive_account_id = azurerm_cognitive_account.example.id
  model {
    format  = "OpenAI"
    name    = "gpt-35-turbo"
    version = "0301"
  }

  scale {
    type = "Standard"
  }
}

This next section is for deploying the network configuration. This also contains DNS and the Private Endpoint configuration.

resource "azurerm_virtual_network" "example-vnet" {
  name                = "vnet01"
  address_space       = ["10.0.0.0/24"]
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
}

resource "azurerm_subnet" "example-subnet" {
  name                 = "snet01"
  resource_group_name  = azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example-vnet.name
  address_prefixes     = ["10.0.0.0/25"]
}

resource "azurerm_private_endpoint" "example-pe01" {
  name                = "pe-openai-we"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  subnet_id           = azurerm_subnet.example-subnet.id


  private_service_connection {
    name                           = "pe-openai-we"
    private_connection_resource_id = azurerm_cognitive_account.example.id
    subresource_names              = ["account"]
    is_manual_connection           = false
  }

      private_dns_zone_group {
    name                 = "default"
    private_dns_zone_ids = [azurerm_private_dns_zone.example.id]
  }
}

resource "azurerm_private_dns_zone" "example" {
  name                = "privatelink.openai.azure.com"
  resource_group_name = azurerm_resource_group.example.name
}

resource "azurerm_private_dns_zone_virtual_network_link" "example" {
  name                  = "example-vnet-link"
  resource_group_name   = azurerm_resource_group.example.name
  private_dns_zone_name = azurerm_private_dns_zone.example.name
  virtual_network_id    = azurerm_virtual_network.example-vnet.id
}

This script just sets up a predefined virtual network with a subnet, private endpoint and a cognitive service running Azure OpenAI. Meaning a secure way to access Azure OpenAI to build your applications using LLMs.