486 words
2 minutes
Use public IP prefix with AKS load balancer

When you create a service in an AKS instance with a public IP, by default it returns an IP from the out-of-control prefix. By pre-assigning/pre-determining public IP prefix, this helps you control the IP addresses that your own services will use for security purposes.

TL;DR#

  • Use annotation: service.beta.kubernetes.io/azure-pip-prefix-id: ${ipPrefix_resourceId}
  • The role “Network Contributor” needs to be assigned to AKS with a scope of public IP prefix resource.
  • This feature has no relationship with create the cluster with your own public IP or prefixes.

Initializing resources#

  1. Prepare the environment variables for deployment
Terminal window
ranNum=$(echo $RANDOM)
rG=aks-ipprefix-${ranNum}
aks=aks-${ranNum}
aksVMSize=Standard_A4_v2
ipPrefix=ipprefix-${ranNum}
ipPrefixLength=30
location=southeastasia
  1. Create resorce group
Terminal window
az group create -n ${rG} -l ${location}
  1. Create public IP prefix resource
Terminal window
az network public-ip prefix create -n ${ipPrefix} -g ${rG} \
--length ${ipPrefixLength}
ipPrefixId=$(az resource list -g ${rG} \
--resource-type Microsoft.Network/publicIPPrefixes \
--query [0].id -o tsv)
  1. Create AKS cluster and get credentials
Terminal window
az aks create -n ${aks} -g ${rG} \
--no-ssh-key \
--node-count 1 \
--node-vm-size ${aksVMSize}
az aks get-credentials -n ${aks} -g ${rG}

Grant permission to AKS with the scope of public IP prefix resource#

By default, AKS does not have permission to create a public IP from a public IP prefix resource unless the public IP prefix resource is being created in node resource group.
In this demo, the “Network Contributor” role will be assigned to AKS managed identity. To set with custom role, see: Permissions of managing a public IP address prefix.

  1. Get Object (principal) ID of AKS
Terminal window
aksIdentityType=$(az aks show -n ${aks} -g ${rG} \
--query identity.type -o tsv)
if [[ "$aksIdentityType" == "SystemAssigned" ]]
then
aksIdentityID=$(az aks show -n ${aks} -g ${rG} \
--query identity.principalId -o tsv)
fi
if [[ "$aksIdentityType" == "UserAssigned" ]]
then
aksIdentityID=$(az aks show -n ${aks} -g ${rG} \
--query identity.userAssignedIdentities.*.principalId -o tsv)
fi
if [[ "$aksIdentityType" == "" ]]
then
aksSPclientID=$(az aks show -n ${aks} -g ${rG} \
--query servicePrincipalProfile.clientId -o tsv)
aksIdentityID=$(az ad sp show \
--id ${aksSPclientID} --query id -o tsv)
fi
NOTE

By default, AKS will be created with a system-assigned managed identity, which is used in this demo. The above command will automatically detect which identity you are using and grant permissions accordingly.

  1. Grant permission of managing pubic IP prefix resource to AKS
Terminal window
az role assignment create --assignee-object-id ${aksIdentityID} \
--assignee-principal-type ServicePrincipal --role "Network Contributor" \
--scope ${ipPrefixId}

Deploy services with same public IP prefix#

In this section, we will deploy 4 services and attach public IPs linked to the same public IP prefix resource and test whether the bound public IPs are from the same prefix.

  1. Deploy 4 services with same public IP prefix
Terminal window
for i in {1..4}
do
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
name: test-svc-${i}
labels:
svc: test
annotations:
service.beta.kubernetes.io/azure-pip-prefix-id: |-
${ipPrefixId}
spec:
selector:
app.kubernetes.io/name: FakeApp
ports:
- protocol: TCP
port: 80
targetPort: 80
type: LoadBalancer
EOF
sleep 5;
done
  1. Check if the bound public IPs are from same prefix
Terminal window
kubectl get svc -l svc=test \
-o custom-columns=NAME:.metadata.name,EXTERNAL-IP:.status.loadBalancer.ingress..ip

Example output:

NAME EXTERNAL-IP
test-svc-1 52.1.111.1
test-svc-2 52.1.111.2
test-svc-3 52.1.111.3
test-svc-4 52.1.111.4

Clean resource#

Terminal window
az group delete -n ${rG} --no-wait
Use public IP prefix with AKS load balancer
https://fuwari.vercel.app/posts/aks-svc-ipprefix/
Author
Joey Chen
Published at
2024-08-21
License
CC BY-SA 4.0