Use public IP prefix with AKS load balancer

486 words

2 minutes

Use public IP prefix with AKS load balancer

2024-08-21

/

When you create a service in an AKS instance with a public IP, by default it returns an IP from the out-of-control prefix. By pre-assigning/pre-determining public IP prefix, this helps you control the IP addresses that your own services will use for security purposes.

TL;DR#

Use annotation: service.beta.kubernetes.io/azure-pip-prefix-id: ${ipPrefix_resourceId}
The role “Network Contributor” needs to be assigned to AKS with a scope of public IP prefix resource.
This feature has no relationship with create the cluster with your own public IP or prefixes.

Initializing resources#

Prepare the environment variables for deployment

1
ranNum=$(echo $RANDOM)
2
rG=aks-ipprefix-${ranNum}
3
aks=aks-${ranNum}
4
aksVMSize=Standard_A4_v2
5
ipPrefix=ipprefix-${ranNum}
6
ipPrefixLength=30
7
location=southeastasia

Create resorce group

1
az group create -n ${rG} -l ${location}

Create public IP prefix resource

1
az network public-ip prefix create -n ${ipPrefix} -g ${rG} \
2
--length ${ipPrefixLength}
3

4
ipPrefixId=$(az resource list -g ${rG} \
5
    --resource-type Microsoft.Network/publicIPPrefixes \
6
    --query [0].id -o tsv)

Create AKS cluster and get credentials

1
az aks create -n ${aks} -g ${rG} \
2
    --no-ssh-key \
3
    --node-count 1 \
4
    --node-vm-size ${aksVMSize}
5

6
az aks get-credentials -n ${aks} -g ${rG}

Grant permission to AKS with the scope of public IP prefix resource#

By default, AKS does not have permission to create a public IP from a public IP prefix resource unless the public IP prefix resource is being created in node resource group.
In this demo, the “Network Contributor” role will be assigned to AKS managed identity. To set with custom role, see: Permissions of managing a public IP address prefix.

Get Object (principal) ID of AKS

1
aksIdentityType=$(az aks show -n ${aks} -g ${rG} \
2
--query identity.type -o tsv)
3

4
if [[ "$aksIdentityType" == "SystemAssigned" ]]
5
then
6
aksIdentityID=$(az aks show -n ${aks} -g ${rG} \
7
--query identity.principalId -o tsv)
8
fi
9
if [[ "$aksIdentityType" == "UserAssigned" ]]
10
then
11
aksIdentityID=$(az aks show -n ${aks} -g ${rG} \
12
--query identity.userAssignedIdentities.*.principalId -o tsv)
13
fi
14
if [[ "$aksIdentityType" == "" ]]
15
then
16
aksSPclientID=$(az aks show -n ${aks} -g ${rG} \
17
--query servicePrincipalProfile.clientId -o tsv)
18
aksIdentityID=$(az ad sp show \
19
--id ${aksSPclientID} --query id -o tsv)
20
fi

NOTE
By default, AKS will be created with a system-assigned managed identity, which is used in this demo. The above command will automatically detect which identity you are using and grant permissions accordingly.

Grant permission of managing pubic IP prefix resource to AKS

1
az role assignment create --assignee-object-id ${aksIdentityID} \
2
--assignee-principal-type ServicePrincipal --role "Network Contributor" \
3
--scope ${ipPrefixId}

Deploy services with same public IP prefix#

In this section, we will deploy 4 services and attach public IPs linked to the same public IP prefix resource and test whether the bound public IPs are from the same prefix.

Deploy 4 services with same public IP prefix

1
for i in {1..4}
2
do
3
cat <<EOF | kubectl apply -f -
4
apiVersion: v1
5
kind: Service
6
metadata:
7
  name: test-svc-${i}
8
  labels:
9
    svc: test
10
  annotations:
11
    service.beta.kubernetes.io/azure-pip-prefix-id: |-
12
      ${ipPrefixId}
13
spec:
14
  selector:
15
    app.kubernetes.io/name: FakeApp
16
  ports:
17
    - protocol: TCP
18
      port: 80
19
      targetPort: 80
20
  type: LoadBalancer
21
EOF
22
sleep 5;
23
done

Check if the bound public IPs are from same prefix

1
kubectl get svc -l svc=test \
2
-o custom-columns=NAME:.metadata.name,EXTERNAL-IP:.status.loadBalancer.ingress..ip

Example output:

1
NAME         EXTERNAL-IP
2
test-svc-1   52.1.111.1
3
test-svc-2   52.1.111.2
4
test-svc-3   52.1.111.3
5
test-svc-4   52.1.111.4

Clean resource#

1
az group delete -n ${rG} --no-wait

Use public IP prefix with AKS load balancer

https://fuwari.vercel.app/posts/aks-svc-ipprefix/

Author

Joey Chen

Published at

2024-08-21

License

CC BY-SA 4.0

Convert public AKS cluster to private cluster

Set up default certificate with the Application Routing add-on

1

TL;DR

2

Initializing resources

3

Grant permission to AKS with the scope of public IP prefix resource

4

Deploy services with same public IP prefix

5

Clean resource