Check what external resources is used by current AKS security principal#

In some scenarios, you may want to use external resources, such as: public IP resources, storage accounts outside of the Node Resource Group (NRG), or custom VNet for AKS. Assigning roles to the new managed identity with the same permissions on these resources is required to ensure that AKS can continue to use them after changing the security principal.

1. Get current AKS security principal object ID

1
aksIdentityType=$(az aks show -n ${aks} -g ${rG} \
2
--query identity.type -o tsv)
3

4
if [[ "$aksIdentityType" == "SystemAssigned" ]]
5
then
6
aksIdentityId=$(az aks show -n ${aks} -g ${rG} \
7
--query identity.principalId -o tsv)
8
fi
9
if [[ "$aksIdentityType" == "UserAssigned" ]]
10
then
11
aksIdentityId=$(az aks show -n ${aks} -g ${rG} \
12
--query identity.userAssignedIdentities.*.principalId -o tsv)
13
fi
14
if [[ "$aksIdentityType" == "" ]]
15
then
16
aksSPclientId=$(az aks show -n ${aks} -g ${rG} \
17
--query servicePrincipalProfile.clientId -o tsv)
18
aksIdentityId=$(az ad sp show \
19
--id ${aksSPclientId} --query id -o tsv)
20
fi
21

22
oldaksIdentityId=${aksIdentityId}

2. Check role assignments for current AKS security principal, excluding the scope of the AKS Node Resource Group (NRG)

1
aksNrgId=$(az group show --query id -o tsv \
2
-n $(az aks show -n ${aks} -g ${rG} \
3
--query nodeResourceGroup -o tsv))
4

5
assignmentList=$(az role assignment list --all --query \
6
"[?principalId=='${aksIdentityId}'&&"'!'"starts_with(scope,'${aksNrgId}')]. \
7
{roleId:roleDefinitionId,name:roleDefinitionName,scope:scope}")
8

9
echo ${assignmentList}

If there is no output, we can assume that AKS currently has no control on resources outside of the Node Resource Group (NRG).

NOTE

When changing the security identity used by AKS, it automatically grants permissions to the new managed identity on Node Resource Group (NRG). So you don’t need to consider permissions on resources in the Node Resource Group (NRG).

Preparing new user-assigned managed identity#

As mentioned above, to convert AKS cluster into private cluster, it must use user-assigned managed identity. Therefore, we will create a new user-assigned managed identity here and ensure that it inherits permissions from the current AKS security principal, excluding the scope of the Node Resource Group (NRG).

1. Create user-assigned managed identity

1
uami=${aks}-newIdentity
2
az identity create -n ${uami} -g ${rG} -o none
3
uamiId=$(az resource list -n ${uami} -g ${rG} \
4
--resource-type Microsoft.ManagedIdentity/userAssignedIdentities \
5
--query [0].id -o tsv)
6
uamiIdentityId=$(az identity show --ids $uamiId \
7
--query principalId -o tsv)

2. Have the new managed identity inherit permissions from current AKS security principal (Conditional skip)

NOTE

If the current AKS security principal has no control over resources outside of the Node Resource Group (NRG), you can skip this section.

1
assignmentNum=$(echo ${assignmentList} | jq length)
2
for ((i=0; i<${assignmentNum}; i++)); do
3
  roleId=$(echo "${assignmentList}" | jq -r '.['$i'] | .roleId')
4
  scope=$(echo "${assignmentList}" | jq -r '.['$i'] | .scope')
5

6
  az role assignment create --role ${roleId} \
7
  --assignee-object-id ${uamiIdentityId} -o none\
8
  --scope ${scope} --assignee-principal-type ServicePrincipal
9
done

Update current AKS cluster to use new user-assigned managed identity#

After all the preparation, we can migrate the current AKS security principal to the new user-assigned managed identity.

1
az aks update -n ${aks} -g ${rG} -o none \
2
--enable-managed-identity --assign-identity ${uamiId}

NOTE

This process DOES NOT automatically rotate the nodes.
If you were using system-assigned managed identity, the old system-assigned identity will be deleted.

CAUTION

If the current type of AKS security principal is service princial, there might be no going back. Check the note in Use a managed identity in Azure Kubernetes Service (AKS).
The process of converting your current AKS cluster to a private cluster will naturally resolve the old and new security principal conflicts, as node rotation occurs during the process. However, if you use AKS to pull images from ACR, you will need to re-grant permissions.

Re-grant permission to access ACR (Conditional skip)#

NOTE

If the AKS is using a system-assigned managed identity, skip this section.

This procedure is required for AKS which was using service principal, or after converting an AKS cluster to a private cluster, your workloads will fail. If you do not have any Azure Container Registry (ACR) resources, you can also skip this step.

Since we are now using managed identities, AKS now relies on the kubelet managed identity to access the Azure Container Registry (ACR). This managed identity is different from the managed identity we just created. These two identities can be the same, but we won’t discuss that here.

1. Check current ACR resources in your subscription

1
az resource list --resource-type Microsoft.ContainerRegistry/registries \
2
--query "[].{name:name,id:id}" -o yaml

2. Re-attach ACR back to AKS

1
az aks update -n ${aks} -g ${rG} --attach-acr {ACR_name/ID}

NOTE

This also applies to scenarios where you use managed identities to access storage accounts without access keys, or likewise. You will need to re-configure them to ensure they have access after a node rotation.