Before the main content#

In the official documentation on this topic, there is indeed a working method provided. However, the method provided in the official documentation is only a working method and is not designed for a multi-tenant environment.
If you follow the method in the official documentation, you may encounter troubles in the following situations:

• You are working with multiple external providers and multiple tenants.
• Your external provider does not want to provide you with access to their subscription.
• Your external provider is having trouble with complex configuration steps.
• If you want to migrate your workload to another AKS cluster, it becomes a hassle. If you follow the official documentation: all external providers will have to reconfigure everything from scratch.

The solution provided in this article will solve these problems.

Before you begin#

• Make sure OIDC issuer feature and workload identity add-on is enabled on your AKS cluster.

From your side#

# Assmuing AKS cluster is in Tenant 1 and target resource is in Tenant 2
tenant1=
tenant2=

# The application you want to create for cross-tenant access
app=

# AKS cluster name and resource group it locates in Tenant 1
rG1=
aks=

# Define Kubernetes resources service account and namespace name
# Both of them will be newly created in this article
# Example has been pre-filled here for convenience.
svc=dev-sa
namespace=dev-ns-$(tr -dc a-z < /dev/urandom | head -c 4)

# New key vault name and new resource group/location in Tenant 2
# Both key vault and resource group will be newly created in this article
rG2=
kv=
location=

# Retrieve OIDC Issuer URI
oidcIssuer=$(az aks show -n ${aks} -g ${rG1} \
    --query oidcIssuerProfile.issuerUrl -o tsv)

az aks get-credentials -n ${aks} -g ${rG1}

az ad app create --display-name ${app} -o none \
    --sign-in-audience AzureADMultipleOrgs

appClientId=$(az ad app list --display-name ${app} \
    --query '[0].appId' -o tsv)
appObjectId=$(az ad app show --id ${appClientId} \
    --query id -o tsv)

fidcParams=$(cat <<EOF
{
  "name": "kubernetes-federated-credential",
  "issuer": "${oidcIssuer}",
  "subject": "system:serviceaccount:${namespace}:${svc}",
  "description": "Kubernetes service account federated credential for ${namespace}:${svc}",
  "audiences": [
    "api://AzureADTokenExchange"
  ]
}
EOF
)

az ad app federated-credential create --id ${appObjectId} \
    --parameters "${fidcParams}" -o none

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: ${namespace}
---
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: ${appClientId}
    azure.workload.identity/tenant-id: ${tenant2}
  name: ${svc}
  namespace: ${namespace}
EOF

From external provider’s side#

az login --tenant ${tenant2}

az ad sp create --id ${appClientId} -o none

# This step is to make it shown as an "Enterprise Application"
az ad sp update --id ${appClientId} -o none \
    --set tags="['WindowsAzureActiveDirectoryIntegratedApp']"

spObjectId=$(az ad sp show --id ${appClientId} \
    --query id -o tsv)

az group create -n ${rG2} -l ${location} -o none
az keyvault create -n ${kv} -g ${rG2} -o none

kvId=$(az resource list -n ${kv} -g ${rG2} \
    --resource-type Microsoft.KeyVault/vaults \
    --query [0].id -o tsv)
kvUri=$(az keyvault show -n ${kv} -g ${rG2} \
    --query properties.vaultUri -o tsv)

userObjectId=$(az ad signed-in-user show --query id -o tsv)

az role assignment create --role "Key Vault Secrets Officer" \
    --assignee-object-id ${userObjectId} -o none \
    --scope ${kvId} --assignee-principal-type User
az keyvault secret set --vault-name "${kv}" \
   -n "my-secret" --value "Hello, World\!" -o none

az role assignment create --role "Key Vault Secrets User" \
    --assignee-object-id ${spObjectId} -o none \
    --scope ${kvId} --assignee-principal-type ServicePrincipal

Final check: access resources from your AKS cluster#

1. Set label and service account in Kubernetes Pod to access target resource

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: quick-start
  namespace: ${namespace}
  labels:
    app.kubernetes.io/name: masl-go
spec:
  replicas: 1
  selector:
    matchLabels:
      app: masl-go
  template:
    metadata:
      labels:
        app: masl-go
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: ${svc}
      containers:
        - image: ghcr.io/azure/azure-workload-identity/msal-go
          name: oidc
          env:
            - name: KEYVAULT_URL
              value: ${kvUri}
            - name: SECRET_NAME
              value: my-secret
EOF

2. Check if Pod can access target resource

kubectl logs -n ${namespace} -l app=masl-go,azure.workload.identity/use="true"

If you see “Hello, World!” in the log, it is successful.

Migrating to new AKS cluster#

Assuming you have a new AKS cluster with the workload identity add-on enabled, and you want to migrate your workloads to the new AKS cluster (and therefore delete the old AKS cluster), here are the steps you need to perform on the new cluster in your tenant:

1. Get client ID of the application with its object ID

appClientId=$(az ad app list --display-name ${app} \
    --query '[0].appId' -o tsv)
appObjectId=$(az ad app show --id ${appClientId} \
    --query id -o tsv)

NOTE

There is no need to register a new application or re-configure permissions.

2. Retrieve OIDC Issuer URI for new AKS cluster

newOidcIssuer=$(az aks show -n ${newAks} -g ${newRg} \
    --query oidcIssuerProfile.issuerUrl -o tsv)

3. Create service account in new AKS cluster

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: ${namespace}
---
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: ${appClientId}
    azure.workload.identity/tenant-id: ${tenant2}
  name: ${svc}
  namespace: ${namespace}
EOF

3. Create federated identity credential using defined parameters

• Get the ID of existing federated identity credential

fidcId=$(az ad app federated-credential list --id ${appObjectId} \
    --query "[?name=='kubernetes-federated-credential'].id" -o tsv)

• Define new parameters

fidcParams=$(cat <<EOF
{
  "name": "kubernetes-federated-credential",
  "issuer": "${newOidcIssuer}",
  "subject": "system:serviceaccount:${namespace}:${svc}",
  "description": "Kubernetes service account federated credential for ${namespace}:${svc}",
  "audiences": [
    "api://AzureADTokenExchange"
  ]
}
EOF
)

• Create federated identity credential

az ad app federated-credential update --id ${appObjectId} -o none \
    --federated-credential-id ${fidcId} --parameters "${fidcParams}"

• Deploy workload in new AKS cluster

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: quick-start
  namespace: ${namespace}
  labels:
    app.kubernetes.io/name: masl-go
spec:
  replicas: 1
  selector:
    matchLabels:
      app: masl-go
  template:
    metadata:
      labels:
        app: masl-go
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: ${svc}
      containers:
        - image: ghcr.io/azure/azure-workload-identity/msal-go
          name: oidc
          env:
            - name: KEYVAULT_URL
              value: ${kvUri}
            - name: SECRET_NAME
              value: my-secret
EOF

kubectl logs -n ${namespace} -l app=masl-go,azure.workload.identity/use="true"

Epilogue#

This is my first post since I lost my job on April 8th due to Executive Order 14117.
While writing this post, I was complaining about the lack of user experience in AKS documentation. As written at the beginning of this post: the official version does not have a thorough understanding of this topic.
This was an accident, and also an opportunity to change my life.
There will be more posts published from my draft list, so stay tuned.