Troubleshooting¶

Start with the health check. It reports problems across Flux, Crossplane packages, XRDs, provider pods, and managed-resource activation:

./infrastructure/verify-crossplane.sh [--context <kube-context>]

A resource does not appear in Azure¶

Follow the reconciliation chain from Flux to the composite resource (XR), then to its composed Azure managed resource:

1. Check Flux¶

flux get kustomizations -A
flux logs --kind Kustomization --name teams --namespace flux-system

2. Check the composite and recent events¶

Replace mvpdagen-app and mvpdagen with the XR name and team namespace:

kubectl get xresourcegroups.azure.platform.example.org -A
kubectl describe xresourcegroup.azure.platform.example.org/mvpdagen-app \
  -n mvpdagen
kubectl get events -n mvpdagen --sort-by=.lastTimestamp

3. Check the composed Azure resource¶

kubectl get resourcegroups.azure.m.upbound.io -A
kubectl describe resourcegroup.azure.m.upbound.io -n mvpdagen

After correcting a team manifest, ask Flux to reconcile it:

flux reconcile kustomization teams -n flux-system --with-source

A resource kind is not recognized¶

When a composition uses a new managed-resource kind, its MRD must be listed in the ManagedResourceActivationPolicy. Otherwise Crossplane never creates the CRD for that kind. Add the MRD name in <plural>.<group> form, then wait for Flux to reconcile the activation policy.

An Entra ID group is not created or updated¶

XSecurityGroup needs Microsoft Graph permissions on the service principal. Authorization_RequestDenied or failed to validate user in the XR events means Group.ReadWrite.All or User.Read.All is missing or lacks admin consent. Re-run the bootstrap as an administrator, or ask one to run:

client_id=$(jq -r .clientId infrastructure/azure-credentials-aks.json)
az ad app permission add --id "$client_id" \
  --api 00000003-0000-0000-c000-000000000000 \
  --api-permissions 62a82d76-70ea-41e2-9197-370581804d09=Role \
  df021288-bdef-4463-88db-98f22de89214=Role
az ad app permission admin-consent --id "$client_id"

Use azure-credentials-kiac.json for the local cluster. UPNs that do not exist in the tenant are skipped and listed in status.unresolvedMembers or status.unresolvedOwners:

kubectl get xsecuritygroups.entraid.platform.example.org -A \
  -o custom-columns=NAME:.metadata.name,MEMBERS:.status.unresolvedMembers,OWNERS:.status.unresolvedOwners

The documentation site is unavailable¶

The site is public only on AKS. The local kiac cluster has no Gateway, certificate, or public DNS record.

The pod reports ImagePullBackOff¶

The GHCR package is private by default, even though the repository is public. Make the docs package public under Packages > docs > Package settings > Change visibility.

The certificate is not ready¶

The initial DNS-01 challenge can take two or three minutes to propagate. If it remains pending, inspect cert-manager:

kubectl describe certificate -n documentation-site docs-site-tls
kubectl get challenge -A
kubectl logs -n cert-manager deploy/cert-manager

A permission error usually means the service principal lost access to rg-public-dns. Let's Encrypt rate-limit errors can happen after repeated cluster rebuilds; switch the Gateway's cert-manager.io/cluster-issuer annotation to letsencrypt-staging while testing.

The hostname does not resolve¶

external-dns reads the address from the Gateway status, not the Service. If the Gateway is not programmed, no DNS record is written:

kubectl get kustomization -n flux-system \
  gateway-api-crds external-dns docs-site-gateway
kubectl get configmap -n flux-system azure-dns-config
kubectl get gateway -n documentation-site -o wide
kubectl logs -n external-dns deploy/external-dns
az network dns record-set a list -g rg-public-dns \
  -z demo.liasis.dev -o table

If gateway-api-crds is blocked by a Helm release Secret larger than 1 MiB, sync the latest manifests; Flux now applies the rendered CRDs directly. If azure-dns-config is missing, set docs.acmeEmail in config-aks.yaml and rerun the AKS bootstrap.

The site is stale after a push¶

The docs workflow builds an image and commits its immutable tag back to the repository. Check that the workflow ran and that the tag in deployment.yaml matches the latest commit.

For setup instructions, see Local cluster setup or AKS cluster setup.