Request Flow: Cloud DNS → GLB → Pod

A request behind a Global External Application Load Balancer touches anycast DNS, edge TLS termination, a WAF evaluation, host/path routing, and a container-native backend lookup before it reaches a pod — in that order. This doc walks the full path once as a sequence diagram, then breaks each hop down on its own. For deeper NEG mechanics, see gke.md's "GKE Load Balancing" section — this file compresses that into one end-to-end narrative, not a re-derivation.

Full Request Path

sequenceDiagram
    participant USER as Browser/Client
    participant DNS as Cloud DNS
    participant GLB as Global External App LB
    participant ARMOR as Cloud Armor
    participant BS as URL Map / Backend Service
    participant NEG as NEG
    participant POD as Pod :8080

    USER->>DNS: DNS query: app.example.com
    DNS-->>USER: A record --> anycast IP 34.x.x.x
    USER->>GLB: TCP SYN + TLS ClientHello to anycast IP
    GLB->>GLB: TLS handshake terminated at nearest Google PoP
    GLB->>ARMOR: evaluate security policy before routing
    ARMOR-->>GLB: allow (or deny-403, request stops here)
    GLB->>GLB: URL map match host/path --> backend service
    GLB->>NEG: forward to a healthy endpoint in the NEG
    NEG->>POD: HTTP GET /api/users, direct to pod IP:port
    POD-->>GLB: HTTP 200 response
    GLB-->>USER: HTTPS 200 response

Layer by Layer

Same path, broken into the hops a request crosses end to end — Cloud DNS, the LB itself, Cloud Armor, then the routing chain down to a pod. Step through it once here, then read each hop's detail below.

1. Cloud DNS

graph LR
    DNS["Cloud DNS public zone<br/>app.example.com"] -->|"A record"| IP["Anycast IP: 34.x.x.x<br/>same IP advertised from every Google PoP"]
    IP -->|"resolves to"| POP["Nearest Google PoP<br/>(no per-region IP to manage)"]

• Plain A record to the LB's static IP is enough — Cloud DNS has no ALIAS type, but the Global LB's IP is static and anycast, so an A record at the apex works natively (unlike Route53, which needs an Alias for a dynamic ALB IP). See services-overview.md's Cloud DNS section for the full record-type comparison.
• One IP serves the whole planet — no per-region DNS answer to compute or fail over between.
• TTL typically 300s for a public zone.

2. Global External Application Load Balancer

Architecturally this is the biggest departure from a regional LB (an ALB, or GCP's own Regional External/Internal App LB): the same anycast IP is advertised simultaneously from every Google PoP worldwide. There's no DNS-level trick making this work — it's BGP anycast, so each client's packets are naturally attracted to whichever PoP is topologically closest, and TLS terminates right there rather than after a trip to a single region.

graph LR
    IP["Same anycast IP: 34.x.x.x"] --> POP1["PoP: Tokyo"]
    IP --> POP2["PoP: Frankfurt"]
    IP --> POP3["PoP: Iowa"]
    POP1 -.->|"backend can live in any region"| BACKEND["Backend service<br/>cross-region failover"]

A regional LB has exactly one IP scoped to one region — covering multiple regions means standing up a separate LB per region and routing between them yourself. The Global LB collapses that into one IP and lets the backend service fail over across regions on its own.

3. Cloud Armor

Cloud Armor's evaluation happens before the request ever reaches a backend — the whole point of putting it at the edge rather than as an in-cluster filter. Two attachment points exist: backend security policies, attached to a backend service and evaluated once the URL map has picked it but still before forwarding to a NEG; and edge security policies, evaluated earlier still at the Cloud CDN caching layer, so even a cache hit never touching your backend can be filtered. Either way, rules are evaluated in priority order (lowest number first, same convention as VPC firewall rules) and the first match wins — allow, deny(403), rate_based_ban, or throttle.

gcloud compute security-policies rules create 1000 \
  --security-policy=my-waf-policy \
  --src-ip-ranges="1.2.3.0/24" --action=deny-403

gcloud compute backend-services update api-backend \
  --security-policy=my-waf-policy --global

4. URL Map → Backend Service → NEG

The URL map matches the request's host and path to a backend service — the same job an ALB's listener rules do. From there, GKE pods are exposed through a Network Endpoint Group: the NEG controller (cloud.google.com/neg annotation) registers each Ready pod's IP:port directly as a NEG endpoint, so the LB's data path goes straight to the pod — no NodePort, no kube-proxy iptables/IPVS hop. That's container-native load balancing in one sentence; gke.md's GKE Load Balancing section walks the annotate → create → sync → remove lifecycle in full.

5. Health Checks

Two independent health signals gate traffic, and they're easy to conflate. Kubernetes readiness decides NEG membership — a NotReady pod gets pulled out of the NEG entirely. The load balancer's own health check is a separate, GCP-managed probe hitting each NEG endpoint's path/port directly, independent of Kubernetes — typical defaults: checkIntervalSec: 5, timeoutSec: 5, healthyThreshold: 2, unhealthyThreshold: 3. A pod can be present in the NEG (Ready, per Kubernetes) while the GCP health check still marks it UNHEALTHY — a firewall blocking Google's health-check ranges (130.211.0.0/22, 35.191.0.0/16) is a common reason those two signals disagree.

Global External vs Internal HTTP(S) vs Passthrough Network LB

graph LR
    subgraph GLOBAL["Global External App LB — L7, anycast"]
        G1["Terminates TLS, routes by host/path"]
        G2["Cross-region backends, one IP"]
        G3["Cloud Armor + Cloud CDN"]
    end
    subgraph INTERNAL["Internal HTTP(S) LB — L7, regional"]
        I1["Terminates TLS, routes by host/path"]
        I2["RFC 1918 only — no internet ingress"]
        I3["East-west, service-to-service"]
    end
    subgraph PASSTHROUGH["External Passthrough Network LB — L4"]
        P1["No TLS termination, IP+port only"]
        P2["Preserves client IP natively"]
    end

Backend Service Load Balancing Modes

GCP doesn't pick one fixed algorithm the way "round robin" implies — it distributes requests based on each backend's balancing mode, which defines what "at capacity" even means.

graph LR
    REQ["Incoming request"] --> POOL{"Endpoints under
max-rate-per-endpoint"}
    POOL -->|"has capacity"| POD1["Pod A"]
    POOL -.->|"at RPS limit, skipped"| POD2["Pod B"]
    classDef ok fill:#27ae60,stroke:#1e8449,color:#fff;
    class POD1 ok;

graph LR
    CONN["New connection"] --> POOL2{"Endpoints under
max-connections-per-endpoint"}
    POOL2 -->|"has room"| POD3["Pod A: 40 conns"]
    POOL2 -.->|"at connection limit"| POD4["Pod B: 200 conns"]
    classDef ok fill:#27ae60,stroke:#1e8449,color:#fff;
    class POD3 ok;

graph LR
    IG{"Instance group
reported CPU"} -->|"below target"| VM1["VM 1: 30% CPU"]
    IG -.->|"above target, skipped"| VM2["VM 2: 85% CPU"]
    classDef ok fill:#27ae60,stroke:#1e8449,color:#fff;
    class VM1 ok;

# Set RATE balancing on a NEG backend (the mode GKE container-native LB uses)
gcloud compute backend-services add-backend api-backend \
  --global \
  --network-endpoint-group=my-neg \
  --network-endpoint-group-zone=us-central1-a \
  --balancing-mode=RATE \
  --max-rate-per-endpoint=100

If Pod is Running but Receiving No Traffic — Debug Flow

flowchart TD
    NTRAFFIC["Pod Running but no traffic"] --> STEP1
    STEP1["1. Check pod logs<br/>kubectl logs pod"] --> STEP2
    STEP2["2. curl pod directly<br/>kubectl exec -- curl localhost:8080/health"] --> HEALTHY{Healthy?}
    HEALTHY -->|No| APPBUG["App bug or not listening<br/>on 0.0.0.0"]
    HEALTHY -->|Yes| STEP3["3. Check Service endpoints<br/>kubectl get endpoints svc-name"]
    STEP3 --> NOEP{Endpoints empty?}
    NOEP -->|Yes| LABEL["Labels don't match Service selector<br/>kubectl get pods --show-labels"]
    NOEP -->|No| STEP4["4. Check NEG has the pod endpoint<br/>gcloud compute network-endpoint-groups list-network-endpoints"]
    STEP4 --> NOENDPOINT{Pod IP missing from NEG?}
    NOENDPOINT -->|Yes| NEGSYNC["NEG controller hasn't synced yet,<br/>or neg annotation missing on Service"]
    NOENDPOINT -->|No| STEP5["5. Check backend service health<br/>gcloud compute backend-services get-health"]
    STEP5 --> UNHEALTHY{Backend unhealthy?}
    UNHEALTHY -->|Yes| HC["Health check misconfigured,<br/>or firewall blocks GFE probe ranges<br/>130.211.0.0/22, 35.191.0.0/16"]
    UNHEALTHY -->|No| STEP6["6. Check Cloud Armor logs<br/>for a DENY decision"]
    STEP6 --> BLOCKED{Requests denied by Cloud Armor?}
    BLOCKED -->|Yes| ARMORFIX["Wrong rule priority or overly broad match<br/>gcloud compute security-policies rules describe"]
    BLOCKED -->|No| STEP7["7. Check readiness probe<br/>kubectl describe pod - Readiness section"]

Full Debug Command Set

# 1. Pod logs (current + previous crash)
kubectl logs <pod> -n <namespace>
kubectl logs <pod> -n <namespace> --previous

# 2. Test app directly (bypass all load balancers)
kubectl exec -it <pod> -n <namespace> -- curl -v http://localhost:8080/health

# 3. Service endpoints — is the pod in the endpoint list?
kubectl get endpoints <service> -n <namespace>
kubectl describe svc <service> -n <namespace> | grep -A5 "Annotations\|Selector"
kubectl get pods -n <namespace> --show-labels

# 4. Is the pod actually registered as a NEG endpoint?
gcloud compute network-endpoint-groups list --zones=us-central1-a
gcloud compute network-endpoint-groups list-network-endpoints <neg-name> \
  --zone=us-central1-a

# 5. Backend service health per endpoint
gcloud compute backend-services get-health <backend-service> --global

# 6. Cloud Armor — any DENY decisions against this traffic?
gcloud logging read \
  'resource.type="http_load_balancer" AND jsonPayload.enforcedSecurityPolicy.outcome="DENY"' \
  --limit=20 --format=json

# 7. Readiness probe status and recent events
kubectl describe pod <pod> -n <namespace> | grep -A 10 Readiness
kubectl get events -n <namespace> --field-selector involvedObject.name=<pod> \
  --sort-by='.lastTimestamp'

# 8. Port-forward to bypass NEG/LB entirely
kubectl port-forward pod/<pod> 8080:8080 -n <namespace>
curl -v http://localhost:8080/health