03 · Local-to-GCP GKE Rosetta Stone¶
Zero Surprises on Cloud Day: Use this side-by-side translation guide when moving from your local Docker + Minikube labs (Labs 01–10) to the GCP Terraform + GKE stack (
acme-labs-plan.mdPhases P0–P9).
1. Visual Architecture Mapping: Local Lab ⇄ GCP Production¶
2. Component-by-Component Translation Table¶
| Architectural Layer | Local Lab Implementation | GCP GKE Implementation | What Transfers 100% | What Is New on GCP (Focus Here in P1–P9) |
|---|---|---|---|---|
| VPC & Subnets | docker network create acme-main (10.10.0.0/24) & acme-ops (10.10.1.0/28) |
google_compute_network (delete_default_routes_on_create=true) + google_compute_subnetwork |
Primary CIDR layout, non-overlapping IPAM rules | Secondary ranges (gke-pods, gke-svcs), proxy-only-subnet (10.10.254.0/23), Master /28 (10.10.250.0/28) |
| Routes & Next Hops | ip route add ... via 10.10.0.250 on nodes & probe |
google_compute_route resources with priorities (900, 999) |
Longest-prefix match, asymmetric routing failure mode | next_hop_gateway = "default-internet-gateway" is required for Cloud NAT to trigger |
| L3/L4 Firewall | iptables -P FORWARD DROP on vpc-router |
google_compute_firewall rules on VPC |
Stateful connection tracking, Authorized Networks concept | Master → Node Webhook firewall (10.10.250.0/28 → 15017, 8443, 9443), Google LB health-check ranges (130.211.0.0/22, 35.191.0.0/16) |
| Bastion / Admin Probe | probe container (10.10.1.3) in acme-ops |
e2-micro VM in ops subnet (--no-address, tag iap-ssh) |
Testing API server & Internal LB from a separate subnet | Identity-Aware Proxy (IAP) TCP forwarding (35.235.240.0/20 → port 22) |
| Cluster & Node Pools | minikube start -p acme --nodes=3 + kubectl label node pool=main |
google_container_cluster (Standard, Zonal, Private) + google_container_node_pool (Spot e2-standard-2) |
max_pods_per_node math, node labels, resource allocatable math |
enable_private_nodes = true, master_authorized_networks_config, dedicated least-privilege Node GSA (gke-nodes) |
| Pod Networking (CNI) | Cilium via Helm (ipam.mode=kubernetes, VXLAN/host routing) |
GKE Dataplane V2 (networking_mode = "VPC_NATIVE", ADVANCED_DATAPATH) |
Cilium NetworkPolicy enforcement, Hubble verdicts |
Alias IPs: GCP VPC natively routes 10.20.0.0/20 without overlay encapsulation inside the VPC |
| Door 1: Private Cloud APIs & Registry | Local registry (10.10.0.200:5000) + /32 route |
Private Google Access (PGA) + Route 199.36.153.8/30 + Private Cloud DNS (*.googleapis.com, *.pkg.dev → private.googleapis.com) |
Pulling images & shipping logs when 0.0.0.0/0 is deleted |
Private DNS CNAME wildcard zones + Artifact Registry Remote Repository (dockerhub-mirror) |
| Door 2: /32 VIP Allow-List | ip route add $VIP/32 via 10.10.0.1 |
google_compute_route per IP in var.vip_allowlist (priority 900) |
Exact /32 longest-prefix override over deleted default route |
Combined with Cloud NAT static IPs so partner APIs can allow-list your egress IP |
| Door 3: Tag-Gated Internet Egress | Node label allowed-open-nat=true + default route on acme-m03 only |
google_compute_route (0.0.0.0/0, priority 999, tags = ["allowed-open-nat"]) + Cloud NAT (ALL_IP_RANGES) |
Pods inherit their node's egress route/tag! Tags cannot filter per-pod. | Cloud NAT MANUAL_ONLY with 2 reserved google_compute_address static IPs |
| Door 4: Hybrid / On-Prem | Route to 10.10.1.0/28 via 10.10.0.250 (vpc-router) |
HA VPN + Cloud Router BGP to onprem-sim VPC (192.168.100.0/24) |
Unaffected by deleting default internet route; masquerade trap | ip-masq-agent config so pod IPs are SNAT'd to node IPs (or advertise pod CIDR over BGP) |
| Cloud / Pod Identity | Kubernetes ServiceAccount (orders, payments, inventory) |
GKE Workload Identity (KSA annotated with iam.gke.io/gcp-service-account: orders-app@...) |
SPIFFE identity (cluster.local/ns/dom-orders/sa/orders) for Istio |
Critical Trap: Default-deny egress blocks GKE Metadata Server (169.254.169.252:988)—must explicitly allow it in NetworkPolicy! |
| CI/CD Identity | Local CLI / GitLab PAT | Workload Identity Federation (WIF) OIDC Pool + CEL attribute condition (root_group_ref = <root>:main:true) |
Zero long-lived JSON service account keys | Branch protection enforced in Google IAM principalSet (pipeline-p vs pipeline-p-ro), not just GitLab UI |
| Observability | app + istio-proxy + fluent-bit + otel-collector → log-aggregator |
Same 4-container pod (stackdriver / googlecloud exporters) → Cloud Logging Sink → central-app-logs |
Volume mounts (emptyDir, subPath), resource requests, Egress holes |
Log Router sink filter (namespace_name=~"^dom-"), _Default bucket exclusion to avoid paying twice |
| North-South Entry | Service type: LoadBalancer (MetalLB 10.10.0.240) |
Kubernetes Gateway (gatewayClassName: gke-l7-rilb) + HTTPRoute |
All 4 ingress breaks (Firewall → NetPol → STRICT mTLS → AuthZ) | Google-managed Envoy proxies live in proxy-only-subnet (10.10.254.0/23); firewall must allow 10.10.254.0/23 → nodes |
| GitOps Delivery | FluxCD bootstrapping clusters/local (10-controllers → 20-configs → 30-apps) |
FluxCD bootstrapping clusters/np |
100% identical YAML structure and dependsOn ordering |
Helm charts pulled from Artifact Registry OCI (oci://us-central1-docker.pkg.dev/...) |
| Live packet capture | docker exec vpc-router tcpdump -nn -i any |
VPC Flow Logs (log_config on subnet) + Packet Mirroring |
Same packet-level verdict on "did the packet even arrive" | Flow logs are aggregated metadata (5-tuple + verdict), not full payloads; enable per-subnet in P3 |
| Reachability simulation | Hubble hubble observe --verdict DROPPED |
Network Intelligence Connectivity Tests (gcloud network-management connectivity-tests create) |
Names which route/firewall decides a packet | GCP tests are simulated (config analysis), not real packets — combine with flow logs |
| Pod event triage | kubectl get pods → describe pod → logs --previous |
Identical kubectl + GKE console Events tab |
The entire debug flow transfers unchanged | Events also surface in Cloud Logging (k8s_event) |
| LB backends | MetalLB L2 VIP → kube-proxy → pod | Google LBs → NEGs (Network Endpoint Groups: pods are LB backends directly) | VIP → pod semantics | No NodePort hop; 502 almost always = failing health checks (wrong path/port or blocked 130.211.0.0/22 + 35.191.0.0/16) |
.env / secrets |
K8s Secret + envFrom/secretKeyRef |
Secret Manager + External Secrets Operator (or CSI driver) | Same pod-level env/volume injection | ESO syncs Secret Manager → K8s Secret; audit + rotation are the real wins |
| Shared database | commerce-db Postgres StatefulSet (dom-data), schemas inventory + sales, orders_rw/inventory_rw roles |
commerce-sql Cloud SQL Postgres (private IP via PSA 100.64.0.0/20) |
Same schemas, roles, grants; single-tx "decrement + order" saga | Cloud SQL Auth Proxy as 5th pod sidecar (DATABASE_URL → 127.0.0.1:5432); IAM DB auth replaces pg passwords |
| Private database | payments-db Postgres StatefulSet (dom-data), payments_rw only |
payments-sql Cloud SQL Postgres |
Same instance-level isolation; payments_rw exists only here |
Instance-level IAM/cloudsql.client binding replaces the payments-db-ingress NetworkPolicy |
| DB seeding | db-seed Kubernetes Job (psql, idempotent) |
SQL migrations in app CI, or google_sql_user/google_sql_database |
Same "schema exists before apps" ordering | Flux 25-data tier ↔ CI migration step before HelmRelease rollout |
| Registry auth | Open local registry (10.10.0.200:5000) |
Artifact Registry via node SA artifactregistry.reader |
Pull without imagePullSecrets |
Pull secrets replaced by node/workload identity; AR mirror keeps Door 1 egress-only |
| Metrics & dashboards | none — logs only (aggregator) | Cloud Monitoring — Golden Signals: latency, traffic, errors, saturation | — | Genuinely new: monitoring_config + Managed Prometheus / dashboards (P6) |
| Distributed tracing | otel-collector → debug exporter (spans to stdout) |
Cloud Trace via googlecloud exporter |
Same OTLP instrumentation & pipeline | Correlate logs↔traces with trace field in log entries; swap is one exporter line |
3. The 6 Genuinely New Things You Only Encounter on GCP¶
Once you complete Labs 01–10 locally, 85% of your GKE platform is already built and debugged. When you switch to acme-labs-plan.md, focus your energy on these 6 cloud-only mechanics:
- GitLab OIDC → GCP Workload Identity Federation (P1–P2): Writing the CEL expression
assertion.project_path.split('/')[0] + ':' + assertion.ref + ':' + assertion.ref_protectedso feature branches cannot impersonatepipeline-p. - Private Google Access + Private DNS Zones (P3 & P8 - Door 1): Routing
199.36.153.8/30todefault-internet-gatewayand creating Private Cloud DNS zones forgoogleapis.com.,gcr.io., andpkg.dev.. - Private Control Plane Webhook Firewall Rule (P3 & P5): Allowing GKE's private master CIDR (
10.10.250.0/28) to reach worker nodes on TCP15017(istiodmutating webhook) and8443/9443. - GKE Metadata Server Egress Hole for Workload Identity (P7): Adding
ipBlock: 169.254.169.252/32port988to every namespace's baseline EgressNetworkPolicysofluent-bitandotel-collectorcan fetch GCP OAuth tokens. - Regional Managed Proxy Subnet & Health Check Ranges (P3 & P9): Provisioning
10.10.254.0/23(REGIONAL_MANAGED_PROXY) and opening VPC firewalls + NetworkPolicies for130.211.0.0/22and35.191.0.0/16. - Managed Observability Stack (P6): Cloud Logging
gcloud logging readfilters (resource.type="k8s_container" severity>=ERROR), Cloud Monitoring Golden Signals dashboards, Cloud Trace with log↔trace correlation, and VPC Flow Logs as yourtcpdumpreplacement.
