DevOps & Cloud Interview Prep: Real Scenarios & Answers

Most engineers assume Karpenter is always the right answer for Kubernetes node autoscaling, but at 500 nodes the tradeoffs around ASG lock-in, provisioner complexity, and migration risk get serious. This episode breaks down when to keep Cluster Autoscaler, when Karpenter wins, and how to articulate both sides clearly in a senior DevOps or SRE interview. Covers real configuration details, scaling l...

A Java service keeps getting OOMKilled in Kubernetes even though memory requests look fine on paper. This episode explains why JVM heap defaults ignore container limits, how to set maximum heap size correctly, and what interviewers expect when they probe your understanding of Java memory in containerized environments. Covers Xmx flags, UseContainerSupport, native memory overhead, and the tradeoffs...

When AWS fires the 2-minute Spot reclaim notice, Karpenter's interruption queue is the difference between a blip and a batch job disaster — here's exactly how to configure it. You'll learn: How to set karpenter.sh/capacity-type in a NodePool to prefer Spot with automatic On-Demand fallback The full interruption flow: SQS queue → cordon → graceful drain → pod rescheduling, all within the 2-minute w...

Automated canary analysis for a Flink-based streaming app is a common senior SRE interview scenario — here's how to wire Prometheus, Loki, and Pyroscope into a production-grade rollout strategy. You'll learn: How to define canary success criteria using Prometheus metrics like consumer lag, throughput, and error rate on Flink jobs Using Loki log queries to surface structured errors in canary vs. ba...

Grafana Mimir storage at 10TB/day scale forces real trade-offs — here's how to configure tiered storage to S3 without bleeding cost or tanking query performance. You'll learn: How Mimir's store-gateway and compactor interact with S3-backed object storage at high ingest volume Configuring blocks_storage with tiered retention — keeping hot blocks in fast storage while offloading cold blocks to S3 Gl...

If your service has a 99.99% SLO and Azure drops a zone for 15 minutes, here's exactly how to calculate the error budget burn rate before your next SRE interview. You'll learn: How to derive total monthly error budget from a 99.99% SLO (~4.38 minutes/month) Why a 15-minute outage consumes roughly 3.4x your entire monthly budget — and how to show that math The burn rate formula interviewers expect:...

Designing a PCI-DSS compliant serverless payments architecture on GCP means getting Confidential VMs, Cloud External Key Manager, and Binary Authorization working together — here's how to answer that in a senior interview. You'll learn: How Confidential VMs provide hardware-level memory encryption to satisfy PCI-DSS data-in-use requirements Why Cloud External Key Manager (CEKM) lets you hold encry...

Deploying EKS clusters across AWS accounts with CDK is a common senior interview scenario — here's how to handle VPC peering, Transit Gateway attachments, and IAM trust policies correctly. You'll learn: How to structure a multi-account CDK app using Stacks across environments with explicit env account/region targets When to use VPC peering vs Transit Gateway for cross-account EKS network connectiv...

Correlating OpenTelemetry traces with CloudWatch Logs Insights across Lambda and Step Functions is a common senior interview scenario — here's exactly how to answer it. You'll learn: How to propagate trace context (W3C TraceContext headers) across Lambda invocations and Step Functions state transitions so trace IDs land in your structured logs Configuring the AWS Distro for OpenTelemetry (ADOT) La...

Splitting a monolithic 4GB Terraform state file into scoped microstates is one of the nastiest live-infrastructure challenges you'll face — here's how to do it without downtime using terraform state rm and moved blocks. You'll learn: Why state files balloon past 4GB and why that breaks plan/apply performance How to use terraform state rm to surgically extract resources without destroying them Usin...

Designing a monorepo CI pipeline that doesn't collapse under 1,000 microservices means getting Bazel remote caching and selective test execution right from the start. You'll learn: How to structure a monorepo CI pipeline so only affected services trigger builds — using Bazel's dependency graph to compute the minimal affected set Configuring Bazel remote caching (local cache, shared remote cache vi...

Learn how to generate dynamic Azure RBAC role assignments using Pulumi with YAML-driven definitions — including tag-scoped conditions like restricting storage access to env:prod resources only. You'll learn: How to define custom Azure RBAC roles in YAML and hydrate them through Pulumi's automation layer Using condition and conditionVersion fields in role assignments to enforce attribute-based acce...

Taming Prometheus cardinality explosion in an Istio service mesh — dropping from 10 million to 500K active series using relabel_configs and recording rules — is exactly the kind of production war story senior SRE interviews dig into. You'll learn: Why Istio telemetry generates cardinality explosions and which high-cardinality labels (source_workload, destination_service, pod IPs) are the usual cul...

A developer pushes a Terraform module with a public S3 bucket — here's exactly how to catch and block it in your Argo CD pipeline using Conftest policy-as-code before it ever reaches production. You'll learn: How Conftest integrates with Argo CD as a pre-sync hook to enforce OPA policies on Terraform plans Writing a Rego rule that flags acl = public-read or block_public_acls = false on aws_s3_buck...

Managing a Terragrunt dependency graph across 500+ modules without hitting circular dependencies or version drift is one of the hardest scaling problems in platform engineering. You'll learn: How to map and audit a large Terragrunt dependency graph using terragrunt graph-dependencies and DAG visualisation tools Patterns for structuring module hierarchies to prevent circular dependencies before the...

External Secrets Operator lets you sync HashiCorp Vault dynamic secrets directly into Kubernetes Secrets — no Vault Agent sidecars, no annotation sprawl. You'll learn: How ESO's ExternalSecret and SecretStore CRDs map Vault paths to Kubernetes Secrets Why dynamic secrets (short-lived, auto-rotated) are preferable to static tokens and how ESO handles lease renewal The auth methods ESO supports for...

Parallel Jenkins jobs deploying Helm charts can deadlock silently — here's how to catch and fix mutex contention before it kills your pipeline. You'll learn: Why concurrent Helm deploys compete for the same release lock and how that surfaces as a deadlock in Jenkins How to run jstack against the Jenkins JVM to capture thread dumps and identify which threads are waiting on a monitor lock Reading mu...

Learn how to enforce CloudFormation stack drift detection at scale using AWS Config rules and Lambda-driven auto-remediation — a common architecture question in senior Cloud and DevOps interviews. You'll learn: How AWS Config detects configuration drift against CloudFormation expected stack states using managed and custom rules Wiring an EventBridge rule to trigger a Lambda function when Config fl...

Reducing DynamoDB Global Tables data transfer costs by 70% is achievable in a multi-region Active-Active setup — if you know where the money is actually going. You'll learn: Why replicated write costs dominate in DynamoDB Global Tables and how to model them accurately Using write sharding and conditional writes to reduce unnecessary replication traffic DAX (DynamoDB Accelerator) placement per regi...

When a database migration fails mid-deploy, your Kubernetes job hooks and Flyway versioning strategy are the difference between a five-minute fix and a 2am incident. You'll learn: How to structure Flyway versioned and undo migrations so a failed V3 doesn't leave your schema in a half-applied state Using Kubernetes init containers and Job postStart/preStop hooks to gate application rollout on migra...

When terraform apply times out creating 100+ IAM roles, the culprit is usually AWS API throttling combined with Terraform's default parallelism — here's how to fix it. You'll learn: Why the default parallelism=10 isn't always safe and when raising it to -parallelism=50 helps vs. hurts How AWS IAM's eventual-consistency model causes race conditions during bulk role creation Batching strategies: spl...

When GitHub Actions pipelines hit thousands of daily builds, your runner strategy becomes a first-class infrastructure decision — here's how to choose between self-hosted runners, larger hosted runners, and the Kubernetes executor. You'll learn: How GitHub-hosted larger runners (up to 64-core) reduce ops overhead versus self-hosted, and where the cost curve flips Self-hosted runner autoscaling wit...

Enforcing FIPS 140-3 compliance on an EKS cluster means locking down every layer — from the OS to the key management hardware — and this episode walks through exactly how Bottlerocket and AWS KMS make that possible. You'll learn: Why Bottlerocket OS ships with a FIPS-validated kernel and how to verify its cryptographic module status at node bootstrap How AWS KMS custom key stores backed by CloudHS...

When you're drowning in 1,000+ alerts a day, AWS Lookout for Metrics can route only the anomalies that matter directly to Slack or Teams — here's how to wire it up. You'll learn: How AWS Lookout for Metrics uses ML to separate real anomalies from noise across CloudWatch, S3, and RDS data sources Routing detected anomalies to Slack or Microsoft Teams via SNS topics and Lambda webhook integrations T...

Auditing cross-account IAM roles is one of those senior interview topics where vague answers kill your chances — here's how to use AWS IAM Access Analyzer and Policy Sentry to give a precise, credible response. You'll learn: How IAM Access Analyzer detects externally accessible roles and flags unintended cross-account trust relationships How Policy Sentry helps you write and audit least-privilege...