AWS ECS Infrastructure with Cloud Map Service Discovery
AWS ECS Infrastructure with Cloud Map Service Discovery
An agriculture-tech company needed its web application split into an independently scaling frontend and backend, running in a private, production-grade AWS network. I built the infrastructure as Terraform: a two-AZ VPC, an ECS cluster on EC2 running two separately scaled services behind their own load balancers, and a bastion host for controlled access. A service-discovery layer and a third mapping service were also designed and built as modules, held back from the live deploy pending a confirmed need.
The problem
The client's application had two very different workloads under one roof, a light frontend and a much heavier backend job that needed more CPU, more memory, and privileged host access to write files to disk. Running both on the same fixed infrastructure meant the backend's needs would either starve the frontend or force the whole stack to be oversized just to keep the backend happy.
Constraints
- ·Two workloads, one platform. A light frontend and a heavy backend had to run on the same ECS cluster without one starving the other.
- ·Backend needs EC2, not Fargate. The backend container runs privileged and mounts a host path for static file storage, which Fargate does not support.
- ·No public path into the private tier. Both service tiers, and any operator access, had to live in private subnets with no direct internet route.
- ·Single engineer. Built end to end by one person, so the module design had to stay reviewable.
- ·One TLS certificate, two public load balancers. Both ALBs terminate HTTPS on the same ACM certificate.
Architecture
A four-module Terraform root (vpc, alb, ecs, bastion) plus a fifth module (cloud-map) that exists in source but is not called. The VPC spans two AZs with a NAT gateway per zone. Two ECS services, frontend and backend, each sit behind their own ALB and run on their own EC2 auto-scaling group, combined on one ECS cluster through a weighted capacity-provider strategy. A bastion host is the only SSH path into the private subnets. Cloud Map (a private DNS namespace plus a backend service-discovery entry) and a third mapping/GeoServer-style ECS service are both fully drafted as modules, with every call into them commented out of the root.
- ›vpc module: 10.0.0.0/16 VPC, 2 public + 2 private subnets across 2 AZs, IGW, NAT gateway per AZ, per-tier security groups
- ›alb module: frontend ALB (port 9000, health check /) and backend ALB (port 5000, health check /get-jobs), both HTTPS on one ACM cert
- ›ecs module: one cluster, frontend service on t3.medium ASG, backend service on t3.xlarge ASG (privileged, host volume mount), shared task/execution IAM roles
- ›bastion module: one t3.small EC2 in a public subnet, referenced by security-group ID from every private-tier security group
- ›cloud-map module: private DNS namespace + backend service-discovery entry, built, its module call and both services' service_registries blocks commented out
- ›mapping tier: a third ECS service, ALB, and task definition in mapping_resources.tf and mapping-main.tf, entirely commented out, never applied
How one request flows
- 01Client hits one of two public ALBs. The frontend ALB and the backend ALB each terminate HTTPS with the same ACM certificate and redirect HTTP to HTTPS.
- 02ALB forwards to its own target group. Frontend forwards to port 9000 with a health check on /. Backend forwards to port 5000 with a health check on /get-jobs.
- 03ECS routes to a running task. Both services sit on one cluster, each on its own EC2 auto-scaling group, combined through a weighted capacity-provider strategy.
- 04The frontend task serves the request directly, a standard awsvpc-networked container logging to its own CloudWatch group.
- 05The backend task runs privileged with a mounted host volume, writing static output to a path only the EC2 launch type allows.
- 06An operator can reach either instance through the bastion, the only security group allowed to SSH into either private tier.
- 07Cloud Map plays no part in this path today. The module that would let the backend resolve by private DNS name exists but its registration is commented out.
Engineering decisions
Two services, two ALBs, one cluster
What. Each service gets its own ALB, target group, and auto-scaling group, combined on a single ECS cluster through a weighted capacity-provider strategy.
Why. A shared ALB with path-based routing would have coupled the two tiers' health checks and TLS listeners together.
Tradeoff. More load balancers to manage, accepted so a backend redeploy or routing change never touches the frontend's listener.
EC2 launch type, chosen over Fargate
What. The whole stack standardized on the ECS EC2 launch type with dedicated auto-scaling groups per service.
Why. The backend needs privileged mode and a host-path volume mount, neither supported on Fargate.
Tradeoff. More infrastructure to manage than a serverless launch type, but the only way to get the backend the access it needs.
Bastion by security-group reference, not CIDR
What. Every ECS security group only accepts SSH from the bastion's own security group, referenced by ID.
Why. Keeps the actual application instances unreachable except through one audited hop, rather than opening a port range to a network block.
Tradeoff. None meaningful, the extra reference costs nothing at apply time.
Cloud Map built, held back from the live stack
What. A private DNS namespace and backend service-discovery entry were built as their own module, but the module call, the ECS service_connect_defaults, and both services' service_registries blocks were left commented out.
Why. The decision was to ship what the client needed now and keep discovery ready to enable later, not to apply infrastructure ahead of a confirmed need.
Tradeoff. The backend has no private DNS name today, callers still resolve through the ALB.
A third tier drafted, not applied
What. A mapping/GeoServer-style ECS service, with its own ALB and task definition, was designed as a third module alongside frontend and backend.
Why. The live backend security group already opens a port to an external, already-existing GeoServer security group by ID, which is the one trace of that integration ever being planned.
Tradeoff. The module exists un-applied rather than being deleted, so the design is not lost if the integration is confirmed later.
What changed along the way
- →Cloud Map was built as a working module before the client confirmed the backend needed private-DNS discovery, so it shipped commented out rather than half-applied.
- →The mapping/GeoServer tier moved from an assumed third live service to a fully drafted but unapplied module once it became clear the integration target sat outside this Terraform stack.
- →The backend's resource footprint (cpu 3584, memory 15800) grew large enough during design that a separate auto-scaling group from the frontend became the only workable scaling shape.
Security and reliability
- ·Private subnets for compute. Both ECS tiers run in private subnets behind NAT, with no direct inbound route from the internet.
- ·Bastion-only SSH. Every ECS security group accepts SSH only from the bastion's security group, referenced by ID, not by CIDR range.
- ·Scoped IAM. Separate ECS task and execution roles, plus dedicated frontend and backend EC2 instance roles, rather than one shared role.
- ·TLS everywhere public. Both ALBs redirect HTTP to HTTPS and terminate on the same ACM certificate.
- ·Deployment circuit breaker. Both ECS services have a deployment circuit breaker configured, without automatic rollback.
Metrics
What shipped
- ✓Two independently deployable ECS services (frontend, backend) each fronted by its own ALB, TLS listener, and health check.
- ✓A two-AZ VPC with NAT egress and least-scoped security groups tying the ALB, ECS, and bastion tiers together.
- ✓A complete AWS Cloud Map service-discovery module (namespace + backend service registry) ready to wire in, left disabled pending the ECS `service_registries` cut-over.
- ✓A third mapping/GeoServer-style ECS service fully drafted in Terraform (cluster, task definition, ALB) but not applied to the account.
- ✓A bastion host giving controlled SSH access into the private subnets for both ECS tiers.
Lessons learned
- ·Building a module ahead of the client confirming they need it is fine, applying it ahead of that confirmation is not, comment it out and ship what is actually needed.
- ·A privileged container with a host-path volume forces the whole tier onto EC2, decide the launch type from the hardest constraint, not the average one.
- ·A bastion referenced by security-group ID rather than CIDR range removes a whole class of accidental exposure for no extra cost.
- ·Two ALBs sharing one ACM certificate keeps TLS management simple without coupling the tiers' routing together.