AWS Machine Learning Deployment with Terraform
Earthquake Building-Damage Detection on AWS SageMaker
A national government ministry needed a way to spot earthquake-damaged buildings from aerial imagery quickly enough to inform response decisions. I trained and deployed a custom Detectron2 object-detection model behind an AWS SageMaker real-time endpoint, wrote the Lambda pipeline that turns raw model output into geolocated GeoJSON, and provisioned the surrounding S3, Lambda, and Elastic Beanstalk infrastructure with Terraform. The result is an upload-in, geolocated-damage-map-out pipeline the ministry's own team can operate.
The problem
After an earthquake, the ministry has aerial or satellite imagery of the affected area within hours, but no fast way to turn thousands of image tiles into a map of which buildings are damaged. Manual review does not scale to a disaster-response timeline. An off-the-shelf object detector will not work either: damaged buildings do not look like any general-purpose dataset's classes, and the imagery comes as raw image tiles with separate georeference metadata (UTM zone, upper-left corner, pixel size per tile), not GPS-tagged photos. What they needed was a model trained on their own damage imagery, a way to run it at the volume a real disaster produces, and output in a geographic format their GIS tools could use directly.
Constraints
- ·Custom detection classes (earthquake building damage), so a pre-trained general object detector was not usable as-is.
- ·Input arrives as image tiles plus separate georeference JSON (UTM zone, corner coordinates, pixel size), not geotagged images.
- ·Government client: infrastructure had to run inside their own AWS account, not a third-party SaaS.
- ·Needed a non-technical upload path for their own staff, not just an API for engineers.
- ·Single engineer building both the ML pipeline and the surrounding cloud infrastructure.
Architecture
Ministry staff upload a batch of image tiles and one georeference JSON through a web UI (ASP.NET Core, hosted on Elastic Beanstalk behind an Application Load Balancer) to an S3 bucket. A Lambda function finds the latest uploaded batch, invokes a SageMaker real-time endpoint running a custom Detectron2 model for every tile, reprojects the returned pixel-space bounding boxes into WGS84 coordinates using the tile's own georeference metadata, and writes the result back to S3 as GeoJSON. A second, separate Lambda only starts a SageMaker notebook instance used for training and exploratory analysis. It does not sit on the live detection path.
- ›Upload web UI (ASP.NET Core MVC) on Elastic Beanstalk, behind an Application Load Balancer
- ›S3 bucket holding uploaded image tiles, georeference metadata, and prediction output
- ›Lambda (sagemaker-executer): finds the latest batch, calls the SageMaker endpoint per tile, reprojects to WGS84, writes GeoJSON
- ›SageMaker real-time endpoint: custom Detectron2 container (FCOS meta-architecture, ConvNeXt-Large backbone)
- ›SageMaker notebook instance: training and EDA, started by a separate Lambda, not on the live request path
- ›Terraform modules: S3, Lambda, Elastic Beanstalk, SageMaker notebook
How one request flows
- 01Ministry staff upload a folder of image tiles plus one georeference JSON through the web UI, landing in S3 under a timestamped upload path.
- 02The prediction Lambda lists the bucket, finds the most recently uploaded folder, and downloads the georeference JSON for that batch.
- 03For each tile, the Lambda downloads the image, sends the raw pixel bytes to the SageMaker real-time endpoint, and gets back detected building boxes with class and confidence.
- 04The Lambda builds a GeoJSON feature per detection in pixel space, then reprojects every coordinate from the tile's UTM zone to WGS84 using the tile's own upper-left corner and pixel size.
- 05The reprojected GeoJSON for the batch is written back to S3 under a timestamped prediction path.
- 06An operator (or the download side of the web UI) pulls the latest prediction folder down locally or hands the GeoJSON straight to GIS tooling.
- 07Separately, when new training data lands in S3, a second Lambda starts the SageMaker notebook instance so the ministry's data team can retrain or re-evaluate the model.
Engineering decisions
Bring-your-own-container SageMaker endpoint
What. Packaged Detectron2, a custom `d2extensions` module, and the trained weights into a Docker image built from `continuumio/miniconda3`, deployed as a SageMaker real-time endpoint rather than using a built-in SageMaker algorithm.
Why. Detectron2 with a custom FCOS/ConvNeXt configuration and a custom extension module is not one of SageMaker's built-in frameworks. A bring-your-own-container endpoint was the only way to serve this exact model.
Tradeoff. More image-build and dependency work upfront (compiling Detectron2 from source, pinning CUDA-matched torch builds) than a managed algorithm would need, in exchange for using the exact model trained for this dataset.
Reprojection done in the Lambda, not the model
What. The SageMaker endpoint returns detections in pixel space. A separate step in the Lambda reprojects every polygon to WGS84 using each tile's own UTM zone and corner coordinates.
Why. Keeping the model's output format tile-agnostic (pixel coordinates) means the same endpoint works regardless of which UTM zone a given tile came from. Geographic reprojection is a per-tile metadata problem, not a model problem, so it belongs in the orchestration layer.
Tradeoff. Every batch must ship an accurate georeference JSON alongside the imagery. If that metadata is wrong, the model's detections are still correct but the map location will not be.
Latest-folder lookup instead of an explicit job id
What. The Lambda finds the batch to process by listing the bucket and taking the most recently modified upload folder, rather than being passed a specific batch identifier.
Why. It let the upload UI stay simple: staff drop a folder in and trigger processing without wiring up a job-tracking system.
Tradeoff. Only correct for one batch in flight at a time. Two uploads close together race on 'latest'. This is a real limitation of the delivered pipeline, not a hidden one.
Separate Lambda for the training-side trigger
What. A second Lambda (`lambda_start.py`) only starts the SageMaker notebook instance. It is not part of the request path that calls the inference endpoint.
Why. Training/EDA and real-time inference have different latency and cost profiles. Keeping them as two Lambdas meant a training run kicking off a notebook instance could never accidentally block or slow down a live detection request.
Tradeoff. None significant. This is a straightforward separation of concerns once the two paths were identified.
One deployed upload UI, chosen deliberately
What. Built the upload web UI in ASP.NET Core and wired it to Elastic Beanstalk in Terraform. A parallel Django implementation was prototyped alongside it.
Why. The ASP.NET Core app was the one that matched the client's existing Beanstalk environment and was carried through to deployment.
Tradeoff. The Django prototype ended up with one feature the deployed app does not have (triggering the prediction Lambda immediately after upload). That gap is real and is called out under What changed rather than left implicit.
What changed along the way
- →Started training and validating the model directly on a SageMaker notebook instance before packaging it as a real-time endpoint, to iterate faster on the ConvNeXt/FCOS configuration.
- →Built the upload UI twice: an ASP.NET Core app wired to the deployed Beanstalk environment, and a Django rewrite that added a straight-after-upload Lambda trigger. The Django version's automatic trigger was never carried back into the deployed ASP.NET app.
- →A folder-level dedupe check (skip a batch that was already processed) was written into the prediction Lambda but left commented out rather than enabled, so re-running against the same upload folder reprocesses it.
- →Validated S3 credential wiring in small standalone .NET console apps before wiring the same calls into the real upload UI, rather than debugging AWS SDK issues inside the full web app.
Security and reliability
- ·Runs entirely inside the ministry's own AWS account (SageMaker, Lambda, S3, Elastic Beanstalk), not a third-party ML SaaS.
- ·IAM roles scoped per Lambda function (separate roles for the endpoint-invoking Lambda, the notebook-starter Lambda, and the Beanstalk service role) rather than one shared role.
- ·SageMaker endpoint invocation and S3 access go through the AWS SDKs (boto3, AWS SDK for .NET), not hardcoded network calls.
- ·Known gap: the prediction Lambda has no explicit batch-id tracking, so two uploads in quick succession can race on 'latest folder'. Documented as a real limitation, not hidden.
Metrics
What shipped
- ✓Delivered a working detection pipeline: custom-trained Detectron2 model served as a SageMaker real-time endpoint, invoked from a Lambda that reprojects results to geolocated GeoJSON.
- ✓Ministry staff can upload imagery and retrieve geolocated damage predictions through a web UI, without needing AWS console access.
- ✓Full infrastructure (S3, Lambda, Beanstalk, SageMaker notebook) reproducible from Terraform.
- ✓Presented the pipeline and the broader ML approach directly to ministry stakeholders and AWS solution architects.
- ✓Handed over with the training path (SageMaker notebook, lifecycle script, training job launcher) kept separate from the live inference path so the ministry's team can retrain independently.
Lessons learned
- ·A bring-your-own-container SageMaker endpoint is worth the extra image-build work when the model needs a framework combination (custom Detectron2 extensions) no built-in algorithm supports.
- ·Keep geographic reprojection out of the model and in the orchestration layer, so the same endpoint serves tiles from any UTM zone.
- ·A 'process the latest folder' design is simple to build and demo, but needs an explicit batch id before it can handle concurrent uploads safely.
- ·Building a throwaway console app to validate SDK credentials before wiring them into the real application catches configuration mistakes cheaply.
- ·When two implementations of the same feature diverge (the Django prototype's auto-trigger versus the deployed ASP.NET app), track the gap explicitly instead of letting it go unnoticed.