GPU Model Hosting & Presigned Upload Pipeline
A Presigned Upload Pipeline for a Custom GPU Model
A team had a working custom model, packaged as their own Docker image, and needed a way for users to upload files that model would process, without routing large media through an application server. I built the upload path and the GPU hosting around it: a presigned-URL API on Lambda and API Gateway, direct client-to-S3 uploads, and a private GPU EC2 instance behind ECS running the client's container image, picking up new files from S3 on a schedule.
The problem
The client had a model ready to run, their own container image, but no infrastructure to receive real file uploads and get them to that model. Proxying uploads through a web server ties request duration and memory to file size, which does not work well for video and large images. They needed callers to push files directly into storage, a GPU instance to run their model on, and a way to connect the two without exposing the model host to the internet. A managed inference service was not the fit, the model is their own, self-hosted, and not something to hand to a third-party endpoint.
Constraints
- ·The model requires a CUDA-capable GPU instance, ruling out Lambda or a CPU-only container for inference.
- ·Large media uploads had to bypass the application tier entirely, not be proxied through it.
- ·The GPU host sits in a private subnet with no path for anything outside the VPC to push data into it directly.
- ·The model is the client's own container image, so the platform had to host it as-is, not replace it with a hosted API.
Architecture
A client requests an upload slot from an API Gateway endpoint, a Lambda validates the file name and returns a presigned S3 PUT URL, and the client uploads the file directly to S3. Because the GPU host lives in a private subnet with no inbound path from outside, a cron job inside the model's own container image polls S3 on a schedule and pulls new files down to run the model against. The GPU workload runs as a single fixed ECS task on a dedicated g5.xlarge EC2 instance, with all CUDA devices exposed to the container. A bastion host in the public subnet is the only way into the private network for an operator.
- ›API Gateway: two stages seen in source, one behind a Cognito client-credentials (M2M) authorizer, one with no auth, both calling the same Lambda
- ›Lambda (presigned-url.py): validates the file name against path traversal, generates a presigned S3 PUT URL with a configurable expiry
- ›S3 bucket: receives the upload directly from the client, never passes through the Lambda
- ›GPU ECS cluster: one fixed g5.xlarge EC2 instance (auto-scaling group pinned at 1, managed scaling disabled), running a task named model-container from a private ECR image
- ›Cron job inside the container image: polls S3 on a schedule and pulls new files, since the private GPU host cannot be reached directly from outside
- ›VPC: 2 public and 2 private subnets across 2 AZs, NAT gateways, an internet gateway
- ›Bastion host: small EC2 instance in the public subnet for SSH access into the private network
- ›ALB: provisioned with an HTTP:80 listener, not attached to the GPU service's target group in the traced code, an HTTPS listener and ACM cert exist commented out
- ›RDS Postgres module: written (400GB gp3, db.t3.medium) but commented out of the root Terraform, not part of the deployed stack
How one request flows
- 01Client calls the API Gateway with the name of the file it wants to upload.
- 02The Lambda validates the file name (rejects empty names, and any containing '/' or '..') and builds the S3 object key, prefixing it with a configured folder path if set.
- 03The Lambda calls S3 to generate a presigned PUT URL with a time-boxed expiry (1800 seconds by default) and returns it.
- 04The client uploads the file directly to S3 using that URL. The file bytes never touch the Lambda or any application server.
- 05A cron job inside the GPU container's own image polls S3 on a schedule and pulls down new files.
- 06The container runs the client's model against the file on the GPU instance.
- 07An operator can reach the private subnet through the bastion host for maintenance, without the GPU instance itself being reachable from the internet.
Engineering decisions
Direct-to-S3 uploads instead of a proxy
What. The Lambda only validates the file name and signs a URL, the client uploads straight to S3.
Why. The upload path scales with S3 instead of a compute tier that would otherwise hold every file's bytes in memory during the request.
Tradeoff. The alternative, streaming the upload through an API endpoint, ties request duration and memory to file size for no real benefit here.
A scheduled pull instead of an event trigger
What. A cron job inside the GPU container polls S3 for new files rather than being invoked by an S3 event.
Why. The GPU host sits in a private subnet with no path for S3 to reach it directly, so it has to go get the file instead of waiting to be handed one.
Tradeoff. This trades instant pickup for a polling delay, acceptable for a batch-style workload, not something to present as a real-time pipeline.
A fixed single GPU instance, not an autoscaled fleet
What. The ECS capacity provider has managed scaling explicitly disabled and the auto-scaling group is pinned at exactly one instance.
Why. A GPU instance is expensive to keep idle, and the traffic this served did not justify a scaled fleet.
Tradeoff. Chose one right-sized instance over premature autoscaling infrastructure for a workload with irregular, low volume.
Filename validation instead of full collision protection
What. The Lambda blocks path traversal in the requested file name but does not deduplicate or namespace names beyond an optional folder prefix.
Why. Stopping a caller from writing outside the intended S3 prefix was the actual security requirement.
Tradeoff. This is not collision protection, two callers uploading the same file name will overwrite each other. Worth stating plainly rather than calling it complete.
A bastion instead of exposing the GPU host
What. The GPU instance and the (undeployed) database sit in private subnets with no public IP, reached only through a small bastion host.
Why. Keeps the instance actually running the model unreachable from the internet, while still giving an operator a way in.
Tradeoff. None significant, a bastion is the standard tradeoff of one extra hop for real network isolation.
What changed along the way
- →SageMaker was evaluated for hosting the model and dropped in favor of a self-managed GPU container, since the model is a client-owned image, not something to rebuild for a managed service.
- →An earlier, unauthenticated version of the upload endpoint exists in source alongside the Cognito-protected one. The two were not reconciled into a single gated path, a real gap rather than an intentional fallback.
- →An application load balancer and an RDS Postgres instance were designed into the Terraform for a larger version of this stack. The load balancer deploys but is not attached to the GPU service, and the database module is commented out, so neither is part of what runs today.
Security and reliability
- ·The intended upload path requires an OAuth2 client-credentials token from Cognito before the API issues a presigned URL.
- ·The Lambda rejects file names containing '/' or '..' before building the S3 key, blocking path traversal outside the configured prefix.
- ·Presigned URLs expire (1800 seconds by default), limiting the window a leaked URL could be used in.
- ·The GPU instance runs in a private subnet with no direct internet exposure, reachable only through the bastion host.
- ·Separate IAM roles for the GPU EC2 instance, the ECS execution role, and the ECS task role, rather than one shared role.
Metrics
What shipped
- ✓Delivered a working upload-to-inference pipeline: presigned S3 uploads feeding a custom GPU model running in a private container on ECS.
- ✓The client's own model image runs unmodified, this project owns the hosting and the upload path around it, not the model itself.
- ✓Private networking with a bastion for operator access, scoped IAM per component, and time-boxed upload URLs.
- ✓A clear, honest map of what is deployed versus designed: the load balancer and database modules exist in code for a larger version of the stack but are not part of what runs today.
- ✓A reproducible infrastructure base that a future load balancer or database rollout can attach to without a rebuild.
Lessons learned
- ·When a compute target cannot be reached from outside its network, a scheduled pull is a simpler and equally valid answer to an event trigger, at the cost of latency.
- ·A fixed single GPU instance is often the right call for irregular workloads, autoscaling a GPU fleet is worth the complexity only once utilization justifies it.
- ·Infrastructure written for a larger future version of a system should be marked clearly as not-yet-live, otherwise it reads as deployed when it is only planned.
- ·Two versions of the same endpoint, authenticated and open, are a gap, not a fallback. Reconciling them early avoids leaving an unauthenticated path live by accident.