2024 · Migration
Migrating a live Cassandra datastore between clusters
a consumer marketplace running on Cassandra · Data migration engineer
The problem
A live database had to move to a new cluster without losing a row. The data had complex shapes and new writes kept arriving.
What I built
I moved it table by table with a migration tool, kept expiry times, and checked row counts on both sides. The hard part was the data shapes, which I mapped carefully.
The result
Row counts matched on both sides after the move, and the application kept running on the new cluster.
Main tools
- Apache Cassandra
- Apache Spark
- DataStax migrator
- Amazon EKS
This is the short version. The full case study is available on request.