Healthcare-data platform hardens AWS identity under live incident response
The platform runs commercial intelligence over healthcare-provider and medical-claims data at multi-billion-row scale — roughly 200 TB that its analysts touch about twice a month. VeUP migrated that estate off Google BigQuery onto an AWS-native data lake: a purpose-built GCS-to-S3 data mover, AWS Glue cataloging, and Amazon Athena query — an architecture chosen over Amazon Redshift because the workload’s access pattern demanded lake economics, not warehouse economics.
The challenge
The platform had hit a cost wall familiar to data-heavy SaaS companies: warehouse-resident storage priced for hot analytics, holding data that was queried roughly twice a month. Per-table storage charges accumulated on hundreds of terabytes of largely cold healthcare-claims history — recurring spend delivering no recurring value. Simply switching warehouses would have moved the problem, not solved it. The estate needed an architecture matched to its true access pattern, a safe path to move ~200 TB across clouds without disrupting the SaaS product, and a query layer that kept the data fully analyzable on demand.
The solution
VeUP engineered the migration as three deliberate moves. First, transport: a VeUP-built managed data mover carried the estate from Google Cloud Storage into Amazon S3, with schema discovery and mapping translating the Google-side structures to their AWS targets — repeatable, resumable, and verifiable at 200 TB scale. Second, structure: AWS Glue crawlers and the Glue Data Catalog cataloged the landed data, turning object storage into governed, discoverable tables. Third, query: Amazon Athena serves the analytics — selected over Amazon Redshift after VeUP walked the customer through the trade-off, since serverless per-query pricing beats provisioned-warehouse economics for data touched twice a month, while remaining fully capable at multi-billion-row scale. The Amazon RDS-backed web tier serving provider profiles runs alongside the lake, unchanged from the product’s perspective.
Production outcomes
| KPI | Result |
|---|---|
| Migration scale | ~200 TB of multi-billion-row healthcare analytics data moved off the non-AWS warehouse into the S3 lake via the purpose-built data mover, with schema mapping preserving analyzability end to end. |
| Economics realigned | Idle warehouse storage charges on twice-a-month data eliminated in favor of S3 storage tiers plus pay-per-query Athena — cost now tracks actual usage instead of data mass. |
| Architecture decision | Athena selected over Redshift through an explicit, customer-facing trade-off analysis — the platform gets serverless analytics with no cluster to size, pay for, or babysit. |
| Lessons & continuation | Match the engine to the access pattern, not the data size — cold multi-terabyte estates belong in a cataloged lake with serverless query; and owning the transport tooling is what makes a cross-cloud move of this scale controlled rather than heroic. |
Amazon S3 (data lake) · AWS Glue (crawlers + Data Catalog) · Amazon Athena (serverless query) · Amazon RDS (profile web tier) · Amazon CloudWatch