Healthcare-data platform hardens AWS identity under live incident response

The platform’s analytics ran over healthcare-provider and medical-claims data at multi-billion-row scale, structured for years around BigQuery’s table model — and BigQuery’s cost model, which was charging warehouse storage rates for data accessed roughly twice a month. Moving to lake economics on AWS only works if the lake stays coherent: every dataset’s schema mapped faithfully from the Google source, structure discoverable rather than tribal, and the whole estate queryable by the product and its analysts on day one. Without a real catalog, a 200 TB lake is just an expensive pile of objects.

VeUP delivered the integration layer with AWS Glue at its center. The migration’s schema discovery and mapping translated each Google-side structure to its AWS target as the VeUP-built data mover carried the estate from GCS into Amazon S3; Glue crawlers then walked the landed data, inferring and registering structure into the Glue Data Catalog as the single authoritative registry of what the platform holds. That catalog is what makes the rest of the architecture work: Amazon Athena queries the S3 data through it — serverless analytics selected over a provisioned warehouse on the economics — while the Amazon RDS-backed web tier continues serving the customer-facing provider profiles alongside. The result is a lake with warehouse discipline: schemas versioned in the catalog, new datasets registered by crawler rather than by folklore, and every downstream consumer — product, analysts, reporting — reading from one definition of the data.