Red Ventures’ Home vertical, which runs Allconnect and MyMove, had grown four separate data stores instead of one source of truth. Data flowed into an Oracle warehouse through SAP BusinessObjects, and separately into Redshift through AWS DMS, with Tableau pulling from both. The result: 89 BI reports that often disagreed with each other, a 60 to 90 minute build time for anything new, and up to four hours just to get an answer.

Underneath it all sat tens of thousands of lines of undocumented Oracle and SAP logic that only outside consultants really understood. Red Ventures already had a mandate to shut down Redshift for Databricks SQL, so every month of delay meant paying for two platforms and losing ground on a projected 40% cost savings and 50 to 60% latency improvement.

Red Ventures owned platform setup, access control, and user enablement. Lovelytics owned the migration itself: moving the data, rewriting the transformation logic, proving it matched the legacy systems, and repointing reporting onto the new platform.

The team migrated 143 Redshift tables and roughly 70 stored procedures and SQL scripts, plus the equivalent Oracle logic, into Delta tables on Databricks. They rewrote the legacy ETL as modular PySpark and Spark SQL orchestrated through Databricks Workflows, then validated a 99% source-to-target match before a single report was cut over.

Red Ventures set an ambitious goal: retire two legacy platforms and unify Home’s fragmented reporting on Databricks in just four months. Lovelytics joined from September to December to help make that goal real, but the mission belonged to Red Ventures from the start.

Migrating the Data and Rewriting the Logic

Red Ventures’ Home business, spanning ACU/ACD, Frontier, MyMove, and Energy, depended on 143 Redshift tables, around 70 stored procedures and SQL scripts, and 60 to 80 Tableau data sources.

Together with Lovelytics, the team lifted reporting tables and history from Redshift and Oracle into Delta on Databricks, careful to preserve existing naming conventions so nothing downstream would break. To move fast without sacrificing quality, the team built a unified script that generated ingestion and merge code for every table.

One engineer used it to deliver historical loads, incremental pipelines, and orchestration for 50 reporting tables in a week and a half. The stored procedures that only outside consultants once understood were rewritten as modular PySpark and Spark SQL notebooks, giving Red Ventures’ own engineers logic they could finally read, trust, and maintain.

Proving the Match Before Cutover

Red Ventures wasn’t willing to cut over until the numbers were proven right. The team set the bar at 99% source-to-target accuracy and built automated validation to hold every migrated table to it, running checks on a schedule throughout a parallel testing window while the business kept operating on the legacy systems.

Every defect was tracked and worked down together, and when the validation tool was ready, it was handed to Red Ventures’ own developers and stakeholders so they could see the proof for themselves.

Governance and Handoff

With Unity Catalog enabled across every workspace, Red Ventures gained automatic lineage, consistent access control, and full audit history for the first time. Tableau reporting was repointed onto the new Databricks tables, including the SQL conversions needed to keep existing workbooks running without a rebuild.

In the final weeks, the focus shifted to the people who would own this going forward. Knowledge transfer put the pipelines back in Red Ventures’ hands, ending the reliance on consultants and giving the internal team full control of the systems it depends on every day.

Red Ventures needed more than a vendor. With a hard deprecation deadline looming, they needed a partner willing to take full ownership of the migration: rewriting tens of thousands of lines of undocumented logic, proving every table matched, and handing the result back to a team that had never worked in this codebase on Databricks before. Lovelytics brought the hands-on migration expertise to make that possible, and just as important, the willingness to work in the open. Running validation transparently alongside Red Ventures’ own developers turned what had been a stalled, consultant-owned ETL layer into something the internal team now fully controls.

Delivered

• Redshift and Oracle Exadata reporting workloads migrated to Databricks on AWS in roughly four months, on schedule, against a hard Redshift deprecation mandate.
• 99% source-to-target data match validated before cutover, with automated validation continuing through parallel testing.
• Reporting consolidated onto a single governed source of truth, closing the discrepancies between the Oracle and Redshift versions of the same reports.
• Tens of thousands of lines of opaque stored procedure and package logic replaced with modular, documented PySpark and Spark SQL that the internal team owns and can maintain.
• Full lineage, access control, and audit through Unity Catalog, replacing an ETL layer that had no lineage at all.
• Dependence on external SAP ETL consultants removed — Red Ventures controls its own transformation logic.
• Platform now positioned for advanced analytics and AI use cases the fragmented estate could not support; Red Ventures went on to engage Lovelytics for generative AI advisory work.

Projected by Red Ventures (from their own internal modernization business case — not post-migration measurements):

• 40–50% reduction in infrastructure and operational cost from retiring Redshift, Oracle, and SAP.
• 50–60% lower query latency on Databricks SQL versus Redshift.
• 70–80% reduction in time to produce a new report.
• 40–50% faster time to market for new products.
• 60% improvement in the effort required to manage the ETL and reporting codebase.
• Support model reduced from multiple offshore engineers to two data engineers.

Red Ventures is carrying that momentum straight into its remaining migration work. With reporting now sitting on a single governed platform, the team has already brought Lovelytics back for a separate engagement on generative AI advisory work, eager to see what else the new foundation can support.