<p>The company's existing SQL server had performance issues that caused database deadlocks. The performance issues also resulted in higher maintenance of server environments and impacted other scheduled jobs. They also noticed some bottlenecks from the reporting and ETL (Extract, Transform, and Load) layers. Further investigation uncovered infrastructural issues with static CPU usage and memory limitations.</p> <p>The company's current infrastructure had many dependencies that limited them from achieving a real-time view of their data. For example, end users wishing to obtain a new service-level agreement (SLA) had to wait up to 15 minutes.</p> <p>They wanted to migrate all their components to a cloud environment that is capable of handling massive amounts of data and performing multiple processes simultaneously.</p> <p>Several challenges were keeping the company from transitioning to the cloud. For instance, data technicians encountered huge data quality issues when attempting to export data from the company's SQL server systems. Another challenge was the extreme complexity involved when trying to implement the company development team's SQL Server Integration Services (SSIS) packages in Google Cloud – the company’s cloud platform of choice. To sum it up, the company was looking for:</p> <ul><li><strong>Accurate data:</strong> Latency and data accuracy were significant drawbacks in the company's SQL server implementation. By removing the dependency on SQL Server, the company can move towards a platform that can enhance their data and provide rich analytics and highly accurate reports.</li> <li><strong>Fast access:</strong> The company wanted to improve their ETL performance and improve query speeds.</li> <li><strong>Data uniformity:</strong> Current data structures were rigid and inconsistent. The company wanted to employ a flexible data model and regain mastery of their data by standardizing its KPIs.</li> <li><strong>Cost control:</strong> When moving to a cloud environment, the company needed to control costs. They wanted to reduce the number of bytes read by a query. Further, they were looking for a way to de-normalize data for reporting needs to fetch data in the smallest time interval possible to minimize usage costs. </li> <li><strong>Modern analytics and reporting:</strong> The company's current SQL Server Reporting Services (SSRS) reports and their self-service and data visualization services suffered performance issues. The existing tools were also tricky to use and lacked up-to-date analytics capabilities.</li> </ul>