Opt in to Web Notifications to stay up-to-date.
Migrated 350 million records (700+ GB) from salesforce to AWS MySQL
Large data migration, including daily ETL processing of 10+GB of data, for an American multi-industry publishing conglomerate.
- There were two source systems primarily – one to hold subscription details and another to hold customer details. There were 20+ tables in these two systems
- There were few other systems such as Demand Tools which was used for cleansing and data deduping
- Millions of rows of data were generated daily between these two systems and this was pushed into Salesforce through a nightly batch process after applying different transformations
- Domo was the platform used to meet the reporting needs of the various stakeholders
- Salesforce license was expiring within 3 months of the project initiation
- Over 350GB of data from Salesforce had to be migrated to the new environment
- Salesforce had complex transformation rules (50+ Apex Triggers) which had to be understood and replicated
- There was no prior documentation available, making the understanding of business rules complex
- All the historical data (350+GB) is migrated from Salesforce into AWS
- All the data transformation rules used in Salesforce have been replicated in AWS
- Data from the two primary source systems and other systems (as appropriate) are now flowing seamlessly into AWS daily