Six-month IBM-to-AWS migration, no user downtime, for a marketing SaaS with 3,500+ brands

Context

Cloud migration of an enterprise-scale product needs an engineering approach and specific expertise. Below we trace the process of adopting new cloud infrastructure and the results.

Challenge

Our client, a major marketing SaaS provider, had insufficient capacity and high costs on its IBM platform. Working with 3,500+ brands in 70+ countries, it helps marketing teams manage campaigns, promote products, generate leads, support sales and build brand awareness. Its customer base and user expectations keep growing, and a competitive market demands regular updates in response to new technology and trends.

The client concluded that the existing infrastructure could no longer cover its business needs. Given the high cost of IT resources, the partnership with the current cloud provider had lost its viability. After deep business analysis and a study of market trends, the client decided to change cloud platforms and asked us to advise on a scalable, cost-effective infrastructure and to carry out the migration with no downtime and no data loss.

Pre-development

We assigned an experienced business analyst to study the client’s business and the cloud market. The marketing software’s breadth of workloads called for a provider with advanced tools available on demand, and a data-security guarantee was another deciding factor.

We chose Amazon Web Services as the most experienced and enterprise-ready provider, with a market position and breadth of services that cover every business need.

Once AWS was agreed with the client, we assigned five people to the migration: project manager, solutions architect, system administrator, cloud security manager and compliance specialist.

Solution

We migrated from IBM to AWS in three stages: data migration, replication and security.

Data migration

• Data profiling. We removed inconsistencies, duplicate values and incorrect information from the existing database.
• Schema conversion. We used the AWS Schema Conversion Tool (AWS SCT) for the heterogeneous database migration.
• Migration build testing. This confirmed that all mappings, transformations and data-quality rules worked as intended.
• Migration execution. We ran the data migration over a weekend so software users were not interrupted.

Replication

We created several database instances, one acting as the primary and the others as replicas. The replicas were configured to pull the difference from the primary once a minute, so every instance stayed current.

If anything happened to the primary, one of the replicas took over and the remaining instances were redirected to the new primary. Even in the worst case, data loss was limited to one minute.

Security

In the cloud environment we carried out all the tasks needed to establish data security:

• Defined user permissions and identities
• Set up logging and monitoring services
• Configured the event management system
• Set up testing and auditing tools
• Automated incident response and recovery

Architecture and tech stack

The Java and Spring Cloud application moved from IBM DB2 to PostgreSQL on Amazon RDS via AWS SCT, running on Amazon EC2 with a primary-replica database topology. Hadoop, Kafka and Apache Flink carry the platform’s big-data and streaming workloads.

• AWS: Amazon EC2, Amazon RDS
• PostgreSQL (migrated from IBM DB2 with AWS Schema Conversion Tool)
• Java, Spring, Spring Cloud
• Hadoop, Kafka, Apache Flink

Results

We completed the application migration to AWS within six months. The client appreciated the dedicated work and the ability to meet every deadline, and confirmed the benefits of the AWS platform:

• Scalability. Software resources can be increased or decreased on demand.
• Security. AWS security tools give reliable protection against malware, DoS, SQL injection and other attacks.
• Flexibility. Resources scale up at peak load and back down afterwards.
• Reliability. High redundancy backs up and restores data on demand.
• Automation. Cloud resource management allocates the requested resources automatically, reducing the daily workload of corporate IT staff.

What made it work

Erbis’s takeaways from this engagement:

• Profile the data before you move it. Cleaning inconsistencies and duplicates on the source side made the schema conversion and the tests trustworthy.
• Rehearse the cut-over with a migration build. Testing mappings and data-quality rules in advance is what made a weekend migration possible.
• Replication is the downtime insurance. A primary-replica topology with one-minute sync bounded the worst case before the switch, not after.
• Staff security and compliance from day one. A cloud security manager and a compliance specialist on a five-person team meant the new environment was governed at launch.