It's important to note down the Virtual Private Network (VPC) information for both source and target database. In the example, we have
- MySQL RDS database in vpc-9da2f3f5
Version: 5.6.23
DB Instance: mysql-rep-test
DB Name: GGMYSQL
User: oggadm
Endpoint: mysql-rep-test.copyxiqzdjjl.us-west-2.rds.amazonaws.com:3306
Security Group: rds-launch-wizard (sg-7299fa16) - Aurora RDS database in vpc-9da2f3f5
Version: Aurora 5.6.10a
DB Instance: pmaurora (pmaurora-us-west-2b)
User: oggadm
Cluster End Point: jinyu-aurora.cluster-copyxiqzdjjl.us-west-2.rds.amazonaws.com:3306
Security Group: rds-launch-wizard-5 (sg-ea7ab891)
For MySQL RDS database, you need to create a customized parameter group, enable binary logging, change the binary log format to ROW format, and let the the binlog_checksum=NONE. You might need to reboot the MySQL instance after the parameter change.
Please update the MySQL database parameter group.
After having the source and target databases ready, You can start the migration by clicking on the Create Migration button on the home screen of the AWS DMS service. You will then see a Welcome page explaining What the AWS DMS is. Click on the Next button after you understand the service.
When setting up the replication instance, you need to specify its VPC and other options.[1] Note that if you like the replication instance to migrate data outside of the VPC, you have to make it Publicly accessible. After you click on the Next button, the replication instance is then being created.
While the replication instance is being created, you can create the source and target database endpoints. You can test for each endpoint. To enable success test, you need to modify the MySQL and Aurora RDS database security group to accept the connections from all IPs. (0.0.0.0/0). After the replication instance is created, you can whitelist the replication instance with its IP address. Next, you can create the migration task.
$ mysql -h mysql-rep-test.copyxiqzdjjl.us-west-2.rds.amazonaws.com -P 3306 -u oggadm -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 35 Server version: 5.6.23-log MySQL Community Server (GPL) Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use GGMYSQL Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> describe TCUSTMER; +-----------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+-------------+------+-----+---------+-------+ | CUST_CODE | varchar(4) | NO | PRI | NULL | | | NAME | varchar(30) | YES | | NULL | | | CITY | varchar(20) | YES | | NULL | | | STATE | char(2) | YES | | NULL | | +-----------+-------------+------+-----+---------+-------+ 4 rows in set (0.09 sec) mysql> select * from TCUSTMER; +-----------+------------------+---------+-------+ | CUST_CODE | NAME | CITY | STATE | +-----------+------------------+---------+-------+ | JANE | ROCKY FLYER INC. | DENVER | CO | | WILL | BG SOFTWARE CO. | SEATTLE | WA | | WILS | BG SOFTWARE CO. | SF | CA | +-----------+------------------+---------+-------+ 3 rows in set (0.10 sec) mysql> insert into TCUSTMER values ('jinyu', 'OGG Replication','Austin', 'TX'); Query OK, 1 row affected, 1 warning (0.10 sec) mysql> commit; Query OK, 0 rows affected (0.10 sec) mysql> select * from TCUSTMER; +-----------+------------------+---------+-------+ | CUST_CODE | NAME | CITY | STATE | +-----------+------------------+---------+-------+ | JANE | ROCKY FLYER INC. | DENVER | CO | | jinyu | OGG Replication | Austin | TX | | WILL | BG SOFTWARE CO. | SEATTLE | WA | | WILS | BG SOFTWARE CO. | SF | CA | +-----------+------------------+---------+-------+ 4 rows in set (0.10 sec)
When using the AWS DMS, the first impression is that it is not for high volume data replication. For a 5-rows table, the initial loading takes 0.5 minutes with the best replication instance configuration (dms.c4.4xlarge). Each CDC update then takes around 1-2 seconds. The test is based on databases and the replication instance in the same region. If the databases and the replication instance are in different regions, the performance will be further impacted by the network condition.
The "hub" architecture used by AWS DMS simplifies the deployment. However, for the source/targets with long distance, keeping the replication instance close to the databases and sending data with network optimizations are needed for better performance. For example, when configuring Oracle GoldenGate, Oracle GoldenGate on EC2 instances are configured on the source and target respectively to perform the replication. Oracle GoldenGate extract pump then sends the data over the network with data compression and TCP/IP communication optimizations.
AWS DMS is easy to use in general. You can configure database endpoints and start the replication task quickly. The monitoring UI also well presented the process status. These makes people like to use it. Recently, it is announced passing 22K database migrations. (Tweet 3/27/2017). However, the selection, filter and transformation UI need to be more intuitive.
- Amazon Documentation -Using a MySQL-Compatible Database as a Source for AWS Database Migration Service
- Amazon Documentation - Accessing MySQL Binlog
- Amazon Documentation - Getting Started (with DMS)