In this session, learn how to move your existing database applications to the cloud. We cover the best practices for planning your migrations, moving your data over, sizing your AWS deployment appropriately, and minimizing downtime. You also hear from some of our customers who have successfully migrated their applications about the techniques they used and the reasons they moved onto the cloud.

10. Internet
VPC
IAM

11. Sync
Replication
Availability Zone 1 Availability Zone 2

14. On-Premise AWS

15. T2 T1 T2
T1
Apply Delta Changes
On-Premise AWS

17. Sync
Replication
QA
Sync
Replication
Production
Development

18. Data Import Guide for MySQL
Data Import Guide for Oracle
Data Import Guide for SQL Server

19. Big Future: Lessons Learned Delivering Our
First Million+ Page View/Day Hybrid Cloud
Solution
Rodney Grilli, Solution Architect, The College Board
November 28, 2012

20. Audience Takeaways
• How to incorporate Amazon's growing number of value-added services as
you deploy new products to AWS
• Criteria for choosing between RDS and self-managed MySQL on EC2
• How to setup and size RDS
• How to migrate existing on-premise databases to RDS
• People, Process, and Technology Lessons that you need to know before
starting your journey

21. Who We Are….

22. Our Goal
Ensure that every student has the opportunity to prepare for, enroll in, and graduate from
college. Our work falls broadly into three categories:
College Readiness College Connection and Success
– Advanced Placement (AP®) • SAT, SAT Subject Tests™ and SAT Readiness Tools
– SpringBoard® • Student Search Service
– ReadiStep™ • bigfuture (our college planning site)
– CollegeEd® • College Search
– CLEP® • CSS/Financial Aid PROFILE®
– PSAT/NMSQT® • Scholarship Search
– EXCELerator™
Advocacy
• Advocacy & Policy Center

23. The Big Future Initiative….

24. Purpose
A comprehensive and innovative online resource that helps all students
aspire, find, connect, and enroll in the right college that sets them on the
path for success in life.
Vision
Be the most used college guidance website, recommended as the top
choice by all students and counselors, and known as the most helpful and
usable source of trusted information and tools.

25. The IT Challenge….

26. • Develop a new product that satisfies all new business goals
– Replacing core features in over three legacy products
– Targeting AWS for the deployment of a number of components
• Migrate all user data from legacy products to new product
– This included College Lists for ~10 million students
• Maintain all user flows from legacy and dependent products to
target locations in the new product
– Legacy products process around 1 million pageviews per day
– Required extensive redirect rules
• Execute the migration in the peak business season

27. The Product….

29. 35
Number of new or extended assets?
24%
Percentage of College Board pageviews?
23%
Percentage of College Board unique daily visitors?
450
Number of Web Service calls/second executed by bigfuture?

30. The Architecture….

31. Architecture Approach
• Use established cloud and SOA principles to create new
system patterns that are optimized for AWS
• Design the product to support degraded modes of
operation
• Conduct sourcing analysis on all architectural
components to maximize optimization at all levels of the
solution

32. 17 Technical Assets 9 Processes Developed
Developed to Pilot State or Extended to Pilot State
• Web Service • AWS Logging • Capacity Management &
Management System • AWS Testing Capability Planning
• AWS Composite • Security Posture & Testing
• AWS Core Services
Application Hosting • Change Management
• AWS Storage • Release Management
Stack
• AWS Content Delivery • Configuration Management
• AWS Business Service
• AWS Disaster Recovery • Deployment Management
Hosting Stack
• Search Index (SOLR) • Automated Test Process
• AWS Load Balancing • Operations & Support
Deployment Pattern
• AWS Service Runbooks
• SEO Warming for New
Management • AWS Training
Domains
• AWS Database Service
• Site Crawler Handling
• AWS Communications for AJAX Heavy Sites
• AWS Monitoring

33. AWS Components
• Amazon Route53 • Elastic Load Balancers
• Amazon Machine • Security Groups
Images (AMIs) • Amazon S3
• Auto Scale Groups • CloudWatch
• AWS Identity & Access
Management (IAM)
• Amazon RDS (MySQL)

35. RDS Use Cases….

38. Migrating the Data….

39. College Planning Database
The biggest table has about 45M rows and the rest are below 10M

40. College List Migration Steps
Step Migration Activity Estimate Timing Notes
Executed during maintenance
1 Run Extract Script < one hour
window
2 Transfer extract files to jumpbox in AWS ~30-60 minutes Use SCP to securely transfer files
Truncate tables in existing production RDS
3 ~ 5 minutes Remove test data
DB
4 Run SQL Load for Account ~1 hour Run 5 file imports in parallel
5 Run SQL Load for College Lists ~4 hours Run 5 file imports in serial
6 Update the date to current timestamp ~ 35 minutes
Take Snapshot of RDS DB and store as Performed on 3/16 cold back up run
7 ~ 3 minutes
contingency and 3/19 final run

41. Why Choose RDS….

42. Advantages
• High Availability through the Multi-AZ DB feature
– The biggest advantage is the stability of the MySQL replication within RDS
– This has been the biggest killer in the past when maintaining MySQL
replication
• MySQL is simple to set up in RDS
– Other commercial RDBMSs require many custom configurations that must
be tweaked (e.g., memory configuration & I/O configuration)
• Less work to maintain

43. Constraints
• The biggest perceived constraint is not having access to log files
– Limited access to performance info that DBAs traditionally derive from log files
• Inability to gain the “Super” privilege
• Limited set of configuration settings
• Time zone is set to UTC and can't be changed
– This presents a challenge for applications set to use a different time zone that use
MySQL date/time functions

44. RDS Benefits for Big Future
• Removed the need to develop a self managed MySQL relational
database capability
– Allowed us to focus more effort on the 16 other assets that had to be developed
• Provides highly available, Multi-AZ, MySQL deployment model
– Removed the need to manage replication within MySQL
• Streamlined the deployment of the web service gateway and
new business service databases

45. How working with RDS is
Different….

46. Activity MySQL RDS
Create, Modify, Reboot, OS Command RDS Console, RDS Command Line Interface, APIs
Delete DB Instances Line
Monitoring Parse Log Files CloudWatch Console, Command Line Interface, APIs
Create, Modify, Delete Mysql client or Mysql client or mysql admin
Databases mysql admin
Tuning, DB Parameter Edit my.cnf RDS Console, RDS Command Line Interface, APIs
Changes (not all parameters can be changed)
Identify slow queries Parse Log Files Enable and query the mysql.slow_log table
Detect locking or SHOW STATUS Enable GoSH and query the global_status_history table
memory issues statement
RDS Users Guide - Appendix: Common DBA Tasks for MySQL
http://docs.amazonwebservices.com/AmazonRDS/latest/UserGuide/Appendix.MySQL.CommonDBATasks.html

47. RDS Roadmap….

48. What we want to do next
• Fully automate RDS provisioning via a Continuous Delivery Pipeline currently
under development
• Strengthen DB security by implementing SSL between components
• Extend database pattern to support multi-region deployments for both read-only
and read/write databases
• Continue to optimize the cost associated with our RDS databases
How RDS can help
• Provide multi-region replication and management features
• Optimize for SSL connections and X509-based authentication/authorization

49. Conclusion
• Leveraging value-added services such as RDS can significantly reduce
your products’ time to market and ongoing operating costs
• RDS is managed differently than a traditional MySQL database, so
developers/DBAs need to be educated on these differences
– Study the many RDS Documents and keep a copy of the Quick Reference Guide
• As with any selection process, you must evaluate the people, process,
and technology aspects of each option to ensure you select the best
solution for your specific needs