AWS – Migration of On-Premises Infrastructure to AWS Cloud

The Migration Process can be split into three parts:

 

 

Before AWS Migration

 

During AWS Migration

 

After AWS Migration

 

 

 

 

AWS Migration: 5 Cloud Migration Steps

 

 

These are the 5 principal AWS Migration steps you need to consider:

 

Planning and Assessment
Migration Tools
AWS Cloud Storage Options
Migration Strategies
Application Migration Options

 

Planning and Assessment

 

The planning and assessment phase is divided into:

 

Financial Assessment
Security & Compliance Assessment
Technical and Functional assessment

 

Financial Assessment

 

Before deciding on-premises to cloud migration, you need to estimate the cost of moving data to the AWS cloud. A careful and detailed analysis is required to weigh the financial considerations of on-premises center versus employing a cloud-based infrastructure.

 

Security and Compliance Assessment

 

Overall risk tolerance
Main concerns around availability, durability, and confidentiality of your data.
Security threats
Options available to retrieve all data back from the cloud

 

Classify your data according to these concerns. This will help you decide which datasets to move to the cloud and which ones to keep in-house.

 

 

Technical and Functional Assessment

 

Assess which applications are more suited to the cloud strategically and architecturally.

 

Points to consider:

 

Which applications or data should best be moved to the cloud first?
Which data can we transfer later?
Which applications should remain on-premises?
Can we reuse our existing resource management/configuration tools?
What do we do about support contracts for hardware, software, and networking?

 

For small-scale data migrations

 

Unmanaged Cloud Data Migration Tools

 

For simple, low-cost methods for transferring smaller volumes of data:

 

Glacier command line interface- On-premises data → Glacier vaults
S3 command line interface- Write commands → Data moves directly into S3 buckets
Rsync- Open source tool combined with 3rd party file system tools. Copy data directly → S3 buckets

 

 

For large-scale data migrations

 

AWS Managed Cloud Data Migration tools

For moving larger volumes of data:

how much data to migrate? Which AWS data migration tool is best suited

Migrate petabytes of data in batches to the cloud AWS Import/Export Snowball
Migrate exabytes of data in batches to the cloud AWS Snowmobile
Connect directly to an AWS regional data center AWS Direct Connect
Migrate recurring jobs, plus incremental changes over long distances Amazon S3 Transfer Acceleration

 

 

Some Practical Strategies for AWS Migration

 

Forklift Migration Strategy

 

This is more suitable for self-contained, tightly-connected or stateless applications. Its a “pick up everything and move it in one go to the cloud” method.

Is best suited to smaller environments.

 

Hybrid Mixed-Migration Strategy

 

This involves moving some parts of an application to the cloud while leaving other parts of the application on-premises.

 

It is best suited to migrating larger systems which run multiple applications. However, it can be more time-consuming to complete the migration in this way.

 

 

Configuring and Creating AMI Images

 

AMIs provide the information needed to launch an EC2 instance.

 

 

Online data transfer from on-premises to AWS

 

Here are the online data transfer options.

 

 

AWS Virtual Private Network

 

There are two options for using AWS VPN:

 

 

AWS Site-to-Site VPN
AWS Client VPN

 

AWS VPN is encrypted, easy to configure and cost-effective for small data volumes. However, it is a shared connection, so not as fast or reliable as other options.

 

 

AWS Virtual Private Network (AWS VPN) establishes secure private connection your network to AWS.

 

 

 

AWS VPN is encrypted, easy to configure and cost-effective for small data volumes. However, it is a shared connection, so not as fast or reliable as other options.

 

 

 

AWS Database Migration Service

 

 

The AWS Database Migration Service as the name suggests handles database migration to AWS. The big advantage of DMS is that the database remains fully operational and usable during the migration.

 

AWS S3 Transfer Acceleration

 

To migrate large quantities of data over longer distances to AWS S3, AWS S3 Transfer Acceleration enables you to do this 50-500% faster yet still using the public internet.

 

Data is routed to S3 via optimized network paths using Amazon CloudFront Edge Locations situated across the globe. This maximizes available bandwidth. You select this service on the S3 Dashboard console, selecting one of two TA options. The transfer acceleration is then activated without any need for special client applications or additional network protocols.

 

AWS DataSync

 

AWS DataSync enables users to automate the migration of on-premises storage to S3 or Amazon EFS and can transfer up to 10 times faster than some open source migration services. It deploys an on-premises software agent which connects to your on-premises storage system via NFS (Network File System) and SMB (Server Message Block) protocols.

 

DataSync also takes care of much of the transfer overhead such as Running instances, encryption, managing scripts, network optimization, and validating data all while transferring data up to 10 times faster than many open source migration services.

 

It can be used to copy data via AWS Direct Connect or public internet to AWS, and is suitable for both one-time data migration, and recurring workflows, as well as for automated backup and recovery actions.

 

 

AWS Direct Connect

 

AWS Direct Connect is a dedicated connection from your on-premises to AWS.

 

As with AWS VPN, Direct Connect provides an encrypted connection between your on-premises environment and AWS.

 

However, Direct Connect does not use the public internet and instead runs via a private connection it establishes which will be either via a 1 GB or 10 GB fiber-optic Ethernet cable used to connect your router to an AWS Direct Connect router. On other words, the Direct Connect solution is part software and part hardware.

 

Because of this dedicated connection, Direct Connect is significantly more costly than using just public internet-and-VPN solutions.

 

But if you need to transfer or stream very large amounts of data back and forth to the AWS Cloud, then a Direct Connect line may be the best solution. However for smaller transfer one-off migrations it is not so suited.

 

 

AWS Storage Gateway

 

Storage Gateway enables users to connect and extend their on-premises applications to AWS storage.

 

Storage Gateway provides cloud-backed file shares and provides a low-latency cache for on-premises applications to access data in AWS.

 

This service has three alternative gateways available:

 

File Gateway: data is stored in S3 using Amazon S3 File Gateway or using fully-managed file shares through Amazon FSx File Gateway.

 

Tape Gateway: this is a virtual tape library (VTL) which integrates with existing backup software for long-term storage on S3 Glacier and S3 Glacier Deep Archive.

 

Volume Gateway: this stores data locally, backing up block volumes with EBS snapshots

 

 

AWS data transfer pricing

 

AWS wants to encourage potential customers to use its platform, so generally speaking it doesn’t charge for migrating data to AWS.

 

However note that there are often charges levied for transferring back out again from AWS.

 

Generally, the charges for data migration depend on the resources and infrastructure used in facilitating the transfer. This will depend on the method you choose, your region/s used, the instances and other resources you use, and how fast the connection is.

 

As from April 2022, inter-Availability Zone (AZ) data transfers within the same AWS Region for AWS PrivateLink, AWS Transit Gateway, and AWS Client VPN are now free of charge.

 

The best way to calculate your exact data transfer costs is to use the AWS Pricing Calculator and the AWS Cost Explorer.

 

 

AWS VPN pricing

 

AWS VPN costs are calculated according to how many hours the connection is active:

 

$0.05 per Site-to-Site VPN connection per hour and per AWS Client VPN connection per hour for connections to US

 

 

AWS Database Migration Service pricing

 

If you’re using AWS Database Migration Service to transfer existing databases to Amazon Aurora, Redshift, or DynamoDB, then you can enjoy free usage for six months.

 

After that time, you only pay for the compute resources, ie instances that you use to port databases to AWS, plus any additional log storage space required.

 

Each DMS database migration instance will include sufficient storage for swap space, replication logs, and data caching to cover the majority of cases.

 

On-demand EC2 instances are priced by hourly usage, depending on how powerful the instance is, and whether you are choosing single or multiple availability zones for your instances.

 

Instance pricing is from $0.018 per hour, up to $21.65 per hour for multi-AZ instances with fastest processor performance and lowest network latency.

 

 

AWS S3 Transfer Acceleration pricing

 

Pricing for AWS S3 Transfer Acceleration service is based on the volume of data you are migrating to S3, rather than how long you are using the connection.

 

examples:

 

Data accelerated via Edge Locations in the United States, Europe, and Japan: $0.04 per GB

 

Data accelerated via all other AWS Edge Locations: $0.08 per GB

 

Transfer Acceleration constantly monitors its own speed, and if speeds are not faster than a standard transfer via public internet then you will not be charged for the service.

 

AWS DataSync pricing

 

For AWS DataSync, you are charged according to the amount of data you transfer via the service. This is currently priced at $0.0125 per gigabyte (GB) of data transferred.

 

AWS Direct Connect pricing

 

Direct Connect is priced by the hour. There are two cost options according to the capacity of your Dedicated Connection:

 

1G: $0.30/hour

 

10G: $2.25/hour

 

If you wish to transfer data out using Direct Connect, then there are additional charges to pay for this facility.

 

AWS Storage Gateway pricing

 

Charges for AWS Storage Gateway are based on the type and amount of storage you use, as well as the requests you make and the volume of data you are transferring out.

 

Data Transfer out from AWS Storage Gateway service to on-premises gateway device is charged between $0.05-$0.09 per GB.

 

Data Transfer in via your gateway device to Amazon EC2 costs $0.02 per GB.

 

 

Some Tips For Minimizing Data Migration Costs

 

Keep your data transfer within a single AWS Region and Availability Zone

 

Utilize cost allocation tags to identify and analyse where you’re incurring your highest data transfer costs

 

Deploy Amazon CloudFront to reduce EC2 Instance/s to public Internet transfer costs, and utilize CloudFront’s free tier for the first year of use (note this is valid only up to 50 GB of outbound data transfer and 2 million HTTP requests per month)

 

Reduce the volume of data that you need to transfer whenever possible before starting the migration.

 

Deploy VPC endpoints to avoid routing traffic via the public Internet when connecting to AWS

 

 

AWS suggest the following schema for deciding on which migration method to choose:

 

 

 

 

 

Time Overhead for Migrating Data to AWS 

 

This is the formula suggested by AWS to determine how long it will take to transfer data to AWS from your on-premises site.

 

 

Number of Days = (Total Bytes)/(Megabits per second * 125 * 1000 * Network Utilization * 60 seconds * 60 minutes * 24 hours)

 

 

Let’s consider a very simple example consisting of just one virtual server machine of say 20GB in total size (no separate file server or other devices in this example)

 

 

So that will give us following calculation:

 

 

don’t forget to convert megabytes to megabits first. So our 20GBytes becomes using the table at https://convertlive.com/u/convert/gigabytes/to/megabits#20 for this:

 

 

Total Bytes will be:

 

 

20 Gigabytes = 21474836480 Bytes

 

 

21 474 836 480 Bytes

 

 

that’s just over 21.4 billion Bytes

 

so Number of Days = (Total Bytes)/(Megabits per second * 125 * 1000 * Network Utilization * 60 seconds * 60 minutes * 24 hours)

 

 

 

Connection & Data Scale Method Duration

 

 

Less than 10 Mbps & Less than 100 GB Self-managed ~ 3 days
Less than 10 Mbps & Between 100 GB – 1 TB AWS-Managed ~ 30 days
Less than 10 Mbps & Greater than 1 TB AWS Snow Family ~ weeks
Less than 1 Gbps & Between 100 GB – 1 TB Self-managed ~ days
Less than 1 Gbps & Greater than 1 TB AWS- Managed / Snow Family ~ weeks

 

 

Post AWS Migration Stage

 

After completing the migration process, make sure you run all necessary tests, and confirm everything is working correctly.

 

In particular you should look at configuring CloudWatch, CloudTrail and other monitoring services, plus AWS Auto Scaling and  CloudFront if required.