Tags Archives: aws


Elasticache is an AWS managed data caching service mainly for databases and applications.


ElastiCache uses one of two open-source in-memory cache engines for its functionality:


Memcached and Redis.



 Elasticache is used to reduce traffic overhead for RDS and some other applications. It is extremely fast as db is held in ram memory.


Your cache must have an invalidation strategy defined to ensure only the most currently used data is stored in the cache.


It can also be used to store user sessions for an application for cases where users may be redirected later to different instances of the application, saving having to re-do the user login session.


But it does require code configurations for apps to be able to query the cache.



ElastiCache includes a feature for master/slave replication and multi-AZ, can be used for achieving cross-AZ redundancy and thus high-availability through the use of Redis replication groups.





Memcached is an ASCII text file memory object caching system. ElastiCache is protocol compliant with Memcached, thus all the tools used with existing Memcached environments can also be used with ElastiCache. This is the simplest caching model and can also be used when deploying large nodes with multiple cores and threads.



Redis is an open-source in-memory key-value store that supports information structures such as lists and sorted sets.


Redis can power multiple databases, as well as maintain the persistence of your key store and works with complex data types — including bitmaps, sorted sets, lists, sets, hashes, and strings.


If Cluster-Mode is disabled, then there is only one shard. The shard comprises the primary node together with the read replicas. Read replicas store a copy of the data from the cluster’s primary node.


Elasticache allows for up to 250 shards for a Redis cluster if Cluster-Mode is enabled. Each shard has a primary node and up to 5 read replicas.


When reading or writing data to the cluster, the client determines which shard to use based on the keyspace. This avoids any potential single point of failure.



Implementing ElastiCache


There are three main implementation steps:


Creating an ElastiCache cluster
Connecting to the ElastiCache cluster from an EC2 instance
Managing the ElastiCache environment from the AWS console



Creating an ElastiCache cluster


This involves choosing and configuring the caching engine to be used. This will be either Redis or Memcached. For each caching engine, configuration parameters differ.


Next, we need to choose the location ie in AWS cloud or On-Premise.


AWS Cloud – This uses the AWS cloud for your ElastiCache instances


On-Premises – In this case, you can create your ElastiCache instances using AWS Outpost.


AWS Outposts is a fully managed service that extends AWS infrastructure, services, APIs, and tools to your own on-site infrastructure.



ElastiCache REDIS Replication –  Cluster Mode Disabled


There are two possible configuration modes for running ElastiCache and REDIS:


Cluster Mode Disabled, and Cluster Mode Enabled:


In this configuration you run ONE PRIMARY NODE of ElastiCache with up to 5 Read Replicas


Note that uses asynchronous replication to maintain the Read Replicas, so there is a lag.


The primary node is always used for read/write. The other nodes are read-only.


There is just ONE SHARD and all shards hold all the data.


Multi-AZ is enabled by default for failovers.



ElastiCache REDIS Replication –  Cluster Mode Enabled 


With Cluster Mode Enabled the data is  partitioned across MULTIPLE SHARDS


Data is divided across all your shards. This helps especially with scaling write transactions.


Each shard consists of a primary node and up to 5 read replica nodes.

Also has multiple AZ availability


Provides up to 500 nodes per cluster with a single master node.

or 250 nodes with 1 master and 1 replica.




Scaling REDIS with ElastiCache


For “Cluster Mode Disabled”:


Horizontal scaling – you scale out or in by adding or removing read replicas


Vertical scaling – you alter the type of the underlying nodes 


Important for exam!


This is done by means of ElastiCache creating a NEW node group with the new type specification for the nodes, then performing a replication to the new node group, and then finally updating the DNS records so that they point to the new node group and not any longer to the old original node group before scaling.


For “Cluster Mode Enabled”:


this can be done in two different ways – online, and offline:


Online: no interruption to service no downtime, but can be some performance degredation during the scaling.


Offline: service is down, but additional configurations are supported


So, when doing horizontal REDIS scaling, you can do online and office rescaling, and you can do resharding or shard rebalancing for this:


Resharding: “resharding” – this means scaling in or out by adding or removing shards.


Shard rebalancing:  involves equally redistributing the keyspaces among the shards as balanced as possible.


Vertical Scaling: you are changing to a larger or smaller node type, this is done online only, relatively straightforward.




REDIS Metrics to Monitor


Evictions: this is the number of non-expired items the cache has removed in order to make space for new writes ie the memory was full.


In this case choose an eviction policy to evict expired items eg least recently used items, LRU  or scale up to a larger node type with more memory, or else scale out by adding more nodes


CPUUtilization: this monitors CPU usage for the entire host, if too high, then scale up to a larger node type with more memory


SwapUsage: this should not be allowed to exceed 50Mb, if it does then verify you have configured enough reserved memory


CurrConnections: no of current connections – see if a specific app is causing this




NetworkBytesIn/Out & NetworkPAcketsIn/Out


ReplicationBytes: vol of data being replicated


ReplicationLag: how far behind the replica is from the primary node










Some ElastiCache use cases


know these for the exam!


Updating and managing leaderboards in the gaming industry


Conducting real-time analytics on live e-commerce sites


Monitoring status of customers’ accounts on subscription-based sites


Processing and relaying messages on instant messaging platforms


Online media streaming


Performing geospatial processes




Pros and Cons of Using ElastiCache


Pros of ElastiCache


Fully-managed – ElastiCache is a fully-managed cloud-based solution.


AWS takes care of backups, failure recovery, monitoring, configuration, setup, software updating and patches, and hardware provisioning.


Improved application performance – ElastiCache provides in-memory RAM data storage that substantially reduces database query times.


Easily scalable – you can scale up and down with minimal overhead


Highly available – ElastiCache achieves high availability through automatic failover detection and use of standby read replicas.


Cons of ElastiCache


Limited and complex integration – ElastiCache doesn’t provide many easy options for integration. And you can only connect Elasticache to databases and applications hosted by AWS.

High learning curve – the Elasticache user interface is not intuitive and the system requires a high learning overhead to properly understand.


High price – You pay only for what you use but the costs of using Elasticache can swiftly rise according to usage.



Comparison of ElastiCache With Redis, CloudFront, And DynamoDB


ElastiCache is very different to all these services.



AWS ElastiCache versus Redis



ElastiCache is an in-memory cache in the cloud. With very fast retrieval of data from managed in-memory caches, Elasticache improves overall response times, and saves relying wholly on slow disk-based databases for processing queries.


Redis stands for Remote Dictionary Server — a fast, in-memory, open-source, key-value data store that is usually implemented as a queue, message broker, cache, and database.


ElastiCache is developed on open-source Redis to be compatible with the Redis APIs, as well as operating seamlessly with Redis clients.


This means that you can run your self-managed Redis applications and store the data in an open Redis format, without having to change the code.


ElastiCache versus CloudFront


While ElastiCache and CloudFront are both AWS caching solutions, their approaches and framework differ greatly.


ElastiCache enhances the performance of web applications by retrieving information from fully-managed in-memory data stores at high speed.


To do this it utilizes Memcached and Redis, and is able in this way to substantially reduce the time applications need to read data from disk-based databases.


Amazon CloudFront is primarily a Content Delivery Network (CDN) for faster delivery of web-based data through deploying endpoint caches that are positioned closer to the traffic source. This saves too much web traffic from further-flung geolocations from having to source content entirely from the original hosting server.


ElastiCache versus DynamoDB


DynamoDB is a NoSQL fully-managed AWS database service that holds its data on solid state drives (SSDs). These SSDs are then cloned across three availability zones to increase reliability and availability. In this way, it saves the overhead of building, maintaining, and scaling costly distributed database clusters.


ElastiCache is the AWS “Caching-as-a-Service”, while DynamoDB serves as the AWS “Database as a Service”.



Pricing of ElastiCache


To use ElastiCache you have to make a reservation- Pricing for this is based on the caching engine you choose, plus the type of cache nodes.


If you are using multiple nodes (ie replicas) in your cluster, then ElastiCache will require you to reserve a node for each of your cluster nodes.



Difference Between Redis and Memcached



REDIS: similar to RDS

multi AZ with auto failover
read replicas used to scale reads and provide HA.


Data durability


provides backup and restore


Primary use case: In-memory database & cache   Cache
Data model: In-memory key-value 
Data structures: Strings, lists, sets, sorted sets, hashes, hyperlog
High availability & failover: Yes 


Memcached by contrast:


Primary use case: Cache
Data model: In-memory key-value
Data structures: Strings, objects
High availability & failover: No



is multi-node data partitioning ie sharding


no HA


non-persistent data


no backup and restore

multi-threaded architecture



Main Points To Remember About REDIS and Memcached


REDIS is for high-availability – memcached has no AZ-failover, only sharding.


Also REDIS provides backup & restore – memcached does not.


Memcached has a multi-threaded architecture, unlike REDIS.




Redis Metrics to Monitor


Evictions: this is the number of non-expired items the cache has removed in order to make space for new writes ie the memory was full.


In this case choose an eviction policy to evict expired items eg least recently used items, LRU


scale up to a larger node type with more memory, or else scale out by adding more nodes


CPUUtilization: this monitors CPU usage for the entire host, if too high, then scale up to a larger node type with more memory


SwapUsage: this should not be allowed to exceed 50Mb, if it does then verify you have configured enough reserved memory

CurrConnections: no of current connections – see if a specific app is causing this




NetworkBytesIn/Out & NetworkPAcketsIn/Out


ReplicationBytes: vol of data being replicated


ReplicationLag: how far behind the replica is from the primary node




Memcached Scaling


Memcached clusters can have 1-40 nodes soft limit


Horizontal scaling: you add or remove nodes from the cluster and use “Auto-discovery” to allow you app to identify the new nodes or new node config.


Vertical scaling:  scale up or down to larger or smaller node types


to scale up: you create a new cluster with the new node type


then update your app to use the new cluster endpoints


then delete the old cluster


Important to note that memcached clusters/nodes start out empty, so your data will be re-cached from scratch once again.


there is no backup mechanism for memcached.



Memcached Auto Discovery


automatically detects all the nodes


all the cache nodes in the cluster maintain a list of metadata about all the nodes


note: this is seamless from the client perspective


Memcached Metrics to Monitor

Evictions: the number of non-expired items the cache evicted to allow space for new writes (when memory is overfilled). The solution: use a new eviction policy to evict expired items, and/or scale up to larger node type with more RAM or else scale out by adding more nodes


CPUUtilization: solution: scale up to larger node type or else scale out by adding more nodes


SwapUsage: should not exceed 50MG


CurrConnections: the number of concurrent and active connections


FreeableMemory: amount of free memory on the host






Continue Reading

AWS DB Parameters

Database or DB parameters specify how your database is configured. For example, database parameters can specify the amount of resources, such as memory, to allocate to a database.


You manage your database configuration by associating your DB instances and Multi-AZ DB clusters with parameter groups. Amazon RDS defines parameter groups with default settings.


A DB Parameter Group is a collection of engine configuration values that you set for your RDS database instance.


It contains the definition of what you want each of these over 400 parameters to be set to.


By default, RDS uses a default parameter group specified by AWS. It is not actually necessary to use a different parameter group.


Each default parameter group is unique to the database engine you select, the EC2 compute class, and the storage allocated to the instance.


You cannot change a default parameter group, so if you want to make modifications then you will have to create your own parameter group.


RDS database engine configuration is managed through the use of parameters in a DB parameter group.


DB parameter groups serve as an effective container for holding engine configuration values that are applied to your DB instances.


A default DB parameter group is created if you make a database instance without specifying a custom DB parameter group. This default group contains database engine defaults and Amazon RDS system defaults based on the engine, compute class, and allocated storage of the instance.



When you create a new RDS database, you should ensure you have a new custom DB parameter group to use with it. If not then you might have to perform an RDS instance restart later on to replace the default DB parameter group, even though the database parameter you want to alter is dynamic and modifiable.


This is the best approach that gives you flexibility further down the road to change your configuration later on.


Creating your own parameter group can be done via the console or the CLI. Self-created parameter groups take as their basis the default parameter group for that particular instance and selected db engine.


After creation, you can then modify the parameters via the console or CLI to suit your needs as you change.


Parameters can either be static or dynamic.


Static means that changes won’t take effect without an instance restart.


Dynamic means a parameter change can take effect without an instance restart.


Dynamic parameters are either session scope or global scope.


Global scope dynamic parameters mean that changes will impact the entire server and all sessions.


Session scope dynamic parameters however are only effective for the session where they were set.


Note however that some parameter variables can have both global and session scope.


In these cases, the global value is used as the default for the session scope and any global change to a parameter that also has a session scope will only affect new sessions.


Another important aspect to bear in mind when creating a DB parameter group:



You should wait at least 5 minutes before creating your first DB instance that uses that DB parameter group as the default parameter group. This allows Amazon RDS to fully complete the create action before the parameter group is used as the default for a new DB instance.



For exam!


Important to know this DB Parameter above all for the exam:


for PostgreSQL and SQLServer:




this forces ssl connections to be used




BUT – for MySQL /MariaDB you must instead use a grant select command:


GRANT SELECT ON mydatabase.* TO ‘myuser’@%’IDENTIFED BY ‘…’ REQUIRE SSL;




Continue Reading

AWS – Migration of On-Premises Infrastructure to AWS Cloud

The Migration Process can be split into three parts:



Before AWS Migration


During AWS Migration


After AWS Migration





AWS Migration: 5 Cloud Migration Steps



These are the 5 principal AWS Migration steps you need to consider:


Planning and Assessment
Migration Tools
AWS Cloud Storage Options
Migration Strategies
Application Migration Options


Planning and Assessment


The planning and assessment phase is divided into:


Financial Assessment
Security & Compliance Assessment
Technical and Functional assessment


Financial Assessment


Before deciding on-premises to cloud migration, you need to estimate the cost of moving data to the AWS cloud. A careful and detailed analysis is required to weigh the financial considerations of on-premises center versus employing a cloud-based infrastructure.


Security and Compliance Assessment


Overall risk tolerance
Main concerns around availability, durability, and confidentiality of your data.
Security threats
Options available to retrieve all data back from the cloud


Classify your data according to these concerns. This will help you decide which datasets to move to the cloud and which ones to keep in-house.



Technical and Functional Assessment


Assess which applications are more suited to the cloud strategically and architecturally.


Points to consider:


Which applications or data should best be moved to the cloud first?
Which data can we transfer later?
Which applications should remain on-premises?
Can we reuse our existing resource management/configuration tools?
What do we do about support contracts for hardware, software, and networking?


For small-scale data migrations


Unmanaged Cloud Data Migration Tools


For simple, low-cost methods for transferring smaller volumes of data:


Glacier command line interface- On-premises data → Glacier vaults
S3 command line interface- Write commands → Data moves directly into S3 buckets
Rsync- Open source tool combined with 3rd party file system tools. Copy data directly → S3 buckets



For large-scale data migrations


AWS Managed Cloud Data Migration tools

For moving larger volumes of data:

how much data to migrate? Which AWS data migration tool is best suited

Migrate petabytes of data in batches to the cloud AWS Import/Export Snowball
Migrate exabytes of data in batches to the cloud AWS Snowmobile
Connect directly to an AWS regional data center AWS Direct Connect
Migrate recurring jobs, plus incremental changes over long distances Amazon S3 Transfer Acceleration



Some Practical Strategies for AWS Migration


Forklift Migration Strategy


This is more suitable for self-contained, tightly-connected or stateless applications. Its a “pick up everything and move it in one go to the cloud” method.

Is best suited to smaller environments.


Hybrid Mixed-Migration Strategy


This involves moving some parts of an application to the cloud while leaving other parts of the application on-premises.


It is best suited to migrating larger systems which run multiple applications. However, it can be more time-consuming to complete the migration in this way.



Configuring and Creating AMI Images


AMIs provide the information needed to launch an EC2 instance.



Online data transfer from on-premises to AWS


Here are the online data transfer options.



AWS Virtual Private Network


There are two options for using AWS VPN:



AWS Site-to-Site VPN
AWS Client VPN


AWS VPN is encrypted, easy to configure and cost-effective for small data volumes. However, it is a shared connection, so not as fast or reliable as other options.



AWS Virtual Private Network (AWS VPN) establishes secure private connection your network to AWS.




AWS VPN is encrypted, easy to configure and cost-effective for small data volumes. However, it is a shared connection, so not as fast or reliable as other options.




AWS Database Migration Service



The AWS Database Migration Service as the name suggests handles database migration to AWS. The big advantage of DMS is that the database remains fully operational and usable during the migration.


AWS S3 Transfer Acceleration


To migrate large quantities of data over longer distances to AWS S3, AWS S3 Transfer Acceleration enables you to do this 50-500% faster yet still using the public internet.


Data is routed to S3 via optimized network paths using Amazon CloudFront Edge Locations situated across the globe. This maximizes available bandwidth. You select this service on the S3 Dashboard console, selecting one of two TA options. The transfer acceleration is then activated without any need for special client applications or additional network protocols.


AWS DataSync


AWS DataSync enables users to automate the migration of on-premises storage to S3 or Amazon EFS and can transfer up to 10 times faster than some open source migration services. It deploys an on-premises software agent which connects to your on-premises storage system via NFS (Network File System) and SMB (Server Message Block) protocols.


DataSync also takes care of much of the transfer overhead such as Running instances, encryption, managing scripts, network optimization, and validating data all while transferring data up to 10 times faster than many open source migration services.


It can be used to copy data via AWS Direct Connect or public internet to AWS, and is suitable for both one-time data migration, and recurring workflows, as well as for automated backup and recovery actions.



AWS Direct Connect


AWS Direct Connect is a dedicated connection from your on-premises to AWS.


As with AWS VPN, Direct Connect provides an encrypted connection between your on-premises environment and AWS.


However, Direct Connect does not use the public internet and instead runs via a private connection it establishes which will be either via a 1 GB or 10 GB fiber-optic Ethernet cable used to connect your router to an AWS Direct Connect router. On other words, the Direct Connect solution is part software and part hardware.


Because of this dedicated connection, Direct Connect is significantly more costly than using just public internet-and-VPN solutions.


But if you need to transfer or stream very large amounts of data back and forth to the AWS Cloud, then a Direct Connect line may be the best solution. However for smaller transfer one-off migrations it is not so suited.



AWS Storage Gateway


Storage Gateway enables users to connect and extend their on-premises applications to AWS storage.


Storage Gateway provides cloud-backed file shares and provides a low-latency cache for on-premises applications to access data in AWS.


This service has three alternative gateways available:


File Gateway: data is stored in S3 using Amazon S3 File Gateway or using fully-managed file shares through Amazon FSx File Gateway.


Tape Gateway: this is a virtual tape library (VTL) which integrates with existing backup software for long-term storage on S3 Glacier and S3 Glacier Deep Archive.


Volume Gateway: this stores data locally, backing up block volumes with EBS snapshots



AWS data transfer pricing


AWS wants to encourage potential customers to use its platform, so generally speaking it doesn’t charge for migrating data to AWS.


However note that there are often charges levied for transferring back out again from AWS.


Generally, the charges for data migration depend on the resources and infrastructure used in facilitating the transfer. This will depend on the method you choose, your region/s used, the instances and other resources you use, and how fast the connection is.


As from April 2022, inter-Availability Zone (AZ) data transfers within the same AWS Region for AWS PrivateLink, AWS Transit Gateway, and AWS Client VPN are now free of charge.


The best way to calculate your exact data transfer costs is to use the AWS Pricing Calculator and the AWS Cost Explorer.



AWS VPN pricing


AWS VPN costs are calculated according to how many hours the connection is active:


$0.05 per Site-to-Site VPN connection per hour and per AWS Client VPN connection per hour for connections to US



AWS Database Migration Service pricing


If you’re using AWS Database Migration Service to transfer existing databases to Amazon Aurora, Redshift, or DynamoDB, then you can enjoy free usage for six months.


After that time, you only pay for the compute resources, ie instances that you use to port databases to AWS, plus any additional log storage space required.


Each DMS database migration instance will include sufficient storage for swap space, replication logs, and data caching to cover the majority of cases.


On-demand EC2 instances are priced by hourly usage, depending on how powerful the instance is, and whether you are choosing single or multiple availability zones for your instances.


Instance pricing is from $0.018 per hour, up to $21.65 per hour for multi-AZ instances with fastest processor performance and lowest network latency.



AWS S3 Transfer Acceleration pricing


Pricing for AWS S3 Transfer Acceleration service is based on the volume of data you are migrating to S3, rather than how long you are using the connection.




Data accelerated via Edge Locations in the United States, Europe, and Japan: $0.04 per GB


Data accelerated via all other AWS Edge Locations: $0.08 per GB


Transfer Acceleration constantly monitors its own speed, and if speeds are not faster than a standard transfer via public internet then you will not be charged for the service.


AWS DataSync pricing


For AWS DataSync, you are charged according to the amount of data you transfer via the service. This is currently priced at $0.0125 per gigabyte (GB) of data transferred.


AWS Direct Connect pricing


Direct Connect is priced by the hour. There are two cost options according to the capacity of your Dedicated Connection:


1G: $0.30/hour


10G: $2.25/hour


If you wish to transfer data out using Direct Connect, then there are additional charges to pay for this facility.


AWS Storage Gateway pricing


Charges for AWS Storage Gateway are based on the type and amount of storage you use, as well as the requests you make and the volume of data you are transferring out.


Data Transfer out from AWS Storage Gateway service to on-premises gateway device is charged between $0.05-$0.09 per GB.


Data Transfer in via your gateway device to Amazon EC2 costs $0.02 per GB.



Some Tips For Minimizing Data Migration Costs


Keep your data transfer within a single AWS Region and Availability Zone


Utilize cost allocation tags to identify and analyse where you’re incurring your highest data transfer costs


Deploy Amazon CloudFront to reduce EC2 Instance/s to public Internet transfer costs, and utilize CloudFront’s free tier for the first year of use (note this is valid only up to 50 GB of outbound data transfer and 2 million HTTP requests per month)


Reduce the volume of data that you need to transfer whenever possible before starting the migration.


Deploy VPC endpoints to avoid routing traffic via the public Internet when connecting to AWS



AWS suggest the following schema for deciding on which migration method to choose:






Time Overhead for Migrating Data to AWS 


This is the formula suggested by AWS to determine how long it will take to transfer data to AWS from your on-premises site.



Number of Days = (Total Bytes)/(Megabits per second * 125 * 1000 * Network Utilization * 60 seconds * 60 minutes * 24 hours)



Let’s consider a very simple example consisting of just one virtual server machine of say 20GB in total size (no separate file server or other devices in this example)



So that will give us following calculation:



don’t forget to convert megabytes to megabits first. So our 20GBytes becomes using the table at https://convertlive.com/u/convert/gigabytes/to/megabits#20 for this:



Total Bytes will be:



20 Gigabytes = 21474836480 Bytes



21 474 836 480 Bytes



that’s just over 21.4 billion Bytes


so Number of Days = (Total Bytes)/(Megabits per second * 125 * 1000 * Network Utilization * 60 seconds * 60 minutes * 24 hours)




Connection & Data Scale Method Duration



Less than 10 Mbps & Less than 100 GB Self-managed ~ 3 days
Less than 10 Mbps & Between 100 GB – 1 TB AWS-Managed ~ 30 days
Less than 10 Mbps & Greater than 1 TB AWS Snow Family ~ weeks
Less than 1 Gbps & Between 100 GB – 1 TB Self-managed ~ days
Less than 1 Gbps & Greater than 1 TB AWS- Managed / Snow Family ~ weeks



Post AWS Migration Stage


After completing the migration process, make sure you run all necessary tests, and confirm everything is working correctly.


In particular you should look at configuring CloudWatch, CloudTrail and other monitoring services, plus AWS Auto Scaling and  CloudFront if required.




Continue Reading

AWS Security Services

Services that provide DDOS Protection on AWS


AWS Shield Standard, free of charge, is activated by default


AWS Shield Advanced – 24×7 premium protetion, fee charged and access to AWS DRP DDOS Response Team v expensive about 3000 USD per month.


AWS WAF filters specific requests based on rules – layer 7 http – for app load balancer, api gateway and CloudFront
you can define web acl : geo block, ip address blocks, sql injection etc



CloudFront and Route 53: uses global edge network, combined with Shield can provide attack mitigation at the edge


You can utilize AWS AutoScaling to leverage up if there is an attack.


You get full DDOS protection by combining Shield, WAF, CloudFront and Route53.



Penetration testing can be carried out by customers for 8 services eg EC2, RDS, CF etc – you don’t need any authorization to do this but you cannot do simulated ddos attacks on your system or dns zone walking on route 53, nor flooding tests


important note:
for any other simulated attacks, contact aws first, to check, otherwise it is not authorized – and could be seen as an infrastructure attack on aws!  


AWS Inspector:


chargable, first 15 days free. not cheap. cost per instance or image scanned.


does automated security assessments, eg for EC2


sends reports to security hub and event bridge


leverages the System Manager SSM agent


for Containers pushing to ECR –  assesses containers as they are moved to ECR


it is ONLY for EC2 and container infra. But only done when needed.


checks packages against CVE – package vulnerability scan


also does network reachability for EC2


that is all.



Logging on AWS – quick overview


aws services generate a wide range of logs


cloudtrail trails, config rules, cw logs vpc flow logs, elb access logs cloud front logs, waf logs,



exam question!
LOGS can be analyzed using AWS Athena if stored on S3.


you should encrypt logs stored on S3 and control the access to them by deploying iam and bucket policies plus mfa.

and always remember:
don’t log a server that is logging! otherwise you create an endless logging loop!


and move logs to glacier for cost saving


and also use glacier vault which locks the logs so they cant be tampered with.



AWS Guard Duty


this uses intelligent threat discovery and ML learning to detect


no need to install any software, works in the backend, only need to activate, but it chargeable
esp analyses cloudtrail, vpc flow logs, dns logs, kubernetes audit logs, looks for unusual api calls etc

you can set up cloudwatch events rules to connect to labda or sns

exam q
also can protect against cryptocurrency attacks, has a dedicated function for it. – comes up in exam



AWS Macie

a fully managed data security and data privacy service which uses ML pattern matching to protect your data


helps identify and alert esp re PII – personal identifiable information


can notify event bridge




AWS Trusted Advisor – only need to know overview for the exam


no need to install, is a service


core checks and recommendations — available for all customers, these are free


can send you a weekly email notification


full trusted advisor for business and enterprise – fee based
and can then create cloudwatch alarms or use apis


cost optimization


looks for underutilized resources – but cost optimizn is not in the free core checks, so you need to upgrade for this.



ec2s ebs cloud front




mfa security used or not, iam key rotation etc, exposed access keys
s3 bucket permissions, security group issues, esp unrestricted ports


fault tolerance — eg abs snapshot age,

service limits

Continue Reading

AWS CloudTrail, CloudWatch, and Config Compared

CloudTrail is a services which provides governance, compliance, and auditing of your AWS account by logging and monitoring account activities.


What’s the difference between CloudWatch and CloudTrail


AWS CloudWatch


CloudWatch s a monitoring tool used for real-time monitoring of AWS resources and applications. It provides a monitoring service which analyzes the performance of the system.  


CloudWatch can be used to detect irregular behaviour in an AWS environments. It monitors various AWS resources including EC2, RDS,  S3, Elastic Load Balancer, etc. It can also be used with CloudWatch Alarms. 



2. AWS CloudTrail


CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS account. It continuously logs and monitors the activities and actions across your AWS account. It provides the event history of your AWS account including data about who is accessing your system.  Remediation actions can also be taken by CloudTrail.      


While CloudWatch reports on the activity and health and performance of your AWS services and resources,  CloudTrail by contrast is a log of all the actions that have taken place inside your AWS environment.


CloudTrail can record API activity in your AWS account and reports an event within 15 minutes of the API call.



It provides auditing services for AWS accounts. In CloudTrail, Logs are saved in an S3 bucket.


However, you can receive notification of specific CloudTrail events immediately by sending them via the CloudWatch Event Bus.


While CloudTrail only writes to your S3 bucket once every five minutes, it sends events to the CloudWatch Event bus in near real-time as these API calls are observed.


CloudWatch monitors performance. For example, tracking metrics of an EC2 instance or keeping track of your Amazon DynamoDB performance or to see how Amazon S3 is performing. CloudWatch allows you to collect default metrics for over 70 AWS services.


It also has a “Custom Metrics” feature that enables you to collect a metric that is specifically important to your system. For example, to measure how people are using your application.



AWS CloudTrail


AWS CloudTrail is principally used for auditing API activity, tracking who did what and when, and securely logging this information to Amazon S3 for later analysis.


Thus CloudTrail keeps track of what is done in your AWS account, when, and by whom. For example, with CloudTrail you can view, search, and download latest activity in your AWS account to check it there are any abnormal or unusual actions and if so, by whom. This type of reporting is called auditing and it is the core service of CloudTrail.



CloudTrail tracks data events and management events:


Data events are object-level API requests made to your resources. For example, when an item is created or deleted in a DynamoDB table.


Management events log changes (mostly creation or deletion changes) to your environment, such as the creation or deletion of the entire DynamoDB itself.


CloudTrail tracks which applications or persons took these actions and stores the details in logfiles. These logfiles are encrypted and stored in S3.



Note that CloudWatch has CloudWatch Alarms which you can configure and metric data is retained for 15 months. CloudTrail on the other hand has no native alarms. However, you can configure CloudWatch Alarms for CloudTrail, but you have to store logs in S3.


In a nutshell:


CloudWatch is for performance. Think of CloudWatch as monitoring application metrics.
CloudTrail is for auditing. Think of CloudTrail as tracking API activity within an account.





AWS Config vs. CloudTrail


In the configuration and monitoring category AWS, there are two major AWS monitoring tools that are similar, and are easy to confuse. They are AWS Config and AWS CloudTrail.


Config and CloudTrail are different tools with different purposes.


What is AWS Config?


AWS Config is a service that lets you set configuration rules for your AWS resources to comply with. It then tracks whether the resources comply with those rules.


Whenever a resource has changed, Config records the change in a configuration history in an S3 bucket. It stores a snapshot of the system at a regular period of time set by you. It also has a dashboard that presents an overview of your resources and their configurations.



What is AWS CloudTrail?


CloudTrail is a logging service that records all API calls made to any AWS service. It records the details of the API call such as which user or application made the call, the time and date it happened and the IP address it originated from.


There is also another AWS logging service called CloudWatch Logs, but unlike CloudWatch Logs which reports application logs, CloudTrail reports on how AWS services are being used in your environment.


Where CloudTrail and Config are Similar


Config and CloudTrail have a number of things in common. Both are monitoring tools for your AWS resources. Both track changes and store a history of what happened to your resources in the past. Both are used for compliance and governance, auditing and security policies. If you notice something unusual or going wrong with your AWS resources, then chances are you’ll see it reported in both CloudTrail and Config.


Where CloudTrail and Config are Different



Note that AWS Config Rules is not a cheap service. There is no free tier, you pay a fee per config item per region.


Though both often report on the same events, their approach is different. Config reports on what has changed in the configuration, whereas CloudTrail reports on who made the change, and when, and from which IP address.


Config reports on the configuration of your AWS resources and creates detailed snapshots of how your resources have changed.


CloudTrail focuses on the events or API calls behind those changes, focusing on users, applications, and activities performed in your environment.


Where CloudTrail and Config work together


By taking a different approach to the same events, CloudTrail and Config make a good combination. Config is a great starting point for ascertaining what has happened to your AWS resources, while CloudTrail can give you more information from your CloudTrail logs.


Config watches and reports on instances of rules for your resources being violated. It doesn’t actually allow you to make changes to these resources from its own console.


By contrast, CloudTrail gives you more control by integrating with CloudWatch Events to allow you to set automated rule-based responses to any event affecting your resources.


In the case of security breaches, if multiple changes have been made by an attacker in a short period of time, Config might not report this in detail.


Config stores the most recent and important changes to resources but disregards smaller and more frequent changes.

CloudTrail by contrast records every single change in its logs. It also has an integrity validation feature that checks if the intruder or attacker manipulated the API logs to cover their activity track.



Should You Use AWS Config or CloudTrail for Security?



Both Config and CloudTrail have a role to play together. Config records and notifies about changes in your environment. CloudTrail helps you find out who made the change, from where, and when.


A good way to think of it is that AWS Config will tell you what your resource state is now or what it was at a specific point in the past whereas CloudTrail will tell you when specific events in the form of API calls have taken place.


So you ought to use both. Config Rules triggers on a change in the status of your system, but it will often only give you an update on the state of the system itself.


CloudTrail meanwhile provides you with a log of every event which details everything that has taken place and when and by whom. This helps identify all the causes that may have led to the security problem in the first place.



Remember also that AWS Config Rules does not prevent actions from happening – it is not a “deny”.


But – you can do “remediations” of resources that are identified as non-compliant. This can be done for example via SSM Automation Documents. Config then triggers an auto-remediation action that you define.




you can use EventBridge to receive notifications from Config, from there you can also send the notifications onto eg Lambda functions, SNS or SQS.

Continue Reading

AWS Additional Monitoring Tools


AWS Config



AWS Config is an AWS fully managed change management solution within AWS. It allows you to track the change history of individual resources and configure notifications when a resource changes.


This is achieved by means of config rules. A config rule represents the desired state that the resource should be in.


Config rules allow you to monitor for systems that fall outside of your set baselines and identify which changes caused the system to fall out of compliance with the baseline. AWS Config is enabled on a per-region basis, so you need to enable it for every region in which you want to use it.


Bear in mind that AWS Config is a monitoring tool and does not actually enforce baselines, nor does it prevent a user from making changes that cause a resource to move out of compliance.


AWS Config enables you to capture the configuration history for your AWS resources, maintain a resource inventory, audit and evaluate changes
in resource configuration, and enable security and governance by integrating notifications with these changes. You can use it to discover AWS resources in your account, continuously monitor resource configuration against desired resource configuration, and check the configuration details for a resource at a given point in time.


AWS Config is used to assess compliance as according to your set internal guidelines for maintaining resource configurations, as well as enabling compliance auditing, security analysis, resource change tracking, and assisting with operational troubleshooting.


AWS Trusted Advisor


AWS Trusted Advisor service analyzes and checks your AWS environment in real-time and gives recommendations for the following four areas:


Cost optimization
Fault tolerance


Trusted Advisor or TA integrates with AWS IAM so you can control access to checks as well as to categories.


The current status of these checks is displayed in the TA dashboard as follows:


Red: Action recommended
Yellow: Investigation recommended
Green: No problem detected


Where the colour is red or yellow, TA provides alert criteria, recommended actions, and relevant resource details, such as details of the
security groups allowing unrestricted access via specific ports.


Six core checks are available for all AWS customers free of charge.


Five checks for security plus one check for performance:


service limits
IAM use
security groups-unrestricted ports
MFA on root account
Elastic block storage public snapshot
RDS public snapshot.




AWS Inspector


AWS Inspector provides for the automation of security assessments. The assessments can be set to run on a schedule or when an event occurs that is monitored by Amazon CloudWatch, or also via an API call. The dashboard shows the assessments, as well as the findings from the various scans that have run.


Amazon Inspector makes use of assessment templates that define which sets of rules you want to run against your environment.


Two types of assessments are offered by AWS Inspector: network assessments and host assessments.


Network assessments don’t require any agent to be installed. However if you want detailed information about processes running on a specific port then you need to install the AWS Inspector Agent.


Host assessments however require the Inspector Agent to be installed. These assessments are far more detailed and scan for things such as vulnerable versions of software, violations of security best practices, and areas that should be system hardened. You can select these assessments set up AWS Inspector.


You create an assessment template in Inspector which you then use to assess your environment by means of an Assessment Run which will then report on its findings.


Templates contain one or more rules packages. A rules package defines what you are checking for. Note that you can’t create custom rules packages; you can use only the rules packages provided by AWS. Currently, these are the rules packages available, listed by assessment type:

Network assessments

Network Reachability: This rules package checks your environment’s network configurations, including your security groups, network access control lists (NACLs), route tables, subnets, virtual private cloud (VPC), VPC peering, AWS Direct Connect and virtual private gateways (VPGs), Internet gateways (IGW), EC2 instances, elastic load balancers (ELBs), and elastic network interfaces (ENIs).


Host assessments

Common Vulnerabilities and Exposures (CVE): This rules package checks your systems to see if they are vulnerable to any of the CVEs reported.


Center for Internet Security (CIS) Benchmarks: This rules package assesses your systems against CIS benchmarks specific to your OS.


There are Level 1 and Level 2 checks. Level 1 is usually safe to implement; Level 2 is more risky as the settings in Level 2 may have unintended side effects. Level 2 is usually used in environments where a very high level of security is required.


Security Best Practices: This rules package assesses how well your environment confirms to security best practices. Eg, it will check that a Linux EC2 instance cannot be logged into via SSH.


Runtime Behavior Analysis: This rules package identifies risky behaviors on your systems, such as using insecure protocols for connecting or open ports that are not in use.




AWS GuardDuty


GuardDuty is the AWS intrusion detection system (IDS) or intrusion prevention system (IPS). It uses threat intelligence feeds and analyzes logs from multiple sources, such as VPC flow logs, AWS CloudTrail event logs, and DNS logs.


GuardDuty can alert you to suspicious activity that could indicate potential issues such as leaked user account credentials, privilege escalation attacks, and possible command-and-control type activities.


GuardDuty scans specifically for three types of activity:


Instance compromise
Account compromise


Reconnaissance is the first step of an attack and was defined in the “Cyber Kill Chain”, developed by Lockheed Martin. During the reconnaissance
phase, an attacker is learning about your environment through actions such as vulnerability scans to probe for IP addresses, hostnames, open ports, and misconfigured protocols.


GuardDuty can detect also utilize threat intelligence feeds to detect IP addresses known to be malicious. You can use findings reported by GuardDuty to automatically remediate the vulnerability become it develops into a security violation.


The next type of activity is instance compromise. This consists of several indicators that may be present, such as malware command and control, crypto miners, unusual traffic levels or unusual network protocols, or communication with a known malicious IP.



Continue Reading

AWS CloudWatch Monitoring Overview


AWS CloudWatch is the basic AWS monitoring service that collects metrics on your resources in AWS, including your applications, in real time.


You can also collect and monitor log files with AWS CloudWatch. You can set alarms for metrics in CloudWatch to continuously monitor performance, utilization, health, and other parameters of your AWS resources and take action when metrics cross set thresholds.


CloudWatch is a global AWS service, so it can monitor resources and services across all AWS regions via a single dashboard.



CloudWatch provides basic monitoring free of charge at 5-minute intervals as a serverless AWS service, thus there is no need to install any additional software to use it.



For an additional charge, you can set detailed monitoring that provides data at 1-minute intervals.



AWS CloudWatch has a feature that allows you to publish and retain custom metrics for a 1-second or 1-minute duration for your application, services, and resources, known as high-resolution custom metrics.


CloudWatch stores metrics data for 15 months, so even after terminatíng an EC2 instance or deleting an ELB, you can still retrieve historical metrics for these resources.



How CloudWatch Works


CW Monitoring Is Event-Driven


All monitoring in AWS is event-driven. An event is “something that happens in AWS and is captured.”


For example, when a new EBS volume is created, the createVolume event is triggered, with a result of either available or failed. This event and its result are sent to CloudWatch.


You can create a maximum of 5000 alarms in every region in your AWS account.


You can create alarms for functions such as starting, stopping, terminating, or recovering an EC2 instance, or when an instance is experiencing a service issue.


Monitoring Is Customizable


You can define custom metrics easily. A custom metric behaves just like a predefined one and can then be analyzed and interpreted in the same way as standard metrics.

One important limitation of CloudWatch – exam question! 


CloudWatch functions below the AWS Hypervisor, which means it functions below or the virtualization layer of AWS.


This means it can report on things like CPU usage and disk I/O…but it cannot see beyond what is happening above that layer.


This means CloudWatch cannot tell you what tasks or application processes are affecting performance.


Thus it cannot tell you about disk usage, unless you write code that checks disk usage and send that as a custom metric to CloudWatch.


This is an important aspect that can appear in the exam. You might be asked if CloudWatch can report on memory or disk usage by default; it cannot.


Monitoring Drives Action


The final piece of the AWS monitoring puzzle is alarms – this is what occurs after a metric has reported a value or result outside a set “everything is okay” threshold.


When this happens, an alarm is triggered. Note that an alarm is not necessarily the same as “something is wrong”; an alarm is merely a notification that something has happened at a particular point.


For example, it could be running some code in Lambda, or sending a message to an Auto Scaling group telling it to scale in, or sending an email via the AWS SNS message service.


Think of alarms as saving you from having to sit monitoring the CloudWatch dashboard 24×7.


One of your tasks as SysOp is to define these alarms.



CloudWatch Is Metric- and Event-Based


Know the difference between metrics and events.

An event is predefined and is something that happens, such as bytes coming into a network interface.


The metric is a measure of that event eg how many bytes are received in a given period of time.


Events and metrics are related, but they are not the same thing.


CloudWatch Events Are Lower Level


An event is something that happens, usually a metric changing or reporting to CloudWatch, but at a system level.


An event can then trigger further action, just as an alarm can.


Events are typically reported constantly from low-level AWS resources to CloudWatch.


CloudWatch Events Have Three Components


CloudWatch Events have three key components: events, rules, and targets.


An event:


the thing being reported. Events describe change in your AWS resources. They can be thought of as event logs for services, applications and resources.


A rule:



an expression that matches incoming events. If the rule matches an event, then the event is forwarded to a target for processing.



A target:



is another AWS component, for example a piece of Lambda code, or an Auto Scaling group, or an email or SNS/SQS message that is sent out.



Both alarms and events are important and it is essential to monitor both.


CloudWatch Namespaces


A CloudWatch Namespace is a container for a collection of related CloudWatch metrics. This provides for a way to group metrics together for easier understanding and recognition.

AWS provides a number of predefined namespaces, which all begin with AWS/[service].


Eg, AWS/EC2/CPUUtilization is CPU utilization for an EC2 instance,



AWS/DynamoDB/CPUUtilization is the same metric but for DynamoDB.



You can add your own custom metrics to existing AWS namespaces, or else create your own custom namespaces in CloudWatch.



exam question:

CloudWatch can accept metric data from 2 weeks earlier and 2 hours into the future  but make sure your EC2 instance time is set accurately for this to work correctly!



Monitoring EC2 Instances


CloudWatch provides some important often-encountered metrics for EC2.


Here are some of the most common EC2 metrics which you should be familiar with for the exam:



CPUUtilization – one of the fundamental EC2 instance metrics. It shows the percentage of allocated compute units currently in use.


DiskReadOps – reports a count of completed read operations from all instance store volumes.


DiskWriteOps – opposite of DiskReadOps, reports a count of completed read operations from all instance store volumes.


DiskReadBytes – reports the bytes read from all available instance store volumes.


DiskWriteBytes – reports the total of all bytes written to instance store volumes.


NetworkIn – total bytes received by all network interfaces.


NetworkOut – total bytes sent out across all network interfaces on the instance.


NetworkPacketsIn – total number of packets received by all network interfaces on the instance (available only for basic monitoring).


NetworkPacketsOut – number of packets sent out across all network interfaces on the instance. Also available only for basic monitoring.




S3 Metrics


There are many S3 metrics, but these are the most common ones you should know:

BucketSizeBytes – shows the daily storage of your buckets as bytes.

NumberOfObjects – the total number of objects stored in a bucket, across all storage classes.


AllRequests – the total number of all HTTP requests made to a bucket.


GetRequests – total number of GET requests to a bucket. There are also similar metrics for other requests: PutRequests , DeleteRequests , HeadRequests , PostRequests , and SelectRequests.


BytesDownloaded – total bytes downloaded for requests to a bucket.


BytesUploaded – total bytes uploaded to a bucket. These are the bytes that contain a request body.


FirstByteLatency – per-request time for a completed request, by first-byte millisecond.


TotalRequestLatency – the elapsed time in milliseconds from the first to the last byte of a request.




CloudWatch Alarms



Alarms Indicate a Notifiable Change



A CloudWatch alarm initiates action. You can set an alarm for when a metric is reported with a value outside of a set level.


Eg, for when your EC2 instance CPU utilization reaches 85 percent.



Alarms have three possible states at any given point in time:


OK : means the metric lies within the defined threshold.

ALARM : means the metric is below or above the defined threshold.


INSUFFICIENT_DATA : can have a number of reasons. The most common reasons are that the alarm has only just started or been created, that the metric it is monitoring is not available for some reason, or there is not enough data at this time to determine whether the alarm is OK or in ALARM state.



CloudWatch Logs


CloudWatch Logs stores logs from AWS systems and resources and can also handle the logs for on-premises systems provided they have the Amazon Unified CloudWatch Agent installed.


If you are monitoring AWS CloudTrail activity through CloudWatch, then that activity is sent to CloudWatch Logs.


If you need a long retention period for your logs, then CloudWatch Logs can also do this.


By default logs are kept forever and never expire. But you can adjust this based on your own retention policies.

You can choose to keep logs for only a single day or go up to 10 years.


Log Groups and Log Streams


You can group logs together that serve a similar purpose or from a similar resource type. For
example, EC2 instances that handle web traffic.



Log streams refer to data from instances within applications or log files or containers.4



CloudWatch Logs can send logs to S3, Kinesis Data Streams and Kinesis Data Firehose, Lambda and ElasticSearch



CloudWatch Logs – sources can be:



CloudWatch Logs Agent,

CloudWatch Unified Agent

Elastic Beanstalk

ECS – Elastic Container Service

Lambda function logs

VPC Flow Logs – these are VPC specific

API Gateway

CloudTrail based on filters

Route53 – logs DNS queries



Define Metric Filters and Insights for CloudWatch Logs

You can apply a filter expression eg to look for a specific IP in a log or the number of occurrences of “ERROR” in the log

Metric filters can be used to trigger CloudWatch Alarms


CloudWatch Logs Insights can be used to query logs and add queries to CloudWatch Dashboards



CloudWatch Logs – Exporting to S3




this can take 12 hours for the data to become available for export – so it is not real time. For this you should use Log Subscriptions.



The API call for this is “CreateExportTask”




CloudWatch Log Subscriptions


You apply a “subscription filter” to the CloudWatch Log before sending it to eg a Lambda function managed by AWS/or to a custom-designed Lambda function and then from there as real-time data on to eg ElasticSearch. Or, you might send it from Subscription Filter and then to Kinesis.



You can also send or aggregate logs from different accounts and different regions to a subscription filter in each region and from there to a common single Kinesis Data Stream and Firehose and from there in near-real time on to eg S3.








Unified CloudWatch Agent



The AWS Unified CloudWatch Agent provides more detailed information than the standard free CloudWatch service.


You can also use it to gather logs from your on-premises servers in the case of a hybrid environment and then centrally manage and store them from within the CloudWatch console.


The agent is available for Windows and Linux operating systems.


When installed on a Windows machine, you can forward in-depth information to CloudWatch from the Windows Performance Monitor, which is built into the Windows operating system.


When CloudWatch is installed on a Linux system, you can receive more in-depth metrics about CPU, memory, network, processes, and swap memory usage. You can also gather custom logs from applications installed on servers.


To install the CloudWatch agent, you need to set up the configuration file.



Continue Reading

AWS NACLs – Network Access Control Lists

The AWS Network Access Control List (NACL) is a security layer for your VPC that acts as a firewall for controlling traffic in and out of one or more subnets.


NACLs vs. Security Groups


NACLs and Security Groups (SGs) both have similar purposes. They filter traffic according to rules, to ensure only authorized traffic is routed to its destination.



NACLs are used to control access to network resources. They reside on subnets and evaluate traffic based on defined rules which you set, and use these rules to determine whether or not traffic should be allowed to pass through the subnet.


NACLs are “STATELESS” which means they require you to create separate rules for BOTH INCOMING AND OUTGOING traffic. Just because a particular data stream is allowed into the subnet, this doesn’t mean it will automatically be allowed out.


NACLs are processed in numerical ie serial order. Thus if you want traffic to be permitted both in and out of a subnet, you have to set network access rules for both directions.


NACLs are automatically applied to everything within that subnet, so there is no need to apply NACLs to individual resources as they are created. This means less network admin overhead for managers.



Security Groups


Security Groups apply to EC2 instances and operate like a host-based firewall. As with NACLs they apply rules that determine whether traffic to or from a given EC2 instance should be allowed.


This provides for more finely tuned traffic control for resources that have specific network traffic requirements.


Security Groups unlike NACLs are stateful; this means that any traffic that is allowed into your EC2 instance will automatically be allowed out again and vice versa.


All security groups rules are evaluated according to a default “deny everything unless allowed” policy. This means that if no ALLOW exists, then traffic will be blocked.


Security Groups must be applied at the time of resource creation and have to be explicitly configured.



Similarities and Differences Between NACLs and Security Groups


Both NACLs and Security Groups utilize rules that prevent unwanted traffic from accessing your network. The rules themselves also look similar. But a notable difference between them is that NACLs allow for DENY rules to be explicitly created.


It is important to ensure that your security group rules and your NACLs are not working against one another. Thus it is important to understand when it is best to use NACLs and when it is best to use SGs.


The major difference between them is in where they are applied. NACLs are applied at the SUBNET level, while Security Groups are applied at the EC2 instance level.


NACLs protect the network while Security Groups protect the resource.


As NACLs are higher up in the architecture, they apply to a much wider set of resources. Any NACL rule you create will therefore impact the operation of every resource located within the subnet.


Security Groups on the other hand only affect the EC2 instances to which they are attached.



When to Use NACLs


NACLs are best used sparingly. Because NACLs apply to the full set of resources in a subnet, their impact is wide and substantial.


NACLs are most effective for filtering external traffic to internal subnets. They can also be useful for applying traffic controls between the subnets themselves.




Best Practices for Using NACLs


Use NACLs sparingly and deploy them based on the function of the subnet they are attached to


Keep NACLs simple and only use them to deny traffic if possible


Restrict who can create or modify NACLs through IAM rules


Build your Security Group rules into your NACLs


Ensure that your inbound and outbound rules make sense ie that they match


When numbering your NACLs, be sure to leave room for future rules


Audit your rules frequently and delete any rules that are unused or redundant


Deploy NACLs to also control your subnet-to-subnet traffic and ensure logical separation between them



Continue Reading

AWS – Choosing an AWS Database

AWS offers several db solutions.


A number of questions need to be considered when choosing an AWS DB solution:


Questions to Consider


What is your need – read heavy, write heavy, balanced?


Throughput volume?


Will this change, does it need to scale, will it fluctuate up and down?, is it predictable or not?


How much data volume to store – and for how long


will it grow?


what is the average object size?


how do you need to access it?


what data durability do you require? what is the “source of truth” for the data?


are there compliance requirements?


what latency requirements – how many concurrent users


what is the data model – how will you query the data, will there be data joins? structured, semi-structured?


strong schema, flexible, reporting needs? searching? RBDMS or NoSQL?


what about license costs – cloud native db such as Aurora possible or not?


Overview of Database Types on AWS


RDBMS such as sql, oltp, this means: RDS, or Aurora, esp good for joins

NoSQL DBs such as DynamoDB – json, elasticache – key/value pairs or Nepture – good for graphs, but no joins and no sql


Object Stores: S3 for big objects, Glacier for backup and archives – may not seem like a DB but it works like one


Data Warehouse solutions eg SQL Analytics/BI, Redshift (OLAP), Athena


Search solutions: eg ElasticSearch (json), for free text unstructureed searches


Graph solutions: Neptune – this displays relationships between data



Overviews of AWS DB Solutions


RDS Overview


its a managed db on the postgresql/myswl/Oracle/SQL level


you must however an ec2 instance and ebs vol type and sufficient size


it supports read replicas and multi-AZ
security is via iam and security groups, kms, and ssl in transit
backup, snapshot and point in time restores all possible


managed and scheduled maintanance


monitoring available via cloudwatch


use cases include:


storing relational datasets rdbms/oltp performing sql queries, transactional inserts, update, delete is possible


rds for solutions architect, considerations include these “5 pillars”:





operations_ small downtimes when failover happens, when maintenance happens, when scaling read replicas, ec2 instances, and restoring from ebs, this requires manual intervention, and when application changes



security: aws is responsible for os security, but we are responsible for setting up kms, security groups, iam policies, authorizing users in db and using ssl



reliability: the multi-az feature makes rds v reliable, good for failover in failure situations


performance: dependent on ec2 instance type, ebs vol type, can add read replicas, storage autoscaling is possible, and manual scaling of instances is also possible


costs: is pay per hour based on provisioned number and type of ec2 instances and ebs usage



Aurora Overview


Compatible API for PostgreSQL and MySQL


Data is held in 6 replicas across 3 AZs
It has auto-healing capability
Multi-AZ Auto Scaling Read Replicas
Read Replicas can be global


Aurora DB can be global for DR or latency purposes
auto Scaling of storage from 10GB to 128 TB


Define EC2 instance type for Aurora instances


same security/monitoring/maintenance as for RDS but also has


Aurora Serverless for unpredictable/intermittent workloads
Aurora Multi-Master for continuous writes failover


use cases: same as for RDS but with less maintenance/more flexibility and higher performance


Operations: less operations, auto scaling storage


security: AWS as per usual, but we are responsible for kms, security groups, iam policy, user auth and ssl


reliability: multi AZ, high availability, serverless and multimaster options


performance 5x performance, plus max 15 read replicas (vs only 5 for rds)


cost: pay per hour acc to ec2 and storage usage, can be lower cost than eg oracle



ElastiCache Overview


it is really just a cache not a database


is a managed Redis or Memcached – similar to RDS but for caches


its an in-memory data store with very low latency
you must provision an ec2 instance type to use it


supports redis clustering and multi-az, plus read replicas using sharding


security is via iam, security groups, kms, and redis auth – NO IAM


backup, snapshot, point in time restores
managed and scheduled maintenance
monitoring via cloudwatch


use case: key/value store, frequent reads, less writes, cache results for db queries, storing of session data for websites, but cannot use sql – latter is important to be aware of.


you retrieve the data only by key value, you can’t query on other fields



operations: same as rds
security: usual can use iam policies,for users Redis Auth, and ssl


reliability: clustering and multi AZ


performance: in memory so extremely fast, read replicas for sharding, very efficient


cost: similar to rds pricing based on ec2 and storage usage





proprietary to AWS
a managed NoSQL DB
serverless, provisioned, auto-scaling, on-demand capacity


can replace elasticache as a key-value store, eg for storing session data


performance is slower than eg rds


highly available, multi-AZ by default, read–writes decoupled, DAW available for read cache


2 options for reads: eventually consistent or strongly consistent


security, authorization-authentication all done via iam


dynamodb streams integrate with lambda


backup-restore and global table feature – if you enable streams


monitoring via cloudwatch


but you can only query on primary key, sort key or indexes – exam q!
so you cannot query on “any” attribute – only the above.


use case:


serverless apps development, small docs 100s kb, distrib serverless cache, doesn’t have sql query language available, has transaction capability now built in




S3 Overview


acts as a simple key/value store for objects


great for big objects up to 5TB, not so good for small objects


serverless, scales infinitely, strong consistency for every operation


tiers for migrating data: s3 standard, s3 IA, s3 one-zone IA, Glacier for backups


features include: versioning, encryption, CRR – cross-region replication


security: IAM, bucket policies, ACL


encryption: SSE-S3, SSE-KMS, SSE-C, client side encryption, SSL in transit


use case: static files, key-value stores for big files and website hosting


operations: no operations necessary!
security: IAM, bucket policies, ACL, encryption set up correctly,


reliability: extremely high, durability also extremely good, multi-AZ and CRR


performance: scales easily, very high read/write, multipart for uploads


cost: only pay for storage used, no need to provision in advance, plus network costs for transfer/retrieval, plus charge per number of requests



Athena Overview


fully serverless database with SQL capability
used to query data in S3
pay per query
outputs results back to S3
is secured via IAM


use cases: one-time sql queries, serverless queries on S3, log analytics



Operations: no ops needed! is serverless


security: via S3 using bucket policies, IAM


reliability: is managed, uses Presto engine, highly available
performance: queries scale based on data size


cost: pay per query/per TB of data scanned, serverless



Redshift Overview


is a great data warehouse system


based on PostgreSQL but not used for oltp


instead, its an OLAP – online analytical processing — analytics and data warehousing


10x better performance than other data warehouses, scales to PBs of data


columnar storage of data rather than row-based


it is MPP – uses a massively parallel query execution engine which makes it extremely fast

pay as you go acc to no of instances provisioned
has an SQL interface for performing the queries.


data is loaded from S3, DynamoDB, DMS and other DBs,
1 to 128 nodes possible with up to 128 TB of space per node!


leader node used for query planning and results aggregation
compute node : performs the queries and sends results to leader node


Redshift Spectrum: performs queries directly against S3 with no load to load the data into the redshift cluster


Backup & Restore, SecurityVPC, IAM, KMS, monitoring


Redshift Enhanced VPC Routing: Copy / Unload goes through VPC, this avoids public internet


Redshift Snapshots and DR


has no multi-AZ node


the snapshots are p-i-t- point in time backups of a cluster, stored internally in S3


they are incremental — only changes are saved, makes it fast

can restore to a new cluster

automated snapshots: every 8 hrs every 5gb or according to a schedule, with a set retention period


manual snapshots: snapshot is retained until you delete it


neat feature:
you can configure Redshift to auto copy snapshots to a cluster of another region
either manually or automatically



loading data into redshift:


there are three possible ways:


1. use kinesis data firehouse loads data into redshift cluster via an s3 copy automatically to an S3 bucket


2. using the copy command manually without Kinesis
from S3 bucket via internet – ie without using enhanced vpc routing to redshift cluster
or via vpc not via internet, using enhanced vpc routing



3. from ec2 instance to redshift cluster using jdbc driver
in this case it is much better to write the data in batches rather than all at once.



Redshift Spectrum


must already have a redshift cluster operational


Spectrum is a way to query data that is already in S3 without having to load it into Redshift


you submit the query which is submitted to thousands of Redshift spectrum nodes for processing



Operations: similar to rds
security: uses iam, vpc, kms, ssl as for rds
reliability: auto healing, cross region snapshop copies possible
performance: 10x performance of other data warehousing systems, uses compression
cost: pay per node provisioned about 10% of cost of other dw systems
vs athena: is faster at querying, does joins, aggregations thanks to indexes


redshift = analytics/BI/Data Warehouse



Glue Overview


is a managed extract transform and load ETL service


used to prepare and transform data for analytics
fully serverless service


fetch data from s3 bucket or rds, send to glue which does extracting, transforming and loading to redshift data warehouse


glue data catalog: catalog of all the datasets you have in aws – ie metadata info about your data


you deploy glue data crawler to navigate s3, rds, dynamo DB, it then writes the metadata to the glue data catalog.


this can then be used b Glue Jobs eTL


data discovery -> Athena, Redshift Spectrum, EMR for analytics


all you need to know about Glue for exam




Neptune Overview


if you hear graphs mentioned in exam, this refers to neptune!


is a fully managed graph database


used for
high relationship data
social networking, eg users friends with users, replied to comment on post of user and likes other comments


knowledge graphs eg for wikipedia – links to other wiki pages, lots of these links
this is graph data – giving a massive graph


highly available across 3 AZs with up to 15 read replicas available


point in time recovery, continuous backup to S3


support for kms encryption, https, and ssl


operations: similar to rds
security: iam, vpc, kms, ssl – similar to rds, plus iam authentication
reliability: multi-AZ and clustering
performance: best suited for graphs, clustering to improve performance


cost: similar to rds, pay per provisioned node


remember neptune = graphs database



AWS OpenSearch


– the successor to AWS ElasticSearch


eg dynamodb only allows search by primary key or index,


whereas with OpenSearch you can search ANY field


often used as a complement to other db


also has usage for big data apps
can provision a cluster of instances
built-in integrations: various – kinesis data firehose, aws IoT, CloudWatch Logs



comes with visualization dashboard


operations: similar to rds


security: is via Cognito, IAM, KMS encryption, SSL and VPC
reliability: multi-AZ and clustering
performance: based on elasticsearch project – open source, pbyte scale
cost: pay per provisioned node


all you need to remember:


used to search and index data


Question 1:
Which database helps you store relational datasets, with SQL language compatibility and the capability of processing transactions such as insert, update, and delete?




Question 2:
Which AWS service provides you with caching capability that is compatible with Redis API?



Good job!
Amazon ElastiCache is a fully managed in-memory data store, compatible with Redis or Memcached.



Question 3:
You want to migrate an on-premises MongoDB NoSQL database to AWS. You don’t want to manage any database servers, so you want to use a managed NoSQL database, preferably Serverless, that provides you with high availability, durability, and reliability. Which database should you choose?


Amazon DynamoDB


Good job!

Amazon DynamoDB is a key-value, document, NoSQL database.



Question 4:
You are looking to perform Online Transaction Processing (OLTP). You would like to use a database that has built-in auto-scaling capabilities and provides you with the maximum number of replicas for its underlying storage. What AWS service do you recommend?


Amazon Aurora


Good job!

Amazon Aurora is a MySQL and PostgreSQL-compatible relational database. It features a distributed, fault-tolerant, self-healing storage system that auto-scales up to 128TB per database instance. It delivers high performance and availability with up to 15 low-latency read replicas, point-in-time recovery, continuous backup to Amazon S3, and replication across 3 AZs.




Question 5:
As a Solutions Architect, a startup company asked you for help as they are working on an architecture for a social media website where users can be friends with each other, and like each other’s posts. The company plan on performing some complicated queries such as “What are the number of likes on the posts that have been posted by the friends of Mike?”. Which database do you recommend?


Amazon Neptune


Good job!
Amazon Neptune is a fast, reliable, fully-managed graph database service that makes it easy to build and run applications that work with highly connected datasets.



Question 6:
You have a set of files, 100MB each, that you want to store in a reliable and durable key-value store. Which AWS service do you recommend?


Amazon S3


Good job!
Amazon S3 is indeed a key-value store! (where the key is the full path of the object in the bucket)


Question 7:
You would like to have a database that is efficient at performing analytical queries on large sets of columnar data. You would like to connect to this Data Warehouse using a reporting and dashboard tool such as Amazon QuickSight. Which AWS technology do you recommend?


Amazon Redshift


Good job!
Amazon Redshift



Question 8:
You have a lot of log files stored in an S3 bucket that you want to perform a quick analysis, if possible Serverless, to filter the logs and find users that attempted to make an unauthorized action. Which AWS service allows you to do so?


Amazon Athena


Good job!
Amazon Athena is an interactive serverless query service that makes it easy to analyze data in S3 buckets using Standard SQL.



Question 9:
As a Solutions Architect, you have been instructed you to prepare a disaster recovery plan for a Redshift cluster. What should you do?


enable automated snapshops, then configure your redshift cluster to autocopy the snapshots to another aws region


Good job!



Question 10:
Which feature in Redshift forces all COPY and UNLOAD traffic moving between your cluster and data repositories through your VPCs?



Enhanced VPC Routing


Good job!


Question 11:
You are running a gaming website that is using DynamoDB as its data store. Users have been asking for a search feature to find other gamers by name, with partial matches if possible. Which AWS technology do you recommend to implement this feature?




Good job!
Anytime you see “search”, think ElasticSearch.



Question 12:
An AWS service allows you to create, run, and monitor ETL (extract, transform, and load) jobs in a few clicks.


AWS Glue


Good job!
AWS Glue is a serverless data-preparation service for extract, transform, and load (ETL) operations.















Continue Reading

AWS DynamoDB

DynamoDB is a fully managed, highly-available database with replication across multiple AZs


NoSQL – not a relational database!


scales to massive workloads


100TBs of storage


fast and consistence, low latency retrieval


integrated with iam for security authorization and admin
enables event-driven programming via DynamoDB Streams


low cost and auto-scaling


standard and infrequent access 1A table class



Basics of DynamoDB



v important – also exam q:
made up of tables – there is NO SUCH THING AS A DATABASE – you only need to create tables!
the db already exists!


each table has a primary key – must be set at creation time
each table can have an infinite number of items ie rows
each item has attributes can be added over time or can be null


max size of an item is 400KB so not good for large objects


data types supported are:


scalar types: string, number, binary, boolean, null
document types: list, map
set types: string set, number set, binary set




have primary key – containing partition key (eg user_id) and sort key (eg game_id)
attributes: eg score, result – these are items which are NOT the primary key


read/write capacity modes


control how you manage your table capacity – ie the read and write throughput


we have 2 possible modes:


provisioned mode, where you specify number of read and writes per second, this is the default, you have to plan the capacity for this in advance, you pay for provisioned read capacity units RCUs and write capacity units (WCUs)


these “capacity units” are used to set your desired capacity for your database! – set these in the web dashboard of dynamodb when you are provisioning.



you can also add auto scaling mode for both rcu and wcu


on-demand mode:

read write automatically scales up and down acc to workload, so no capacity planning needed
you pay for your use, is more expensive 2-3x more


good for unpredictable workloads which can be large or small varying


need to know for exam!


remember always you just create tables, never a database with dynamodb!


DynamoDB Accelerator DAX



a fully manageable, highly available seamless in-memory cache for dynamodb


helps solve read congestion by means of memory caching


microseconds latency for cached data


does not require any application logic modification – so it is fully compatible with existing dynamodb api


applications can access the database via DAX for much faster reads


TTL is 5 mins default for dax data

dax is different from elasticache in that it is meant for dynamodb and does not change the api
food for individual object cache


good for storing aggregation result where further processing required



DynamoDB Stream


very easy – is an ordered stream of data which represents items when created or updated or deleted


can then be sent on to kinesis datastreams
can be read by lambda or kinesis client library apps


data is retained for 24 hrs



use cases:


reacting to changes in real time eg welcome emails to new users
to do analytics
to insert into derivative tables or into elastic serarch
or implement cross region replication



DynamoDB Global Tables



are cross-region


two way or multi-way replication


to provide for low latency across regions


it is
active-active replication


this means apps can read and write to the table from any region


must enable dynamodb streams for this


TTL expiry: automatically deletes items after an expiry timestamp


eg 1 month for each row in the table



DynamoDB Indexes – only need to know at high level for exam


basically, these allow you to query on attributes other than the primary key


2 types:


gsi – global secondary and lsi – local secondary


(all you need to know for now)


by default you query on the primary key, but you can use gsi or lsi to query on other attributes.





these allow you to write to 2 tables at the same time:


the transaction however MUST write to both or else none – in order to keep the tables accurate – so data consistency is maintained









Continue Reading

AWS Lambda

Serverless Services in AWS include:


API Gateway
Kinesis Data Firehose
Aurora Serverless
Step Functions


exam tests heavily on serverless knowledge!


AWS Lambda




virtual functions – server to manage
limited by time – short execution processes
runs on demand only, only billed when you are actually using it
the scaling is automated


the benefits of Lambda:


easy pricing – pay per request and compute time


free tier covers 1 million lalbda requests and 400k of GB compute time


integrated with all AWS services and programming languages
easy monitoring via CloudWatch
easy to allocate more resources per functions –
up to 10GB of RAM! possible


also, increasing RAM improves CPU and network


language support:


node.js – javascript
java 8
c .net core
c powershell
custom runtime api eg rust


lambda container image — this must implement the lambda runtime api


ecs and fargate are preferred for running arbitrary docker images



lambda integrates with


api gateway


cloudwatch events and eventbridge


cloudwatch logs
sns and sqs
cognito – reacts when a user logs in eg to a database


lambda use case:


thumbnail image creation


new image uploaded to s3 then triggers a lambda function to generate a thumbnail of the image
this is pushed to s3 and meta data to dynamo db.


another example:


a serverless CRON job to run jobs


but for cron you usually need to have a server running, but with lambda you can do this without a server!


eg cloudwatch events or eventbridge every hour triggers a lambda function, this is instead of the cronjob!


pricing for lambda:


pay per calls first 1mill requests are free


then 20c per 1 mill requests


pay per duration in increments of 1 ms


400k GBseconds of compute time per month is free, charges thereafter on rising scale


very cheap to run lambda so it is very popular



you can run jobs using many different program languages


you enter your code in lambda web console and lambda then runs the code for you.


you can have lambda respond to events from various sources – eg data processing, streaming analytics, mobile or iot backends


lambda takes care of scaling for your load, you don’t have to do anything here!
ie seamless scaling



to create a lambda function you have 4 possibilities:


author from scratch
use a blueprint – these are pre configured functions
container image
browse serverless app repository



Lambda Limits per region
important for exam…




mem allocation 128 mb to 10 gb in 1mb increments


max exec time is 900 secs


env variables 4kb


disk capacity in the function container in /tmp is 512 mb


concurrency executions 1000 – can be increased




function deployment size compressed .zip is 50mb


size of uncrompressed deployment code plus dependencies is 250mb


can use the /tmp to load other files at startup


size of env variables is 4kb


the exam may ask you question to see if you think lambda can be used or not acc to the requirement for the task… you need to know these above limits in order to judge suitability of lambda for the task.





if you are deploying a CloudFront cdn and you want to deploy lambda globally


how to implement request filtering


you can use lambda@edge for this


you deploy it alongside each region in your cloudfront cdn


you can use lambda to modify the viewer/origin requests and responses of cloudfront:


this can be:


after cloud front receives a request – viewer request
before cloud front forwards the request to the origin – origin request


after cloudfront receives the response from the origin – origin response
before cloudfront forwards the response to the viewer – viewer response


you can also generate responses to viewers without having to send a request to the origin!


important to know this high level overview for exam.


use cases:


website security/privacy


dynamic web application at the Edge


intelligent routing across origins and data

bot mitigation at the Edge


real-time image transformation
a/b testing
user authentication and authorization


user prioritization
user tracking and analytics


Continue Reading

AWS S3 Storage

S3 is the AWS object storage system. S3 stores objects in flat volumes or containers called buckets, rather than a hierarchical file system.


Buckets must have globally unique name – across ALL accounts of AWS – not just your own!


defined at region level – important!



bucket naming convention – must know this for exam!


no uppercase, no underscore,, 3-63 chars long,


must start with lowercase letter or number


must not be an ip address




are files, must have a key – the full path to thta file




s3://my-bucket/my_file.txt -> my_file.txt is the key


you can add “folder” names but they are just a prefix ie tag not a real file directory system



so if you have




then /my_folder/my_file.txt

is the object key for this file



object max size is 5TB but you cann only upload 5GB in one go, to upload larger objects, you have to use multi-part upload


metadata can be added, also tags


and version id system if enabled


you can block public access if you want for a bucket


you receive an ARN or amazon resource name for the bucket



2 ways to open an S3


in the console click on object action and open


or via the url public object url


but for this you must set the permission for access to the bucket


bucket must be public access


pre-signed url – you give the client temp credentials to open the file



you can version your files, but have to enable versioning at the bucket level. Best practice to use versioning, this protects against unintended deletes and you can roll back.


Any files existing before versioning is enabled will not have previous versions available.


if you click on objects -> list versions, you will see the available versions listed.


you can easily roll back in this way. So versioning should be enabled!



S3 Encryption


exam question!

4 methods of encryption for S3 are


SSE-S3 – encrypts S3 objects using keys managed by AWS


SSE-KMS – uses AWS key management service to manage the keys


SSE-C – to manage own keys yourself


Client-Side Encryption


important to know which is best suited for each situation!




managed by AWS S3


object encrypted server side


uses AES-256 algorithm


must set header “x-amx server-side-encryption”: “AES256”





managed by AWS KMS


gives you control over users and an audit trail


object is encrypted server side


set header “x-amx server-side-encryption”: “aws:kms”





server side with your own keys outside of aws


so s3 does NOT store the key


https has to be used for this as you will be sending your key


the actual client side data encryption key in every http header


s3 then uses the key to encrypt the data on s3 at the bucket.



Client Side Encryption


this happens before transmitting data to s3 at the client side


and decryption on the client side.


customer fully manages the keys and encryption/decryption.


there is a client library called aws s3 encryption client which you can use on your clients.



encryption in transit ssl or tls


https is “in flight” – you always use https for your in flight: mandatory


uses ssl or tls certificates




S3 Security



first there is user-based


uses IAM poliicies, which api calls are allowed from a specific user




sets bucked wide rules from S3 console, allows cross account



not very common (NOT in exam)

oacl – object access control list acl – this is finer grain


bacl – bucket access control list acl – this is less common



IAM principal can access an S3 object if


user iam permissions allow it or the resource policy allows it


and no explicit deny exists



s3 bucket policies


they are json based policies


actions: they allow a set of api to allow or deny


principle is the account or user the policy applies to


use the s3 bucket policy to


grant public access to the bucket

force encryption at upload to the bucket

grant access to another account – cross account access



Bucket settings for block public access


used to block public access


3 kinds:


new acls
any acl
new public bucket or access point policies


block public or cross account access to buckets or objects through ANY public bucket or access point policies.


but exam will not test you on these.


created to prevent company data leaks


can also be set at account level


networking: supports vpc endpoints


S3 Logging and Audit


s3 access logs can be stored in other s3 buckets
api calls can be logged in CloudTrail


user security:


MFA Delete can be required to delete objects for versioned buckets


pre-signed urls valid for a limited time only


use case


eg to download a premium product or service eg video if user is logged in as a paid up user or has purchased the vid or service




S3 Websites


s3 can host static websites for public access


url will be






if you get 403 forbidden error then make sure bucket policy allows public reads – bucket must be publicly accessible for this.



CORS Cross-Origin Resource Sharing


web browser mechanism to allow requests to other origins while visiting the main origin


eg http://example.com/app1 and http://example.com/app2


a CORS header is needed for this – and the other origin must also allow the request.



the web-browser does a “pre-flight request” first – asking the cross origin site if it is permitted – then if yes, then eg get put delete


these are the cors method access-control-allow-methods





exam question!
if a client does a cross-origin request to an “3 bucket then you must enable the correct CORS headers.


you can allow for a specific origin, or for * ie all origins



S3 MFA Delete


forces a user to generate a code from a device eg mobile phone before doing some operations on S3


to activate MFA-Delete enable versioning on the S3 bucket


it is required to permanently delete an object version

and to suspend versioning on the bucket


not needed to enable versioning or list deleted versions




only bucket owner ie root account can enable/disable MFA-Delete


only possible via CLI at present.




first create an access key for the bucket in the web console of iam



then configure the aws cli to use this key


download the key file, and then set up a cli with your access key id and secret access key




aws configure –profile root-mfa-delete-demo


you are then prompted to enter the access key id and secret access key


then you run


aws s3 ls –prifle root-mfa-delete-demo to display



then do:


aws s3api put-bucket-versioning –bucket demo-mfa-2020 –versioning-configuration Status=Enabled, MFADelete=Enabled –mfa “<here enter the arn-of-mfa-device and-the-mfa-code-for-the-device>” –profile root-mfa-delete-demo


you can then test by uploading an object


and try deleting the version – you should get a message saying you cannot delete as mfa authentication delete is enabled for this bucket…


so to delete you must use the cli mfa-delete command and your chosen device eg mobile phone mfa – or alternatively for this demo just disable the mfa-delete again. then you can delete as per usual.




To force encryption you can set a bucket policy which refuses any api “put” calls to an object that does not have encryption headers


alternatively you can use the default encryption option of S3


important for exam: bucket policies are evaluated *before* default encryption settings!




S3 Access Logs


– you can log to another bucket, or you can use AWS Athena

first, very important – and potential exam question!


NEVER set your logging bucket to be the monitored bucket or one of the monitored buckets! because this will create a big infinite loop! which means a huge AWS bill!


always keep logging bucket and the monitored bucket/s separate! – ie set a separate different target bucket that is NOT being logged!


tip: make sure you define a bucket with the word “access-logs” or similar in the name, so that you can easily identify your login bucket to avoid logging it by mistake.


S3 Replication – Cross Region (CRR) and Single Region (SRR)


– must enable versioning


the copying is asynchronous


buckets can belong to different accounts


must grant proper iam permissions to S3 for this


CRR: you synchronize against different regions


used for compliance, lower latency access, replicating across different accounts


SRR: used for log aggregation, live replication between eg production and test and development accounts


note: after activating only the new objects get replicated. to replicate existing objects, you need to use…

S3 Batch Replication feature


for DELETEs: you can replicate the delete markers from source to target

but deletions with a version id are not replicated – this is to avoid malicious deletes


and there is no “chaining” of replication…

this means eg if bucket 1 replicates to bucket 2 and 2 replicates to bucket 3, then bucket 1 is not automatically replicated to 3 – you have to explicitly set each replication for each pair of buckets.


first you have to enable versioning for the bucket


then create your target bucket for the replication if not already in existence – can be same region for SRR or a different region for CRR


then select in origin bucket:


management -> replication rules, you create a replication rule and you set the source and destination.


then you need to create an iam role:


and specify if you want to replicate existing objects or not


for existing objects you must use a one time batch operation for this



S3 Pre-Signed URLs


can generate using sdk or cli


uploads: must use sdk
downloads can use cli, easy


valid fo 3600 secs ie 1hr default, can change


users are given a pre-signed url for get or put


use cases:
eg to only allow logged in or premium uses to download a product eg video or service
allow a user temporary right to upload a file to your bucket



S3 Storage Classes


need to know for exam!

S3 offers


Standard General Purpose, and
Standard-Infrequent-Access (IA)
One Zone-IA
Glacier Instant Retrieval
Glacier Flexible Retrieval
Glacier Deep Archive
Intelligent Tiering



can move objects between classes or use S3 Lifecycle Management service


Durability and Availability




S3 has very high durability, 99.9 11×9!




how readily available the service is available.


S3 standard is about 1 hr per year out of availability




big data, mobile, gaming, content distribution




less frequent access
lower cost than standard


99.9% available


good for DR and backups




v high durability but 99.5% availability – not so high


thus best used for secondary backup copies or recreatable data


Glacier storage classes:


low-cost object storage for archiving or backup


you pay for storage plus a retrieval charge


Glacier Instant Retrieval IR


millisecond retrieval, min storage 90 days


Glacier Flexible Retrieval


1-5 mins to recover


standard 3-5 hrs



bulk 5-12 hrs – is free
min storage duration 90 days


Glacier Deep Archive


best for long-term only storage


12 hrs or bulk 48 hrs retrieval
lowest cost


min 180 days storage time



intelligent tiering


allows you to move objects between tiers based on monitoring
no retrieval charges

enables you to leave the moving between tiers to the system























Continue Reading

AWS ElastiCache for Redis/Memcached

ElastiCache is an in-memory DB cache. 


ElastiCache supports two open-source in-memory caching engines: Memcached and Redis


applications query ElastiCache (EC), called a “cache hit”


– if not available it seeks from RDS and then stores in EC


this relieves load on RDS


cache must have an invalidation strategy set in order to ensure the most relevant data is cached -not so easy in practice


is used for managed Redis or 1memcached


is an in-memory db with v high performance and low latency,


reduces load for read-intensive workloads


helps the application operate stateless


AWS takes care of maintenance, upgrades, patching, backups etc.


BUT you have to make major application code changes to query EC!


EC can also be used for DB user session store


which avoids users having to reauthenticate and relogin to the DB



Difference between Redis and Memcached




is similar to RDS:


allows for multi-az with auto-failover


use read replicas


data durability and high availability rather like RDS


backup and restore features





uses multi-node for partitioning of data, known as sharding


NO high availability and no replication


it is not a processing cache, so not persistent


no backup and no restore


has multi-threaded architecture



pure cache with no high availability or data protection for failure – a simpler option


Deployment Options


Amazon ElastiCache can use on-demand cache nodes or reserved cache nodes.


On-demand nodes provide cache capacity by the hour, resources are assigned when a cache node is provisioned.


You can terminate an on-demand node at any time. Billing is monthly for the actual hours used.


Reserved nodes use 1-year or 3-year contracts.


Hourly cost of reserved nodes is much lower than hourly cost for on-demand nodes.




REDIS Use Case:


need to know for exam!


Gaming Leaderboards, as these require intensive processing-


REDIS uses “sorted-sets”, this ensures accurate real-time data is generated for the leaderboard – important!



EC Security – important for exam


The EC caches do not use IAM policies, except for AWS API-level security




you use Redis Auth to authenticate – sets a password/token when creating the redis cluster


on top of security groups


supports inflight ssl





uses SASL authentication (more advanced)


3 kinds of patterns for EC – need for exam


lazy loading – all read data is put in the cache, but can become stale, but future data is only loaded when it cant find the data present in the cache, it reads from db and then copies to cache

write-through – adds or updates cahce data when written to DB – there is no stale data


Session Store: stores temp session data in cache using a TTL




DB TCP Ports


PostgreSQL: 5432


MySQL: 3306


Oracle RDS: 1521


MSSQL Server: 1433


MariaDB: 3306 (same as MySQL)


Aurora: 5432 (if PostgreSQL compatible) or 3306 (if MySQL compatible)


For the exam make sure to be able to differentiate an “Important Port” vs an “RDS database Port”.


Continue Reading

AWS Aurora

AWS Aurora is a high-performance highly available database engine for AWS.


Proprietary AWS tech, not open source


supports Postgres and MySQL db


is AWS cloud optimized and claims 5x performance improvement over MySQL on RDS


and 3x improvement on Postgres on RDS


Storage grows automatically in blocks of 10GB to 128TB


can have up to 15 replicas vs 5 mysql replicas


replication also much faster and failover is almost instantaneous,
failover very fast within 30 secs, and it is HA high availability native


self-healing with peer-to-peer replication


and supports cross-region replication


stores 6 copies of your data across 3 AZs


4 copies of 6 needed for writes
3 out of 6 needed for reads


provides a writer endpoint that points to the master db.

you can have asg auto-scaling on top (but max 15 replicas)



patching, updating etc is done by AWS


also provides BackTrack service – you can restore a point in time without any extra backup routine


important for the exam!


READER ENDPOINT connects automatically to ALL the read replicas, so this provides for connection level load balancing for the read replicas


storage is highly striped across 100s of volumes


security similar to RDS


encryption at rest using KMS



automated backups, snapshots and replicas are also encrypted


encryption in flight uses SSL


you can authenticate using IAM, same as with RDS


but you are responsible for protecting your instance with security groups



also important – for exam!


you cannot SSH into Aurora





Aurora Read Replica Auto Scaling


You create a writer endpoint for DB writes, while reads go to a single separate reader endpoint, which connects to multiple aurora DBs.


As auto scaling takes place and more read DBs are added, these are connected to the reader endpoint


However, you can also create a separate custom endpoint for specific other traffic purposes, eg a read analytics software which needs to connect and which generates intensive traffic load


or you might want to have a set of read replicas which have different instance type to the others, again, you can use custom endpoints for this, this creates an additional endpoint.


Aurora Serverless


automated db instance and auto-scaling


no need for capacity planning


use case:


infrequent, unpredictable workloads


billing is pay per second, can be more cost-effective


client talks to a proxy-fleet and in the backend Aurora creates the necessary instances.



Important for EXAM!

Aurora Multi-Master


every node is Read-Write


useful for immediate failover for the WRITE node – offers high availability for the writer node.


Global Aurora


you can have


Cross Region Read Replicas


useful for Disaster Recovery (DR)


easy to set up


Global Database (recommended)


you have one primary region for read-write


and up to 5 secondary read only regions with v low replication lag
plus up to 16 read replicas per secondary region


provides for very low latency


when you promote another region for DR the RTO recovery time overhead is less than 1 minute.


Aurora Machine Learning


you can add ML predictions to apps via sql



– sagemaker
– comprehend – for sentiment analysis


can be used for fraud detection, ad targeting, sentiment analysis, product recommendations


Continue Reading

ASG Auto Scaling Groups

Auto Scaling Groups or ASGs provide a way to scale in and out with your instances and infra.


Scale out: add instances for increased workload


Scale in: remove instances for decreased workload


you can set a minimum, desired, and maximum capacity no of instances


automatically connect new instances to a load balancer


recreate an instance if a previous one is terminated eg if unhealthy


ASG is FREE! – but you pay for the underlying instances


ASGs also work with load balancers


You create a Launch Template (used to be called Launch Configuration – now deprecated)


in this you set:


AMI and instance type


EC2 user data script, if used
EBS volumes
security groups
ssh key pair
IAM roles for the instances
network/subnet info
lb info


plus also set
min,max,initial capacity
scaling policies


CloudWatch Alarms also integrate with ASGs.


you set a metric to be monitored by CW and this then triggers the ASG activity eg scale in policy or scale out policy.




ASG Termination Policy

by default, ASG looks for the AZ which has the most number of instances, and then deletes the one within that AZ which has the oldest launch configuration.

ASG always seeks to balance number of instances across AZs by default.



Lifecycle Hooks


when an instance is launched you can first determine that it goes first into a pending state, you then move it into pending proceed.. then it goes into in service state.


if no pending state, then it goes straight to in-service state.



also for teminating, you can set a terminating wait state, this is so you have time to carry out some other actions first.



Launch Configs – these are legacy deprecated, you have to recreate each time.
Launch Templates – new, are versionable, recommended by AWS


only use Launch Templates from now on!







Continue Reading

AWS Load Balancers

NOTE: health checks for EC2 instances are crucial when using  load balancers, because you do not want to send traffic to an EC2 instance or other service if it is not working properly.


You set up your security group for the load balancer, your endpoints eg EC2 instances should only accept traffic from the load balancer security group and not from the external internet. This is an enhanced security mechanism.




Types of Load Balancer in AWS



ELB Elastic Load Balancer

CLB Classic Load Balancer (deprecated)

ALB Application Load Balancer

NLB Network Load Balancer

GWLB Gateway Load Balancer





ELB Elastic Load Balancer


is a managed load balancer,

aws guarantees it will work, takes care of upgrades and availability

costs more than setting up your own load balancer, but is more convenient and less overhead for you

is integrated with many aws services




CLB Classic Load Balancer



is deprecated, don’t use for new installs


operates on tcp layer 4 and http/https layer 7


health checks are based on above


fixed hostname


ALB Application Load Balancer


works at layer 7 http


balances to multiple http servers machines ie target groups


also can be multiple applications on SAME machine eg via containers


supports websocket as well


and redirects from http to https


can route acc to target url path eg example.com/users and example.com/posts


also based on hostname eg


one.example.com and two.example.com


also query string or headers in the url


good for micro services and container-based apps eg docker and amazon ecs


also have port mapping feature


comparison with old classic lb: you would need additional clbs to do the same with one alb if you want different routing




NLB Network Load Balancers


operates at layer 4


forwards TCP/UDP traffic to instances


high volume traffic, millions of requests per sec

low latency 100ms vs 400ms for ALB


NLB has one static ip per AZ, supports Elastic IP


Useful for having 2 incoming points for traffic to your network


use case:


when you need extreme performance or tcp udp traffic


Note: NLB is NOT in the free-tier pricing!



GWLB  Gateway Load Balancer 


esp used for firewalls, intrusion detection, prevention system (IDS/IDPS), deep packet inspection systems etc


can also be used to manage a fleet of 3rd party network virtual appliances running on aws


operates at layer 3 network layer ip packets


has 2 functions:


1. transparent network gateway – a single point of entry/exit for traffic


2. load balancer to distribute traffic to your virtual appliances


exam tip:
GENEVE protocol port 6081 is the gateway load balancer


EC2s must be private addresses for GWLB



Sticky Sessions or Session Affinity


this means the same client is always connected to the same instance behind a load balancer to complete a transaction


this works for CLBs and ALBs


uses a cookie with an expiry date.


this is to ensure a user does not lose his session data


but – it can cause an imbalance within the balanced load cluster





application-based cookie

– custom cookie, is generated by the target, can include any attribute


– application cookie – generated by load balancer, cookie name is AWSALBAPP


but some names are reserved: AWSALB, AWSALBAPP AWSALBTG


duration-based cookie


– generated by load balancer
cookie name is AWSALB for ALB and AWSELB for CLB





Cross-Zone Load Balancing


a point to note about cross-zone load balancing…


if this feature is  ON, then it will ensure each INSTANCE gets the equal amount of share of traffic as all other instances.


but if this feature is OFF , then it will vary between the instances depending on how many instances in each AZ, if this is unequal eg one AZ has fewer EC2s than others, then it will be unequally divided among the actual EC2s although equally shared out at the AZ 1lb level.


Be aware:


CZ-LB is enabled by default for ALB – and cannot be disable –  but for NLB it is disabled by default – but you pay extra if you want to enable it. 


but for CLB: it is disabled, but you can enable, and it is free to enable




SSL/TLS and AWS Load Balancers


encrypts via “in-flight” in-transit encryption


SSL: secure sockets layer

TLS: transport layer security, the newer ssl version


public SSL certificates are issued by certificate authorities (CAs)


eg Globalsign, Digicert, GoDaddy etc


have an expiry date, must be renewed


Load Balancer uses an X.509 SSL certificates, can be managed via ACM – the AWS certificate manager


you can create your own certificate


clients can also use SNI server name indication – client must declare which hostname it wants in the SSL handshake. Server then finds the correct SSL certificate or else returns the default one.




SNI Server Name Indication for SSL


solves problem of loading multiple SSL certificates onto one webserver to serve multiple websites.


only works with alb and nlb and cloudfront, not with clb



Elastic load balancer elb only supports


CLB – only 1 SSL certificate


must use multiple clbs for more than one certificate


ALB and NLB 


supports multiple SSL certificates and uses sni to make it work



Connection Draining and load balancers


CLB call it connection draining
ALB and NLB: call it deregistration delay


it allows some time for instances to complete in-flight SSL requests while instance is unhealthy or de-registering


it stops lb sending requests to the instance during this period


you can set a period of between 1 and 3600 secs, default is 300 secs, or disable, by setting to 0.


set a low value if requests are short


if there tend to be longer requests, eg for uploads, downloads etc… then set a higher value.


Continue Reading

AWS Snow Family

AWS Snow Family


for data migration in and out of AWS:




Snowball Edge




for Edge computing data migration:



Snowball Edge


These are offline devices that perform data migration for migration that would take more than a week by network transfer.


Snowcone and Snowball Edge are sent by post using a physical route rather than digitally.


Snowball Edge, used for TB or PB data transfer


pay per data transfer job



2 versions


Snowball Edge Storage Optimized:


80TB max of HDD capacity for block volume and S3 compatible object storage


Snowball Edge Compute Optimized:


42TB of HDD capacity for block volume and S3 compatible object storage



use case for Edge:


large data cloud migration, Disaster recovery prep, disaster recovery action


For Snowcone:


a much smaller device, can withstand harsh environments, very light and secure


used for smaller size transfers, up to 8TB storage


10 times less than snowball edge.


you must provide own battery and cables


can be sent to AWS in post or via internet – and use AWS DataSync to send data.



Snowmobile is a truck – very high-scale transfers.


request snowball devices on the AWS website


install a snowball client on servers


connect the snowball to your servers, then copy the file,


then ship the device back


many EBs of data 1EB = 1PB


very high security, staffed,


best for more than 10PB data transfer


Edge Computing:


somewhere creating data or needing data but has no internet access or limited access…


snowball edge or snowcone can be embedded into your edge site

so it gives you a way to transfer data despite having no strong internet connection


The Snowcone and Edge can run EC2 instances and Lambda functions!

good for preprocessing data, transcoding media screens, machine learning at the edge


Snowcone 2 CPUs 4GB RAM, wired/wi-fi, powered by usb-c or optional battery


Snowcone Edge:

many more vCPUs up to 52vCPUs and 208 GiB RAM
optional GPU
object storage clustering available


can get discount pricing 1 or 3 years



Also AWS OpsHub – management software for snow family


This enables you to manage the device from your client pc or laptop, you install on your client





Continue Reading

AWS CloudFront

CloudFront is the AWS CDN Content Delivery Service


CF offers:


ddos protection against webserver attacks
web application firewalls WAF and Shield
improved web content read service through edge region caching (currently 216 edge locations globally, number steadily increasing) ie content caching


can use both https for external traffic and also forward https traffic internally – latter not usually possible due to TLS certificate limitations


can be used for:


s3 buckets CF origin access identity or CF OAI


– this is an IAM role used for CloudFront connections to S3.



two ways to upload content to S3:



you can  use CF to upload files to S3 – ie ingress traffic


other option is to use Custom Origin http


can use ALB
can use EC2 Instance
can use S3 website – must enable the bucket as a static web instance
any http backend eg on premises webserver


two different architectures are possible acc to whether you have an ALB or not with CF.


if EC2 is the origin, (origin means your website content original site) then the security group for the EC2 must allow public access


ie the edge location sites to access the EC2 instance


if an ALB is attached, then the ALB security group MUST be public – but the EC2 instance can in this case be private access, ie need not give public access. in this case your ALB is your “origin”. 



these are two different architectures – be sure to understand for the exam!




the edge location always serves cached content originally from your S3 site ie your “origin”.



so, web users send request to your CDN CF IP at the edge location – if the content is not already present on the CF Edge Location then it forwards the request to your S3 bucket and sends the result back to the edge location of your CF ip… 



it traverses the security group, so the ip of the ec2 instances ie the origin, must be public and allow public ip of the edge location




CF Geo Restriction


You can whitelist or blacklist users from specific countries, eg for copyright etc reasons.


What is the difference between CF and S3 Cross Region Replication?




a global edge network

caches for a short TTL
good for static content


S3 Cross Region Replication:


must be set up for each region you want to replicate


files updated in near real-time


read only


good for dynamic content that needs to be available at low-latency in a few regions



Continue Reading

AWS EBS & EFS Elastic Block Storage & Elastic File System

EBS is a network drive you can attach to ONLY ONE INSTANCE at any one time



important: EBS is not a network file system!

is bound to an AZ


think of it like a network USB stick


30GB per month free of type SSD or magnetic GP2 or GP3 volume


EBS can have some latency on the AWS network


can be detached from one EC2 and attached to another, eg for failovers, big advantage!


but cannot be moved to other AZs, unless you do a snapshot and then move.


have to provision in advance and the level of IOPS you want


you pay according to this after the first 30GB.



BUT you can have 2 EBS volumes attached to an EC2 – that is not limited.


however they are bound to an AZ


they can be attached on demand, do not have to be actively attached.



EBS Delete on Termination attribute – is enabled by default for EBS root volume


but not for other EBS volumes as it is disabled for the latter by default


you can change this…
advantage: to preserve root volume when EC2 instance is terminated.


You can transfer an EBS volume to another region or az by means of using snapshots.


You can move snapshots to archive which is much cheaper.


Snapshots are usually gone once deleted, but if you have recycle bin enabled then you can retrieve them for a limited time period according to retention rule you set.



EBS Volume Types


6 types:


gp2 /gp3 SSD general purpose balances price/performance


io1/io2 SSD high performance for mission critical low latency/high throughput workloads


st1 HDD low cost HDD for frequently accessed throughput intensive workloads


sc1 HDD lowest cost HDD for less frequently accessed workloads


EBS volumes are by size, throughput, iops


NOTE: only gp2/gp3 and io1/op2 can be used as boot volumes



General Purpose SSD


cost-effective storage


system boot vols, virtual desktops, dev and test env


1GiB – 16TiB




3000 iops and 125 miB/s


can go up to 16k iops and 1000 MiB/s




small vols can do burst iops to 3000


size of volume and iops are linked max iops is 16k


3 iops per GB means at 5,334 GB we have max iops



Provisioned iops ssd


for critical biz apps with sustained iops performance

or apps that need more than 16k iops


good for db workloads


io1/io2 4GiB – 16TiB


max piops 64k for nitro ec2 and 32k for others


can increase piops indep of storage size


io2 gives more durability and more iops per gib


io2 block express 4gib – 64tib


very low latency
max piops 256k



with hdd:


cannot be a boot vol

125 mib to 16tib


throughput optimized hdd st1:


big data, data warehousing and log processing


max througput is 500 mib/s max iops 500


cold hdd sc1


for infreq accessed data
where low cost is important
max throughput is 250 mib/s max iops 250




you *dont* need to know these details for the exam, but be aware of the main difference in the variations




EBS Multi-Attach – for io1/io2 family


attaches same ebs volume to multiple instances in same az at same time


each instance has full r/w permissions to the vol


use case:


for high app availability in clustered linux apps eg teradata


apps must be able to manage concurrent write ops


only thus for very specific workloads,


you must use a cluster aware file system ie NOT ext4 xfs etc.



EFS Elastic File System


is a managed NFS-type system that can be mounted on multiple EC2s in multiple AZs.


highly available, scalable, but expensive, you pay per usage per gigabyte


use cases:


content management,


web and data sharing, wordpress


uses nfs4.1


uses security groups to control access



compatible only with linuxbased ami’s and not windows!


can enable enctryption at rest using kms


is a posix linux system with standard file api


scales automatically by itself!


exam question:


efs performance and storage classes:


efs scaling:


can support 1000s of concurrent nfs clients 10gb throughput


can grow to petabyle size automatically


performance mode


– set at efs creation time
general purpose default – latency sensive use cases such as webserver, cms etc


to maximum i/o — gives higher latency, highly paralleluse for this io1 : this is best for big data applications, eg media processing etc


throughput mode
— bursting 1tb = 50mib/s and up to 100 mib/s


provisioned: set your throughput regardless of storage size eg 1 1gib/s per 1tb storage



EFS Storage Classes


you can set up storage tiers for lifecycle management


eg move to another tier after N days…




– standard tier – used for frequently accessed files
– infrequent access tier (IA) -efs-ia: costs to retrieve files


but lower price to store, enable efs-ia by means of a lifecycle policy



Availability and Durability of EFS


2 options:


standard: can set up EFS to be multi AZ
one-zone: you use one AZ only, backups are default enabled, compatible with IA


90% cost saving

exam will ask you which tier /storage class you should use for which use case, and you need to be aware of the cost implications!




Differences Between EBS & EFS


must know for exam!




can only be attached to only one instance at a time
are locked into an AZ


gp2. io increases if disk size increase
io1: can increase the io independently.


to migrate an EBS across AZs


– take a snapshot
– restore the snapshot to the other desired AZ


note that ebs backups use up io and so you should not run them when your app has heavy traffic overhead


also, root ebs volumes get terminated by default if or when the ec2 instance gets terminated! very important to be aware of this
– but you can disable this



EFS by comparison:


can mount on 100
s of instances across AZs!


it is multi-AZ, multi-client/instance


can be used to share data


only available for linux posix, not windows!



efs more expensive than EBS but can use efs-ia to reduce costs



so: efs is more for multi instances


ebs is more for one instance





Instance Store


instance store: is an ephemeral local instance drive just for an instance – you lose it with the instance when the instance is deleted.





Continue Reading

AWS Route 53

Route 53 is the AWS DNS service.


Highly available
authoritative DNS – you the customer can update the dns records
is also a domain registrar


AWS provides 100% SLA guarantee availability


You define how you route traffic to a specific domain.


domain name
record type eg A or AAAA
value ie ip number
routing policy
TTl – time to live for the record caching time


different record types


A, AAA, CNAME, NS – essential to know


A – maps hostname to ipv4 address
AAAA – maps hostname to ipv6 address
CNAME – maps hostname to another hostname
you cannot create CNAMES for top record of domain eg example.com but you can for eg www.example.com


NS – the name servers for the hosted zone


Hosted Zones


are a container for dns records



public hosted zones

for internet available ips. Any client can request


private hosted zones

for not publicly available ips, within VPCs which can only be accessed within the subnet
this enables you to make ips resolvable within the private network ie internally, not publicly via internet.



otherwise they work the same way – public hosted and private hosted.


you pay 50c per month per hosted zone



from your CLI you can then check your domain registration and ip records with
nslookup or dig


do apt install bind-utils -y to install them if not installed yet on the machine.


dig <your domain name and machine>


nslookup <your domain name and machine>




TTL Time To Live



set in seconds


TTL: client will cache a lookup for the set TTL time period, this is to relieve DNS server from too much request and response traffic


high TTL:


less traffic, lower costs
but possibly not up-to-date records


low TTL:


more traffic, more costly
records more likely to be fully accurate



CNAME vs Alias


you need to use Alias for mapping to an alias hostname, not CNAME!


sometimes you may want to map a hostname to another hostname


CNAME does this, but only for non-root domains ie eg www.example.com not example.cmm
The CNAME maps to the root domain name only!


Alias: this works for both root and non-root domains
always either A for ipv4 or AAAA for ipv6


you cannot set the TTL for Alias, this is set by Route 53 automatically


you can use as aliases things like


elastic load balancers ELB
API Gateways
Elastic Beanstalk
S3 websites
VPC interface endpoints
Global accelerator


Important: You *cannot* set an ALIAS for an EC2 DNS name!




Route 53 routing policies




you set the routing policy in the Route 53 Dashboard for the dns record



simple policy


you can specify multiple records in the same record but then a random one is chosen by the client


can’t be associated with health checks



weighted policy


you set a % of requests to go to each resource you specify


eg to different EC2 instances


to do this you assign each record a relative weight


the weights don’t need to add up to 100


but the DNS records involved must have same name and type

CAN be associated with Health Checks

use cases: load balancing between regions, testing new application versions


NOTE if you assign a weight of 0 to a record then the resource will not receive any traffic!


Also, if ALL records have a weight of 0 then all records will be equal! ie balanced responses






you want to redirect to the resource with the least latency, ie closest to us in terms of SPEED of internet

latency based on traffic between users and AWS Regions


so depends on traffic speed, not necessarily same as geographical closeness


Can use Health Checks





Health Checks


HTTP Health Checks are only for PUBLIC resources


If one region is down, then we can use a Health Check in Route 53.


These provide for automated DNS failover


the check can be against:

an endpoint eg app server

an other health check eg calculated health checks


cloudwatch alarms eg for dynamodb, rds


To pass an HTTP health check the endpoint must respond with 2xx or 3xx status codes


you can combine up to 256 health checks into a single health check using OR, AND, or NOT 


and define how many must pass 


How to perform health checks for private hosted zones

use a CloudWatch Metric and Alarm then create a Health Check that monitors the alarm!





Failover Policy (Active-Passive Failover)


you associate your DNS record with a health check – essential for this


but you can only have one primary and one secondary record


so, you need to first set up your health check for each machine,


then you reference them in your dns records


policy: set to failover, and the type: primary or secondary


and then associate the respective health check you have defined.


then do the same for the secondary record and instance.






this is where user is physically based


use cases:


website localization
restrict content distribution
simple load balancing method




enables you to specify “bias values” for specific geo regions


1- 99: more traffic to the resource
-1 to -99: less traffic to the resource


can be for AWS resources, specifying aws-region or non-AWS resources , specified by latitude/longitude


exam tip:


this can be useful when you need to shift traffic from one region to another




Multi-Value Policy


multi-value or multi-value answer is similar to an ELB but it is a client-side load balancer in effect.


used to route traffic to multiple resources but the client chooses which to use


can associate with Health Checks – up to 8 checks for each multi-value query


NOT a substitute though for an ELB!




Route 63 Traffic Policies


You can use these to define your DNS policy.



These make it easier to set policies.


Continue Reading