Amazon EC2 has two ways to recover an instance when an underlying host problem causes a system status check to fail: simplified automatic recovery and CloudWatch action-based recovery. Simplified recovery is enabled by default for supported instances. A CloudWatch alarm is an optional way to choose the alarm threshold and receive notifications. Neither mechanism repairs a guest operating system or application failure, and neither guarantees that recovery will succeed.

What EC2 instance recovery does

Instance recovery attempts to move a running instance from an impaired host to another host. If it succeeds, the instance comes back with the same instance ID, Availability Zone, network addresses, attached EBS volumes, and instance metadata. The operating system experiences an unplanned reboot: memory contents are lost and uptime starts over. CloudWatch action-based recovery also loses data on instance store volumes. Keep durable application data on EBS or another appropriate persistent service, and maintain backups for data you cannot recreate.

Recovery addresses an individual instance. It does not replace a multi-instance or multi-AZ design, recover a terminated instance, or prove that the application and its dependencies are healthy. For broader EC2 storage and availability trade-offs, see Amazon EC2 explained. For application-level recovery exercises, see our AWS disaster recovery testing guide.

AWS documents two separate mechanisms in its automatic instance recovery guide:

  • Simplified automatic recovery is enabled by default at launch for supported instance types. It does not require a CloudWatch alarm. AWS reports attempt outcomes through AWS Health events.
  • CloudWatch action-based recovery uses a CloudWatch alarm on StatusCheckFailed_System. It lets you choose the alarm evaluation and add an SNS notification, but requires an eligible instance and alarm configuration.

Check the failure type and instance eligibility

Choose the recovery path based on the failed check. EC2 status checks run automatically and report whether the instance or its underlying infrastructure is impaired. A recovery action cannot fix every cause of an outage.

Metric What a failure indicates Appropriate response
StatusCheckFailed_System An underlying AWS host or network problem, such as loss of power or host connectivity. Simplified recovery may act automatically. A CloudWatch recover action can be configured for this metric.
StatusCheckFailed_Instance A problem in or affecting the guest operating system, such as incorrect networking, startup configuration, or exhausted memory. EC2 can also report an instance check failure when a system check fails. Investigate the guest. A CloudWatch reboot action can be used for an instance check; the recover action does not support this metric.

See AWS's EC2 status check descriptions for the full distinction. A high CPU alarm is unrelated to host recovery, and an instance check by itself does not permit the recover action.

Recovery support depends on the recovery mechanism, instance type, and configuration. AWS maintains the current requirements for simplified recovery and CloudWatch action-based recovery. Both mechanisms require the instance to be running, available replacement capacity, and no blocking AWS Health service event. Neither supports instances in an Auto Scaling group, using an Elastic Fabric Adapter, on a Dedicated Host, or undergoing scheduled maintenance. Simplified recovery also excludes metal instance sizes and instances with instance store volumes. CloudWatch action-based recovery supports metal sizes and some instance-store configurations; for those supported configurations, the instance store data is lost during recovery.

Do not rely on an old instance-family list. For CloudWatch action recovery, check the current support flag in the EC2 console's Instance types page, or query it in the target Region:

aws ec2 describe-instance-types \
  --filters Name=auto-recovery-supported,Values=true \
  --query "InstanceTypes[*].[InstanceType]" \
  --output text \
  --region us-east-1

This filter identifies instance types that support CloudWatch action-based recovery; it does not remove the other restrictions above. The EC2 console also prevents adding a recover action to an unsupported instance.

Create a CloudWatch recovery alarm

Use an alarm when you need a configurable response to a system status check failure or want an SNS notification with the recovery result. If the default simplified mechanism meets your needs, a CloudWatch alarm is not required to enable it.

Choose the system status check metric

For recovery, use the per-instance StatusCheckFailed_System metric in the AWS/EC2 namespace. It reports 0 when the system check passes and 1 when it fails. Set the threshold to greater than 0. The CloudWatch console procedure from AWS uses the Minimum statistic and two of two breaching periods; its EC2 console procedure shows Average and two consecutive periods. The CLI example below follows the CloudWatch procedure and requires the system check to fail throughout both periods.

AWS's recover-alarm procedure uses two one-minute periods. The StatusCheckFailed_System metric is available at one-minute frequency by default at no charge, so it does not require paid detailed monitoring. If an EC2 console workflow does not offer a one-minute period, create the alarm through the CloudWatch console or CLI. See AWS's status-check metric reference and recover action procedure.

Understand the IAM permissions

The identity creating or modifying an EC2-action alarm needs permission to create CloudWatch alarms and iam:CreateServiceLinkedRole. CloudWatch creates and uses the AWSServiceRoleForCloudWatchEvents service-linked role for EC2 alarm actions; that role includes ec2:RecoverInstances. You do not add this permission to the EC2 instance profile. See AWS's documentation for CloudWatch service-linked roles.

Create the alarm in the console or CLI

In the EC2 console, select the instance, then choose Actions > Monitor and troubleshoot > Manage CloudWatch alarms. Create an alarm for Status check failed: system, set the threshold to greater than zero, use two consecutive one-minute periods, and choose Recover as the alarm action. Add an existing SNS topic if operators should receive notifications. Some EC2 console workflows may disable the one-minute period unless detailed monitoring is enabled; that is not a requirement of the status-check metric. If the form blocks a one-minute period, create the metric alarm in the CloudWatch console or with the CLI instead of enabling paid detailed monitoring for this metric. The console checks whether the selected instance supports the recovery action.

For a two-of-two one-minute alarm with SNS notifications, replace the example instance ID, Region, account ID, and topic with values from your account. The SNS topic must already exist, and email subscribers must confirm their subscriptions.

aws cloudwatch put-metric-alarm \
  --alarm-name "recover-system-check-i-0123456789abcdef0" \
  --namespace AWS/EC2 \
  --metric-name StatusCheckFailed_System \
  --dimensions Name=InstanceId,Value=i-0123456789abcdef0 \
  --statistic Minimum \
  --period 60 \
  --evaluation-periods 2 \
  --datapoints-to-alarm 2 \
  --threshold 0 \
  --comparison-operator GreaterThanThreshold \
  --treat-missing-data missing \
  --alarm-actions arn:aws:automate:us-east-1:ec2:recover arn:aws:sns:us-east-1:123456789012:ec2-recovery-alerts \
  --region us-east-1

The recovery action ARN follows the arn:aws:automate:REGION:ec2:recover form; the AWS CLI reference lists this as a valid alarm action. The alarm's region, metric dimension, and action ARN must refer to the same Region and instance. Use a new alarm name: put-metric-alarm replaces the configuration of an existing alarm with the same name.

EC2 status-check metrics can briefly have missing data even when an instance is healthy. AWS recommends treating missing data as missing for alarms with EC2 actions, rather than treating it as a failure. The command sets that behavior explicitly. See CloudWatch missing-data guidance.

Observe alarm and recovery outcomes

CloudWatch action-based recovery can publish alarm and recovery outcome messages to the SNS topic associated with the alarm. Confirm each email subscription; an unconfirmed subscription will not receive email. AWS Health also records CloudWatch recovery outcomes as AWS_EC2_INSTANCE_AUTO_RECOVERY_SUCCESS or AWS_EC2_INSTANCE_AUTO_RECOVERY_FAILURE. Simplified recovery uses AWS_EC2_SIMPLIFIED_AUTO_RECOVERY_SUCCESS and AWS_EC2_SIMPLIFIED_AUTO_RECOVERY_FAILURE. You can view these events in AWS Health or route supported events with EventBridge.

Check the CloudWatch alarm history, the instance's status checks, and the relevant AWS Health event after an attempt. An alarm action runs when the alarm enters ALARM; it is not repeatedly invoked just because the metric remains in that state. A later attempt requires another state transition. AWS also enforces a maximum daily allowance for recovery attempts, but does not publish a fixed attempt count in the recovery guide.

Troubleshoot a recovery that did not complete

First confirm that the alarm watches the correct instance ID in the correct Region, uses StatusCheckFailed_System, and reached ALARM. Then check that the instance is running, its type and configuration are eligible, and the alarm has the recover action. If you cannot create the action, verify the instance's recovery support and the caller's CloudWatch and service-linked-role permissions.

A configured alarm does not guarantee a successful migration. Recovery can fail when AWS reports a service event, replacement capacity is unavailable, or the instance has reached its daily recovery-attempt limit. AWS may require manual intervention if the system check continues to fail. Review the recovery event and follow the relevant CloudWatch recovery troubleshooting guidance; do not assume that retrying the same alarm immediately will resolve a host or capacity issue.