When an AWS DataSync transfer fails, start with the task execution record and follow the failing phase to the agent, network path, storage location, or task option. This guide uses read-only AWS CLI commands to inspect a run, then shows what to correct before you retry. For a broader migration plan, see AWS Content Migration Best Practices.

1. Read the task and execution state

A task defines the source, destination, and transfer options. Each run is a task execution. Set the task ARN and inspect its current configuration and recent runs:

export TASK_ARN='arn:aws:datasync:region:account-id:task/task-id'

aws datasync describe-task --task-arn "$TASK_ARN" \
  --query '{status:Status,mode:TaskMode,source:SourceLocationArn,destination:DestinationLocationArn,options:Options}'

aws datasync list-task-executions --task-arn "$TASK_ARN" --max-results 5 \
  --query 'TaskExecutions[].{status:Status,executionArn:TaskExecutionArn}'

Choose the failed or unexpected execution ARN from that list, then inspect its phase results, settings, and counters:

export EXECUTION_ARN='arn:aws:datasync:region:account-id:task/task-id/execution/exec-id'

aws datasync describe-task-execution --task-execution-arn "$EXECUTION_ARN" \
  --query '{status:Status,mode:TaskMode,options:Options,overrides:OverrideOptions,result:Result,filesSkipped:FilesSkipped,filesTransferred:FilesTransferred,filesVerified:FilesVerified,filesFailed:FilesFailed}'

These describe and list calls are read-only. Use the Region and AWS CLI profile that contain the task. Compare the execution’s Result fields and error detail with its CloudWatch logs or task report; a summary status alone may hide skipped files or a failure in one phase. See AWS guidance for task execution statuses, CloudWatch logging, and task reports.

  • Queued: another execution may be using the same agent, or an earlier execution of the same task may be ahead in the queue. A task can queue work that uses different filters or manifests; an agent runs one task at a time.
  • Launching: check that the agent is powered on and online, then trace its connection to the DataSync service.
  • Preparing: DataSync lists source and destination data to decide what to copy. Large namespaces can take minutes or hours; wait while the listing progresses, and use logs or support if it remains stalled.
  • Transferring or verifying: use the phase result, task logs, counters, and storage-side metrics to locate the failing operation.

2. Check the agent and its service connection

For transfers that use an agent, confirm the registered agent is online and identify its endpoint type:

export AGENT_ARN='arn:aws:datasync:region:account-id:agent/agent-id'
aws datasync describe-agent --agent-arn "$AGENT_ARN" \
  --query '{status:Status,lastConnectionTime:LastConnectionTime,endpointType:EndpointType}'

If it is offline, first check that the VM or EC2 instance is running and has working DNS, time synchronization, and the expected route to its DataSync service endpoint. Check the required destinations and ports for the endpoint type you chose. AWS manages DataSync agents and updates their software; do not reinstall the appliance as a routine recovery step. If power and network checks do not explain a persistent offline state, collect the agent ARN and diagnostics for AWS Support. A brief offline period can occur during an agent update.

During activation, a browser or client may need temporary access to the agent on TCP port 80 to retrieve an activation key. DataSync closes that port after activation, and the key expires after 30 minutes if unused. A port 80 timeout during later transfers is not evidence that the agent is unhealthy. See AWS instructions for agent activation and agent management.

Trace the actual network path

DataSync can use different paths for agent-to-service control traffic and for reading or writing storage. Identify the locations and endpoint type before changing a firewall rule:

  • Public or FIPS service endpoint: the agent needs outbound TCP 443 to the activation, control-plane, and data-plane endpoints listed for its Region. Basic mode agents also need the documented bootstrap access before activation.
  • VPC or FIPS VPC service endpoint: the agent needs TCP 1024–1064 to the DataSync VPC endpoint for control traffic and TCP 443 to the task network interfaces for data traffic. Port 22 is an optional AWS Support channel, not a normal transfer requirement.
  • Non-AWS file or object storage: the agent must separately reach that source or destination using the storage protocol and port configured for the location.
  • AWS storage such as S3, EFS, or FSx: DataSync may reach the storage through task network interfaces in your VPC. Check their subnet routes and security groups, plus the storage service’s own access rules.

Use AWS’s DataSync network requirements for the exact direction, destination, and port. Allow only the documented flow between the relevant sources and destinations; broad inbound rules do not solve a missing route or a blocked outbound connection.

Resolve EniNotFound without changing agent permissions

EniNotFound means a network interface allocated to the DataSync task was deleted. It does not mean that the agent needs permission to create an ENI. DataSync manages these task interfaces. Inspect the task’s source and destination network-interface ARNs with this read-only command, then confirm the subnet and security-group path:

aws datasync describe-task --task-arn "$TASK_ARN" \
  --query '{sourceEnis:SourceNetworkInterfaceArns,destinationEnis:DestinationNetworkInterfaceArns}'

After restoring any required network access, manually restart the task; DataSync recreates missing interfaces when it starts. Avoid deleting interfaces associated with active tasks.

3. Verify source and destination access

Test the path from the network where the DataSync agent runs, using the same server name, export or share path, and identity configured in the location. A TCP probe can show whether a port accepts connections, but it does not prove that the location credentials can read or write the required files.

NFS and Amazon EFS

For an NFS server reached by an agent, DataSync uses TCP 2049 for NFS 4.0/4.1, or TCP 111 and 2049 for NFS 3.x. The default setting is AUTOMATIC, which selects NFS 4.1. Check the exported path, network reachability, and POSIX permissions on every directory in the path: DataSync needs read access to a source, write access to a destination, and execute permission to traverse directories. Keep export access scoped to the agent and required operation. A source can use an appropriate read-only identity. A destination may need root access to write ownership and permissions when those metadata are being preserved; grant it only when the configured transfer requires it.

DataSync does not support NFSv4 ID mapping. If logs show FsNfsIdMappingEnabled or ownership appears as an unexpected UID/GID, check ID mapping and use an NFSv3 location when that feature is enabled. See AWS guidance for NFS locations and location errors.

For an Amazon EFS location, DataSync’s AWS-side network interfaces use the subnet and security-group configuration on that location. Check that the selected EFS path exists, the file system has reachable mount targets, and the security groups permit NFS on TCP 2049 between the DataSync interfaces and EFS. This AWS-side path does not depend on an on-premises agent reaching the EFS mount target.

SMB file servers

Check that the agent can reach the file server on TCP 445 (or TCP 139 if your server uses it), that the share and subdirectory are correct, and that the configured identity can mount the share and access the needed files and metadata. Automatic SMB negotiation is the default; choose a specific protocol version only when negotiation fails, and prefer SMB 3.0.2 or later. For Kerberos, use the server’s domain name, verify the principal and keytab match, and allow the required path to the KDC (typically TCP 88). DataSync does not support DFS Namespaces; configure the underlying file server and share. See AWS guidance for SMB locations and permissions.

S3 and other AWS locations

For an S3 access error, match the failing API operation in the logs to the DataSync location role, bucket policy, and (when used) KMS key policy. For EFS or FSx, check the file system’s location settings, subnet, security groups, and protocol permissions. Follow the network path for the actual source/destination pair rather than assuming every transfer passes through the agent.

4. Check task mode and transfer options

TaskMode and TransferMode control different things. A task uses BASIC or ENHANCED mode; its data-transfer option is CHANGED or ALL. Snowball and online/offline are not TransferMode values.

  • CHANGED (the default) transfers data and metadata that differ between locations. It is the usual choice for recurring synchronization.
  • ALL copies all source data without comparing source and destination first.
  • PreserveDeletedFiles=PRESERVE (the default) keeps destination-only files. REMOVE deletes destination files that are absent at the source. REMOVE cannot be used with TransferMode=ALL, because ALL does not scan the destination to find deletions.
  • OverwriteMode=ALWAYS (the default) lets changed source data overwrite destination data; NEVER protects destination data from overwrites. This is a separate decision from end-of-transfer verification.

Before a retry, inspect the task’s current options and any execution overrides. If deletions are enabled, confirm the destination is safe to reconcile before starting the task. For the complete option definitions and defaults, see AWS’s file and object handling options.

Basic and Enhanced have different supported location pairs, agent types, limits, metrics, and verification behavior. Basic supports all supported DataSync location pairs, subject to its item quotas. Enhanced is available only for the following supported pairs: transfers among S3, EFS, and FSx for Lustre without an agent; between NFS, SMB, or HDFS and supported AWS storage using an Enhanced agent; and between Azure Blob or other cloud object storage and supported AWS storage (no agent is needed when one endpoint is S3). The AWS identity creating an Enhanced task needs iam:CreateServiceLinkedRole; this is a caller permission, not a permission assigned to the agent. See AWS documentation for DataSync service-linked roles. Check the current task mode support matrix before creating locations or agents. DataSync does not let you change a task’s mode after creation.

5. Recover, retry, and verify

Treat verification as evidence

DataSync checks data integrity during transfer. By default, Basic mode also performs a full point-in-time scan at the end; Enhanced mode verifies transferred data. The additional end-of-transfer check depends on VerifyMode and task mode:

  • ONLY_FILES_TRANSFERRED checks the transferred data and metadata. This is the default for Enhanced mode.
  • POINT_IN_TIME_CONSISTENT scans the selected source and destination data at the end to check that they are synchronized. This is the default for Basic mode. It is unavailable for Enhanced mode and for transfers to S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive, and can take much longer on large datasets.
  • NONE skips the additional end-of-transfer scan; DataSync still performs integrity checks during transfer. It is not a repair for permission, network, or data mismatch errors.

These options are not interchangeable: use one that matches the required assurance and supported task mode. If verification reports a mismatch, inspect the listed paths and source/destination errors in CloudWatch logs or a task report. A file changed or deleted while DataSync was transferring or verifying can cause a mismatch; rerun after writes settle, then confirm the verified count and application-level data expected at the destination. See AWS guidance for verification modes and verification failures.

Handle memory and large-dataset errors

For Cannot allocate memory, check that an agent meets the current VM or EC2 requirements for its mode and dataset. Basic mode has item quotas, and directory count and metadata size affect memory use. If appropriate, split the transfer with include/exclude filters while preserving the task’s intended deletion and synchronization scope. Enhanced mode supports far larger datasets for supported location pairs, but it requires an Enhanced task and, when applicable, an Enhanced agent. If a NoMem service error persists, save the task execution ARN and logs and contact AWS Support.

Verify a corrected run

After changing the specific root cause, start a new execution and confirm it reaches SUCCESS. Review the available phase results and reconcile transferred, skipped, failed, deleted, and verified counts with the expected scope. Enhanced mode can prepare, transfer, and verify concurrently, and exposes different counters from Basic mode, so interpret the fields for the selected mode instead of expecting every Basic phase field. Check the destination data and required metadata, especially after changing overwrite, deletion, filters, or permissions. If errors remain, stop repeated retries and use the new execution’s logs and report to isolate the next failing path.

FAQs

Does DataSync move or copy files?

DataSync copies from source to destination; it does not remove the source data. PreserveDeletedFiles=REMOVE can remove destination files that are not present in the source when used with a compatible transfer mode.

Why is a task taking a long time in Preparing?

DataSync inventories files, objects, directories, and metadata before deciding what to transfer. A large or slow source or destination can make this phase take hours. Check progress and storage performance before treating the phase as a failure.

Can I turn off verification to fix a task?

No verification setting fixes a transfer error. VerifyMode=NONE skips the additional end-of-transfer scan but keeps in-transfer integrity checks. Find and correct the error reported by the execution and its logs, then verify the result to the level your migration requires.