Label Image Datasets with Amazon SageMaker Ground Truth
Published on · Updated on
Updated Ground Truth availability and workforce status, documented image task and manifest choices, and corrected security, output, automation, and label-quality guidance.
Amazon SageMaker Ground Truth helps teams create image labels for machine learning by coordinating human review and, for some built-in tasks, automated data labeling. The workflow depends on the label format your model needs: an image-level class, boxes around objects, or a class for each pixel.
Availability: AWS closed Ground Truth access to new customers on July 30, 2026. Existing customers can continue using the service, but AWS says it does not plan to add new features. AWS scheduled Amazon Mechanical Turk's permanent closure for September 30, 2026; because that date has passed, confirm your account's current workforce options and plan around an eligible alternative. This guide is for existing Ground Truth customers. See AWS's current Ground Truth documentation and workforce guidance before planning a job.
What is Amazon SageMaker Ground Truth?
Ground Truth provides worker interfaces for labeling data and stores job results in Amazon S3. You provide the images, task instructions, label choices, and workforce; the resulting annotations can be reviewed and prepared for a model-training workflow. Human labels still need quality checks that reflect the image domain and the way the model will use them.
Check access, workforce, and data permissions
For an existing customer account, the basic requirements are:
- Access to Ground Truth in the SageMaker AI console or through the SageMaker API.
- Images and an input manifest stored in Amazon S3 in the AWS Region used for the labeling job.
- An IAM execution role that can read the input data and write job output to S3. The role may also need CloudWatch permissions and AWS KMS permissions for the encryption settings you choose.
- A workforce approved for the data and task. Ground Truth documents private and vendor-managed workforces; review the current availability, terms, and access controls for your account.
A private workforce consists of workers you select, such as employees or subject-matter experts. An AWS account can have one private workforce per Region, with multiple work teams. A vendor-managed workforce is provided through an AWS Marketplace subscription; review its worker availability, schedule, price, and contract terms. Choose based on the expertise required, data handling terms, access controls, and schedule. A workforce choice alone does not guarantee confidentiality: workers must be able to view the images to label them. See AWS's current guidance for private workforces and vendor workforces.
Ground Truth sends input data to workers. AWS says it encrypts job data in transit and at rest; output uses the account's default AWS-managed S3 key unless you select a customer-managed KMS key. Encryption protects transfer and storage, but it does not replace least-privilege IAM permissions or review of workforce access and vendor terms. For an explanation of S3 permissions, see our guide to Amazon S3 access management. Read AWS's details on Ground Truth encryption and execution-role permissions.
Choose the image task and prepare the input
Select the task by the annotation your model consumes. Ground Truth's built-in image tasks include:
| Task | What workers label | Use it when |
|---|---|---|
| Image classification | One class or several labels for an entire image. | The target describes the image as a whole, such as the product category or scene. |
| Object detection | A class and bounding box for each visible object. | The model must locate and classify objects. |
| Semantic segmentation | A class mask that assigns pixels to categories. | The model needs pixel-level regions, such as road, sky, or vegetation. |
| Label verification and adjustment | A review of existing labels, with corrections where needed. | You need workers to check or adjust results from an earlier labeling job. |
The built-in image task list does not include a separate instance-segmentation task. If your model needs a distinct mask for each object, confirm that your planned annotation workflow can produce that output before collecting labels. See AWS's built-in task types.
Write label definitions before creating the job. Explain how to handle borderline examples, partially visible objects, overlap, empty images, and cases that should be left unlabeled. Include representative examples in the worker instructions, and keep label names and downstream schema consistent.
For image files, the input manifest is a UTF-8 JSON Lines file: each line is a complete JSON object. Use source-ref with the image's S3 URI, for example:
{"source-ref":"s3://example-image-data/images/photo-001.jpg"}
{"source-ref":"s3://example-image-data/images/photo-002.jpg"}Keep the manifest and images in S3 and in the same Region as the labeling job. Each line can also carry other fields that your application needs; Ground Truth passes those fields through to the output. See AWS's guide to input manifest files.
Create and monitor a labeling job
If your account already has Ground Truth access, create a built-in labeling job in the SageMaker AI console or with the API. Provide the input manifest, an S3 output location, the task type, worker instructions, and the approved workforce. Set the number of workers per image and task time limits to fit the work and review plan; more worker responses can support annotation consolidation but add work and cost.
Check that the job completes and that output appears in the expected S3 location. Ground Truth writes an output manifest as the job proceeds, so do not treat an in-progress file as a complete dataset. For a failed job, check its status and error details, then confirm that the execution role can read every input object and write to the output location.
For image tasks, automated data labeling is available for single-label image classification, object detection with bounding boxes, and semantic segmentation. (AWS also supports single-label text classification.) It is not supported for multi-label image classification, custom task types, or streaming jobs. AWS currently sets a minimum of 1,250 objects and suggests at least 5,000; check the live quotas before sizing a job. AWS describes automated labeling as active learning: the service sends examples to people, trains and evaluates a model, auto-labels items that meet its task-specific confidence requirements, and sends other items to workers. This can reduce manual work on eligible datasets, but it does not guarantee a lower total cost; training and inference incur charges, and some items may still require human labels. Review current quotas and costs before enabling it. See AWS's guidance on automated data labeling.
Review labels for quality
Start with a pilot that includes typical images and difficult cases. Review the annotations with someone who understands the domain, note disagreements and recurring mistakes, and refine instructions before labeling the full dataset. After the job, check for missing labels, unexpected classes, inconsistent boundaries, and images that workers could not interpret.
- Use a representative review sample and compare it with labels independently checked by a qualified reviewer. Choose acceptance criteria that match the impact of mistakes in your application.
- If you assign multiple workers to each image, inspect consolidated labels and disagreements. For like-for-like labels from the same job, Ground Truth confidence values can help prioritize review; AWS warns they are not absolute correctness scores and should not be compared across jobs or between human and automated labels.
- Use a verification or adjustment job when reviewing an existing supported label set. Keep corrections traceable to the source image and job.
- For automated labels, test results on a representative sample against human-reviewed labels before using them for training or inference. AWS recommends testing or fine-tuning the automated-labeling model before production use.
No task setting or confidence score guarantees that labels are correct for your model. Retain the reviewed sample, label definitions, and job output so you can investigate errors and reproduce how the dataset was assembled.
Use the output in a training workflow
Ground Truth writes the job's output manifest to the S3 location you selected. The output.manifest file is JSON Lines: each record carries the source image and its assigned label attribute, with metadata such as the task type and whether a human or automated labeling produced the label. Other input fields are preserved. The annotations directory can also contain individual worker responses and intermediate files.
For the built-in image classification, object detection, and semantic segmentation tasks, the augmented manifest can be used as input to a SageMaker training job. Inspect the task-specific label schema and confirm that the training code expects it; do not assume every model consumes the same format. See AWS's reference for labeling job output data and using input and output data.
Next steps
For existing Ground Truth customers, a reliable image-labeling workflow begins with an eligible workforce, a task that matches the model's expected annotations, a valid S3 manifest, and instructions tested on a pilot. Review output against a representative human-checked sample, then pass the task-specific manifest to the training pipeline. Check AWS's current documentation for account availability, workforce options, quotas, and job behavior before starting work.