AWS SAM Pipelines: Setup Guide

AWS SAM Pipelines make deploying serverless applications easier by automating CI/CD workflows. Here's everything you need to know to get started:

Key Benefits:

• Fewer Errors: Automates deployments, reducing manual mistakes.
• Faster Deployments: Saves time, so you can focus on coding.
• Simplified Workflows: Lowers operational overhead.

Quick Setup Overview:

1. AWS Account Prep: Create an IAM user, assign permissions, and set up access keys securely.
2. Install AWS SAM CLI: Ensure Python 3.7+, Docker, and Git are installed. Use choco (Windows) or brew (macOS) to install the CLI.
3. Git Repo Setup: Organize your project with a clear structure and push to version control.
4. Configure AWS Resources: Set up IAM roles, an S3 bucket for artifacts, and define pipeline YAML templates.
5. Pipeline Initialization: Use sam pipeline init --bootstrap to create pipeline files and configure stages like dev, staging, and production.

Tips for Success:

• Use AWS Systems Manager or Secrets Manager to manage sensitive data.
• Monitor your pipeline with AWS CloudWatch for performance and alerts.
• Troubleshoot issues by checking IAM permissions, build logs, and CloudFormation stack events.

This guide gives you the steps to configure, deploy, and monitor serverless applications efficiently.

Setup Requirements

Prepare the following steps to ensure your SAM Pipeline runs smoothly.

AWS Account Setup

Here's what you'll need to do:

• Create an IAM user: Set up a dedicated IAM user with programmatic access specifically for pipeline operations.
• Assign Permissions: Attach policies that allow access to:
  • AWS CloudFormation
  • AWS Lambda
  • Amazon S3
  • AWS CodeBuild
  • AWS CodePipeline
• Generate AWS access keys: Save these keys securely for use in your pipeline.

To keep your account secure, stick to the principle of least privilege. Only grant permissions that are absolutely necessary for your pipeline's tasks.

AWS SAM CLI Installation

The AWS SAM CLI is essential for working with SAM Pipelines. Install it on your development machine by following these steps:

1. System Requirements

Ensure your system meets these requirements:

• Python 3.7 or newer
• Docker (for local testing)
• Git 2.0 or newer

2. Installation Steps

For Windows users:

choco install aws-sam-cli

For macOS users:

brew install aws-sam-cli

3. Verify Installation

Run this command to confirm the installation:

sam --version

Once installed, move on to setting up your Git repository for version control and pipeline integration.

Git Repository Setup

1. Repository Structure

Organize your project repository like this:

my-sam-app/
├── template.yaml
├── src/
│   └── functions/
├── tests/
└── pipeline/
    └── pipeline.yaml

2. Version Control Best Practices

Adopt these practices for effective version control:

• Use feature branches for development work.
• Keep the main branch stable and production-ready.
• Add a detailed .gitignore file to exclude unnecessary files.
• Set up branch protection rules to prevent accidental changes.

3. Initial Configuration

Initialize your repository and push the initial setup:

git init
git remote add origin <repository-url>
git add .
git commit -m "Initial SAM project setup"
git push -u origin main

With these steps completed, your environment will be ready for SAM Pipeline operations.

AWS Resource Configuration

Set up AWS resources to enable the SAM Pipeline.

Pipeline IAM Roles

Define IAM roles to manage pipeline operations effectively.

1. Pipeline Execution Role

Create a role with permissions for pipeline tasks:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:*",
                "s3:*",
                "lambda:*",
                "codebuild:*",
                "codepipeline:*"
            ],
            "Resource": "*"
        }
    ]
}

2. CodeBuild Service Role

Set up a service role for CodeBuild:

aws iam create-role --role-name sam-pipeline-build-role \
    --assume-role-policy-document file://trust-policy.json

Attach the required policy:

aws iam attach-role-policy --role-name sam-pipeline-build-role \
    --policy-arn arn:aws:iam::aws:policy/AWSCodeBuildAdminAccess

Artifact Storage Setup

Create an S3 bucket to store pipeline artifacts:

aws s3 mb s3://your-pipeline-artifacts-bucket \
    --region us-east-1

Enable versioning for the bucket:

aws s3api put-bucket-versioning \
    --bucket your-pipeline-artifacts-bucket \
    --versioning-configuration Status=Enabled

Add encryption to the bucket for security:

aws s3api put-bucket-encryption \
    --bucket your-pipeline-artifacts-bucket \
    --server-side-encryption-configuration '{
        "Rules": [
            {
                "ApplyServerSideEncryptionByDefault": {
                    "SSEAlgorithm": "AES256"
                }
            }
        ]
    }'

CI/CD Service Integration

After setting up IAM roles and artifact storage, integrate CI/CD services.

1. Create Pipeline Configuration

Define the pipeline in a YAML template:

Resources:
  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      Name: sam-app-pipeline
      RoleArn: !GetAtt PipelineRole.Arn
      ArtifactStore:
        Type: S3
        Location: !Ref ArtifactBucketName
      Stages:
        - Name: Source
          Actions:
            - Name: Source
              ActionTypeId:
                Category: Source
                Owner: AWS
                Provider: CodeCommit
                Version: '1'
              Configuration:
                RepositoryName: !Ref RepositoryName
                BranchName: main
              OutputArtifacts:
                - Name: SourceCode

2. Set Build Specifications

Add a buildspec.yml file to your repository:

version: 0.2
phases:
  install:
    runtime-versions:
      python: 3.9
  build:
    commands:
      - sam build
      - sam deploy --no-confirm-changeset --no-fail-on-empty-changeset
artifacts:
  files:
    - template.yaml
    - packaged.yaml
  discard-paths: yes

3. Configure Pipeline Triggers

Set up automatic triggers for the pipeline:

aws codepipeline put-webhook \
    --pipeline-name sam-app-pipeline \
    --webhook-name pipeline-webhook \
    --authentication-configuration Type=GITHUB_HMAC,SecretToken=your-secret \
    --target-pipeline sam-app-pipeline \
    --target-action Source \
    --filters Name=event,Value=push

Pipeline Setup and Configuration

Pipeline Initialization

To set up your pipeline, run the following command: sam pipeline init --bootstrap. This interactive setup will guide you through creating two key files: pipeline/pipeline.yaml and pipeline/config.yaml. You'll also be prompted to select your CI/CD system, set configurations, and define environment names and regions.

Here's what these files do:

• pipeline/pipeline.yaml: Defines the main pipeline infrastructure.
• pipeline/config.yaml: Manages environment-specific settings.

Pipeline Stage Configuration

Set up your pipeline stages to align with your deployment process. Below is an example configuration:

Stages:
  - Name: Development
    DeploymentPreferences:
      Type: Linear10PercentEvery1Minute
      Alarms:
        - !Ref ErrorAlarm
    Variables:
      Environment: dev
      StackName: sam-app-dev

  - Name: Staging
    DeploymentPreferences:
      Type: Linear10PercentEvery2Minutes
      Alarms:
        - !Ref ErrorAlarm
        - !Ref LatencyAlarm
    Variables:
      Environment: staging
      StackName: sam-app-staging

  - Name: Production
    DeploymentPreferences:
      Type: Canary10Percent5Minutes
      Alarms:
        - !Ref ErrorAlarm
        - !Ref LatencyAlarm
        - !Ref ThrottlesAlarm
    Variables:
      Environment: prod
      StackName: sam-app-prod

Template Customization

Once your stages are defined, refine your pipeline by adding build commands, environment variables, and approval steps.

1. Add Custom Build Commands

Modify your buildspec.yml to include custom build steps:

version: 0.2
phases:
  install:
    runtime-versions:
      nodejs: 14
    commands:
      - npm install -g typescript
      - npm ci
  pre_build:
    commands:
      - npm run lint
      - npm run test
  build:
    commands:
      - npm run build
      - sam build
      - sam deploy --no-confirm-changeset
  post_build:
    commands:
      - npm run integration-test

2. Configure Environment Variables

Define environment-specific parameters in your pipeline configuration:

Parameters:
  EnvironmentName:
    Type: String
    Default: dev
    AllowedValues:
      - dev
      - staging
      - prod

  ApiDomainName:
    Type: String
    Default: api.example.com

  LogRetentionDays:
    Type: Number
    Default: 14

3. Add Approval Gates

Include manual approval steps for critical stages like production:

Stages:
  - Name: ApproveProductionDeployment
    Actions:
      - Name: ManualApproval
        ActionTypeId:
          Category: Approval
          Owner: AWS
          Provider: Manual
          Version: '1'
        Configuration:
          CustomData: "Please review and approve production deployment"
        RunOrder: 1

This setup ensures proper oversight and allows for tailored configurations across different environments.

sbb-itb-6210c22

Pipeline Operation

Version Control Integration

Make sure all necessary SAM pipeline configuration files are stored in your repository. To keep sensitive information like API keys or credentials secure, use AWS Systems Manager Parameter Store or Secrets Manager instead of including them in your source code. Once everything is set, you can start the pipeline run.

Running the Pipeline

After integrating version control and committing your configuration changes, the pipeline will automatically deploy as part of your CI/CD workflow. If needed, you can also trigger deployments manually through the AWS CodePipeline console.

Pipeline Monitoring

Use AWS tools to keep an eye on your pipeline's performance. The AWS console dashboard provides status updates and logs, while CloudWatch monitors metrics and sends alerts to help with troubleshooting.

Troubleshooting Guide

AWS SAM Pipelines can encounter issues during setup or execution. Use these steps to address common problems effectively.

IAM Permission Errors

Insufficient IAM permissions can disrupt pipeline operations. Here's how to handle these errors:

• Review CloudTrail Logs: Check CloudTrail logs to identify any failed API calls.
• Update Role Policies: Ensure your pipeline execution role has the necessary permissions. For example:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateStack",
                "s3:PutObject",
                "lambda:UpdateFunctionCode"
            ],
            "Resource": "*"
        }
    ]
}

• Check Resource-Level Access: Confirm that your roles have the required access to resources like S3 buckets and Lambda functions.

Build Error Solutions

Build failures often stem from configuration or dependency issues. Here's how to troubleshoot:

• Verify Build Specifications: Make sure your buildspec.yml file includes the correct runtime and build commands.
• Check Dependencies: Ensure all required packages are included in your dependency files (e.g., package.json for Node.js or requirements.txt for Python).
• Review Build Logs: Look at CodeBuild logs for errors. Common issues to watch for include:
  • Missing environment variables
  • Mismatched Python or Node.js versions
  • Insufficient memory during the build process

Deployment Error Fixes

Deployment failures can be resolved using these steps:

• Validate Templates: Use the SAM CLI to validate your template locally:

sam validate --template template.yaml

• Check Stack Events: Look at CloudFormation stack events for detailed error messages about failed resource creation.
• Monitor Resource Limits: Ensure you aren't exceeding AWS service quotas in your target region. Common limits include:
  • Lambda concurrent executions
  • CloudFormation stack resource counts
  • IAM role trust relationships

Summary

Here's a quick recap of the key points covered. AWS SAM Pipelines simplifies deploying serverless applications by automating CI/CD workflows. To make it work, focus on three main areas: configuring AWS resources, setting up the pipeline, and keeping an eye on its performance.

During pipeline setup, you'll define deployment stages and link critical AWS resources like IAM roles and S3 buckets to ensure secure and smooth operations.

For the pipeline to run smoothly, you'll need:

• A well-configured AWS account
• The AWS SAM CLI installed and ready
• Version control integration
• Properly assigned IAM permissions

The automated process not only saves time but also improves development by:

• Reducing deployment errors
• Keeping workflows consistent
• Allowing for faster updates
• Offering tools to monitor performance

Use CloudWatch to track how your pipeline is performing and make adjustments as needed. These steps can help you manage your deployments more efficiently.

Related Blog Posts

• 10 Best Practices for End-to-End Testing AWS Apps
• Test Serverless Apps Locally with AWS SAM CLI
• AWS SAM Local: Setup Guide
• Monitor Lambda Errors with CloudWatch