c4compile
Compile outputs fun
Subscribe/Contact me

AWS CloudFormation: 04 - Build and deploy pipeline

Published 2 years agoAWS, CloudFormation

This is continued from AWS CloudFormation: 03 - Collect logs .

Background

I want to build my software and deploy it to the web server.

What we need?

  1. Install CodeDeploy agent on the EC2 instances to deploy new software.
  2. A CodeBuild setup to build the software.
  3. A CodeDeploy setup to deploy the software.
  4. A CodePipeline setup to build and deploy the software.

How to do it?

This is what we are going to do:

Open 03-cloudwatch-template.yaml from previous post and save as 04-codepipeline-template.yaml .

Launch Template

We will need to setup the CodeDeploy agent on the EC2 instances. Modify these resources:

1   WebServerServiceRole:
    Type: AWS::IAM::Role
    Properties:
...
      Policies:
        - PolicyName: WebServerServicePolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Resource: !Sub ${CodePipelineBucket.Arn}/*
                Action:
                  - s3:Get*
                  - s3:List*
... 		The IAM Policy for the EC2 instance to access the build artifacts on the S3 bucket.
2   WebServerLaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
...
    Metadata:
      AWS::CloudFormation::Init:
...
        Install:
          commands:
            install:
              command: !Sub |
...
                cd /home/ec2-user
                wget https://aws-codedeploy-${AWS::Region}.s3.${AWS::Region}.amazonaws.com/latest/install
                chmod +x ./install
                sudo ./install auto 		Add the steps to install CodeDeploy agent.
        Configure:
          files:
            /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json:
              content: !Sub |
                {
                  "logs": {
                    "logs_collected": {
                      "files": {
                        "collect_list": [
                          { 
                            "file_path": "/var/log/nginx/error.log", 
                            "log_group_name": "${CloudWatchLogGroup}",
                            "log_stream_name": "nginx"
                          }, { 
                            "file_path": "/var/log/aws/codedeploy-agent/codedeploy-agent.log", 
                            "log_group_name": "${CloudWatchLogGroup}",
                            "log_stream_name": "codedeploy-agent"
                          }
                        ]
                      }
                    }
                  }
                } 		Add the configuration to allow CodeWatch agent to collect log file of CodeDeploy agent.

Build and Deploy Pipeline

We will need to setup a pipeline to build and deploy our software. Add these resources:

1   CodePipelineBucket:
    Type: AWS::S3::Bucket 		The S3 bucket to store the source artifacts and build artifacts.
2   CodeBuildServiceRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: codebuild.amazonaws.com
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: CodeBuildServicePolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Resource: !Sub ${CodePipelineBucket.Arn}/*
                Action: s3:* 		The IAM Role to allow the CodeBuild project to access the S3 bucket.
3   CodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      ServiceRole: !Ref CodeBuildServiceRole
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        Type: LINUX_CONTAINER
        Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
        ComputeType: BUILD_GENERAL1_SMALL 		The CodeBuild project to build the software.
      Source:
        Type: CODEPIPELINE
        BuildSpec: |
          version: 0.2
          phases:
            build:
              commands:
                - cd write-it-down-web
                - npm install
                - npm run build 		The steps to build the software.
                - mkdir scripts
                - |
                    cat > scripts/before_install.sh <<EOF
                    #!/bin/bash
                    rm -rf /usr/share/nginx/html/*
                    EOF 		Add a script to run before installing the software.
                - |
                    cat > appspec.yml <<EOF
                    version: 0.0
                    os: linux
                    files:
                      - source: build
                        destination: /usr/share/nginx/html
                    hooks:
                      BeforeInstall:
                        - location: scripts/before_install.sh
                    EOF 		Add a spec file so CloudDeploy agent knows how to deploy the software.
          artifacts:
            files:
              - appspec.yml
              - build/**/*
              - scripts/**/*
            base-directory: write-it-down-web 		Tell the CodeBuild project how to pack the build artifact.
      LogsConfig:
        CloudWatchLogs:
          Status: DISABLED
        S3Logs:
          Status: ENABLED
          Location: !Sub ${CodePipelineBucket.Arn}/BuildLogs 		Store the build log in S3 bucket.
4   CodeDeployServiceRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: codedeploy.amazonaws.com
            Action:
              - sts:AssumeRole 		The IAM Role to allow the CodeDeploy service to access the required resources.
      Policies:
        - PolicyName: CodeDeployServicePolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Resource:
                  - arn:aws:autoscaling:*:*:autoScalingGroup:*:autoScalingGroupName/CodeDeployTestAPICall
                  - !Sub arn:aws:autoscaling:*:*:autoScalingGroup:*:autoScalingGroupName/${WebServerAutoScalingGroup}
                Action: autoscaling:*
                Effect: Allow 		It needs to access to the auto scaling group so it can manage the EC2 instances on behalf of it.
              - Resource: "*"
                Condition:
                  StringEquals:
                    ec2:ResourceTag/aws:autoscaling:groupName: !Ref WebServerAutoScalingGroup
                Action: ec2:*
                Effect: Allow 		It needs to access to the EC2 instances created by the auto scaling group so it can manipulate them.
              - Resource: !Ref WebServerLoadBalancerTargetGroup
                Action: elasticloadbalancing:*
                Effect: Allow 		It needs to access to the load balancer so it can control the traffic when deploying.
              - Resource: !Sub ${CodePipelineBucket.Arn}/*
                Action: s3:*
                Effect: Allow 		It needs to access to the S3 bucket so it can download the build artifacts.
              - Resource: "*"
                Action:
                  - autoscaling:Describe*
                  - ec2:Describe*
                  - elasticloadbalancing:Describe*
                Effect: Allow 		And some other non-resource specific policies.
5   CodeDeployApplication:
    Type: AWS::CodeDeploy::Application
    Properties:
      ComputePlatform: Server 		A CodeDeploy application.
6   CodeDeployDeploymentGroup:
    Type: AWS::CodeDeploy::DeploymentGroup
    Properties:
      ApplicationName: !Ref CodeDeployApplication
      ServiceRoleArn: !Sub ${CodeDeployServiceRole.Arn}
      AutoScalingGroups:
        - !Ref WebServerAutoScalingGroup
      LoadBalancerInfo:
        TargetGroupInfoList:
          - Name: !Sub ${WebServerLoadBalancerTargetGroup.TargetGroupName}
      DeploymentConfigName: CodeDeployDefault.OneAtATime
      DeploymentStyle:
        DeploymentOption: WITH_TRAFFIC_CONTROL
        DeploymentType: IN_PLACE 		A CodeDeploy deployment group that define how to deploy the software.
7   CodePipelineServiceRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: codepipeline.amazonaws.com
            Action:
              - sts:AssumeRole 		The IAM Role to allow the CodePipeline service to access the required resources.
      Policies:
        - PolicyName: CodePipelineServicePolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Resource: !Sub ${SourceCodeConnectionArn}
                Action: codestar-connections:UseConnection
                Effect: Allow 		It needs to use the source code connection string to pull the source code from the repository.
              - Resource: !Sub ${CodePipelineBucket.Arn}/*
                Action: s3:*
                Effect: Allow 		It needs to access to the S3 bucket.
              - Resource: !Sub ${CodeBuildProject.Arn}
                Action: codebuild:*
                Effect: Allow 		It needs to access to the CodeBuild project.
              - Resource:
                  - !Sub arn:aws:codedeploy:*:*:application:${CodeDeployApplication}
                  - !Sub arn:aws:codedeploy:*:*:deploymentgroup:${CodeDeployApplication}/${CodeDeployDeploymentGroup}
                  - arn:aws:codedeploy:*:*:deploymentconfig:CodeDeployDefault.*
                Action: codedeploy:*
                Effect: Allow 		It needs to access to the CodeDeploy service.
8   CodePipeline:
    Type: AWS::CodePipeline::Pipeline
      RoleArn: !Sub ${CodePipelineServiceRole.Arn}
      ArtifactStore:
        Type: S3
        Location: !Ref CodePipelineBucket
      RestartExecutionOnUpdate: true 		This is the pipeline to buid and deploy the software. It will use the S3 bucket created earlier to store all the artifacts.
    Properties:
      Stages:
        - Name: Source
          Actions:
            - Name: Default
              ActionTypeId:
                Category: Source
                Owner: AWS
                Provider: CodeStarSourceConnection
                Version: "1"
              OutputArtifacts:
                - Name: SourceArtifact
              Configuration:
                ConnectionArn: !Sub ${SourceCodeConnectionArn}
                FullRepositoryId: !Sub ${SourceCodeRepository}
                BranchName: master
                OutputArtifactFormat: CODE_ZIP 		Define where is the source code.
        - Name: Build
          Actions:
            - Name: Default
              ActionTypeId:
                Category: Build
                Owner: AWS
                Provider: CodeBuild
                Version: "1"
              InputArtifacts:
                - Name: SourceArtifact
              OutputArtifacts:
                - Name: BuildArtifact
              Configuration:
                ProjectName: !Ref CodeBuildProject 		Define the build stage.
        - Name: Deploy
          Actions:
            - Name: Default
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Provider: CodeDeploy
                Version: "1"
              InputArtifacts:
                - Name: BuildArtifact
              Configuration:
                ApplicationName: !Ref CodeDeployApplication
                DeploymentGroupName: !Ref CodeDeployDeploymentGroup 		Define the deploy stage.

Parameters

We need to add a few parameters to know where is the source code repository. Add these parameters:

1   SourceCodeConnectionArn:
    Type: String
    Default: <your source code connection> 		The source code connection string. You can create one with AWS developer tools:
2   SourceCodeRepository:
    Type: String
    Default: chimin/write-it-down 		The source code repository name.

Update the stack then you should be able to see the CodePipeline. Execute the pipeline and you should get the software built and deployed to the EC2 instances.

Next we will do AWS CloudFormation: 05 - Create a web server Docker instance .