GitLab, CodeCommit, CodeBuild, CodePipeline, CodeDeploy

Terraform

terraform
resource "aws_codepipeline" "pipeline" {
	name = "my-pipeline"
	role_arn = aws_iam_role.codepipeline.arn

	artifact_store {
		location = aws_s3_bucket.pipelien_bucket.bucket
		type = "s3"
	}

	stage {
		name = "Source"

		action {
			name = "Source"
			category = "Source"
			owner = "AWS"
			provider = "CodeCommit"
			version = 1
			output_artifacts = ["source"]
			configuration {
				BranchName = "develop"
				RepositoryName = aws_codecommit_repository.my_repository.repository_nmae
			}
		}
	}

	stage {
		name = "Build"

		action {
			name = "Build"
			category = "Build"
			owner = "AWS"
			provider = "CodeBuild"
			version = "1"
			run_order = 2
			input_artifacts = [
			"source"]
			output_artifacts = [
			"build"]
			configuration = {
				ProjectName = aws_codebuild_project.my_project.name
			}
		}
	}

	stage {
		name = "Deploy"

		action {
			name = "Deploy"
			category = "Deploy"
			owner = "AWS"
			provider = "ECS"
			version = 1
			run_order = 1
			input_artifacts = ["Build"]

			configuration {
				ClusterName = aws_ecs_cluster.my_clustername
				ServiceName = aws_ecs_service.my_service.name
				FileName = "${var.file_name}"
			}

		}
	}
}

gitlab-ci.yml

image: golang:1.15

veriables:
	REPO_NAME: gitlab.com/xxxxx/microservice

before_script:
	- mkdir -p $GOPATH/src/$(dirname $REPO_NAME)
	- ln -svf $CI_PROJECT_DIR $GOPATH/src/$REPO_NAME
	- cd $GOPATH/src/$REPO_NAME

stages:
	- test

test:
	stage: test
	script:
		make test

やはり terraform も結構使われてるのね。

gitlab container repositoryにpushせずに、直接ecrにpushする場合

build-demo-app:
	stage: build
	script:
		- docker build demo-app:latest
		- docker push xxxx.dkr.ecr.ap-northeast-1.amazonaws.com/demo-app-${CI_BUILD_REF_NAME}:latest

deploy-demo-app:
	stage: deploy
	script:
		- aws deploy create-deployment --application-name demo-app-${CI_BUILD_REF_NAME} --cli-input-json file://deployment.json --region ap-northeast-1

GitLab, CodePipeline, CodeDeployでデプロイする手順

1. GitLabにpush
2. GitLab CI/CDでS3にプロジェクトをアップロード
3. CodePipelineでS3アップロードを検知
4. CodeBuildでS3にアーティファクトを保存
5. CodeDeployでEC2インスタンスにデプロイ
-> CodePipelineでどうやってS3のアップロードを検知を設定するか…
–> codepipelineでソースプロバイダーにAmazon s3を選択し、検出オプションを変更するはCloudWatch Eventsを選択する

– artifactとは?
artifactとはパイプラインアクションによって処理されるアプリケーションのソースコード、構築されたアプリケーション、依存関係、定義ファイル、テンプレートなどのデータの集合体。アーティファクトは幾つかのアクションによって生成され、他のアクションによって消費される。ZipにしてS3アーティファクトバケットに保存される。

### 手順
プロジェクト作成
IAMユーザとS3バケットの作成
GitLab CI/CDの設定
デプロイ用EC2インスタンを作成/設定
CodeBuild設定
CodeDeploy設定
CodePipeline設定

$ tree
.
├── appspec.yml
├── buildspec.yml
└── hooks
└── restart.sh

$ docker run –rm -it -p 8080:8080 -v $(pwd):/app openjdk:11 bash
$ cd /app
$ .gradlew build
$ java -jar ./build/libs/cisample-0.0.1-SNAPSHOT.jar –server.port=8080

– S3 bucket versioningとは?
同じバケット内でオブジェクトの複数のバリアントを保持する手段のこと。バージョニング機能を使用すると、バケットに保持されたすべてのオブジェクトのすべてのバージョンを保持、取得、復元することができる。

### GitLab CI/CDの設定
pushを検知したらAWSのS3にプロジェクトの内容をアップロードする処理を追記する
GitLab側に変数として値を保持しておいて実行時に参照するように設定しておく

.gitlab-ci.yml

image: "alpine:3"

stages:
  - build
  - push

build:
  state: build
  script:
    - apk -no-cache add zip
    - zip -qr src.zip *
    - zip -u src.zip .gitlab-ci.yml
  artifacts:
    paths:
      - src.zip

deploy:
  stage: push
  script:
    - AWS_CLI_VERSION="1.18.31"
    - |-
      apk -no-cache add \
        python \
        py-pip \
        mailcap
    - pip install --upgrade awscli=$AWS_CLI_VERSION python-magic
    - mkdir ~/.aws
    - |-
      cat << EOS > ~/.aws/config
      [default]
      region = ap-northeast-1
      output = json
      EOS
    - |-
      cat << EOS > ~/.aws/credentials
      [default]
      aws_access_key_id = ${aws_access_key_id}
      aws_secrete_access_key = ${aws_secrete_access_key}
      EOS
    - aws s3 cp ./src.zip s3://${bucket_name}/src.zip

buildspec.yml

version: 0.2

phase:
  build:
    commands:
      - echo start build at `date`
      - ./gradlew build
artifacts:
  type: zip
  files:
    - ./appspec.yml
    - ./hooks/restart.sh
    - ./build/libs/cismaple-0.0.1-SNAPSHOT.jar
  name: artifact
  discard-paths: yes

appspec.yml

version: 0.0
os: linux
files:
  - source: /
    destination: /usr/local/app
hooks:
  AfterInstall:
    - location: restart.sh
      timeout: 180

なるほど、GitLabを使う場合でもCodePipelineを使用するのは同じなのね。フローは理解した。

[Docker] CodeDeployとCodePipeline

buildspec.yml

version: 0.2

env:
  variables:
    AWS_REGION_NAME: ap-northeast-1
    ECR_REPOSITORY_NAME: hpscript
    DOCKER_BUILDKIT: "1"

phases:
  install:
    runtime-versions:
      docker: 19

  pre_build:
    commands:
      - AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)
      - aws ecr --region ap-northeast-1 get-login-password | docker login --username AWS --password-stdin https://${AWS_ACCOUNT_ID}.dkr.ecr.ap-northeast-1.amazonaws.com/hpscript
      - REPOSITORY_URI=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION_NAME}.amazonaws.com/${ECR_REPOSITORY_NAME}
      - IMAGE_TAG=$(echo ${CODEBUILD_RESOLVED_SOURCE_VERSION} | cut -c 1-7)
  
  build:
    commands:
      - docker image build -t ${REPOSITORY_URI}:${IMAGE_TAG} .
  post_build:
    commands:
      - docker image push ${REPOSITORY_URI}:${IMAGE_TAG}
      - printf '{"name":"%s","ImageURI":"%s"}' $ECR_REPOSITORY_NAME $REPOSITORY_URI:$IMAGE_TAG > imageDetail.json

artifacts:
  files:
    - imageDetail.json     

appspec.yaml

version: 1
Resources:
- TargetService:
    Type: AWS::ECS::Service
    Properties:
      TaskDefinition: <TASK_DEFINITION>
      LoadBalancerInfo:
        ContainerName: test-container
        ContainerPort: 80

taskdef.json

{
	"executionRoleArn": "arn:aws:iam::hoge:role/escTaskExectionRole",
	"containerDefinitions": [
		{
			"logConfiguration": {
				"logDriver": "awslogs",
				"options": {
					"awslogs-group": "/ecs/fuga-def",
					"awslogs-region": "ap-northeast-1",
					"awslogs-stream-prefix": "esc"
				}
			},
			"portMappings": [
				{
					"hostPort": 80,
					"protocol": "tcp",
					"containerPort": 80
				}
			],
			"cpu": 256,
			"readonlyRootFilesystem": true,
			"memoryReservation": 512,
			"image": "<IMAGE1_NAME>",
			"essential": true,
			"name": "app"
		}
	],
	"memory": "1024",
	"taskRoleArn": null,
	"compatibilities": [
		"EC2",
		"FARGATE"
	],
	"family": "test-def",
	"requiresCompatibilities": [
		"FARGATE"
	],
	"networkMode": "awsvpc",
	"cpu": "512"
}

$ tree
.
├── Dockerfile
├── appspec.yaml
├── buildspec.yml
├── index.html
└── taskdef.json

$ git push ${CODECOMMIT_REPO_URL}

### CodePipeline
Action: Deploy
Action provider: Amazon ECS(Blue/Green)
Region: Asia Pacific(Tokyo)
Input artifact: BuildArtifact, SourceArtifact

AWS CodeDeploy application name: AppECS-test-container-service
AWS CodeDeploy deployment group: DgpECS-test-container-service
Amazon ECS task definition: SourceArtifact
AWS CodeDeploy AppSpec file: SourceArtifact

Input artifact with image details: BuildArtifact
Placeholder text in the task definition: IMAGE1_NAME
Variable namespace: DeployVariables

### エラーメッセージ
An AppSpec file is required, but could not be found in the revision

What’s wrong with you?

[Docker] CodeBuildを使いたい

$ tree
.
├── Dockerfile
└── index.html

0 directories, 2 files

$ sudo docker build -t myapache .
$ sudo docker run -dit –name myapp -p 8080:80 myapache

buildspec.yml

version: 0.2

env:
  variables:
    AWS_REGION_NAME: ap-northeast-1
    ECR_REPOSITORY_NAME: hoge
    DOCKER_BUILDKIT: "1"

phases:
  install:
    runtime-versions:
      docker: 19

  pre_build:
    commands:
      - AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)
      - aws ecr --region ap-northeast-1 get-login-password | docker login --username AWS --password-stdin https://${AWS_ACCOUNT_ID}.dkr.ecr.ap-northeast-1.amazonaws.com/hpscript
      - REPOSITORY_URI=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION_NAME}.amazonaws.com/${ECR_REPOSITORY_NAME}
      - IMAGE_TAG=$(echo ${CODEBUILD_RESOLVED_SOURCE_VERSION} | cut -c 1-7)
  
  build:
    commands:
      - docker image build -t ${REPOSITORY_URI}:${IMAGE_TAG} .
  post_build:
    commands:
      - docker image push ${REPOSITORY_URI}:${IMAGE_TAG}  

### AWS CodeBuild
– Project configuration
Project name: test
Build Badge: Enable build badge

– Source
Source provider: AWS CodeCommit
Repository: 指定
Reference type: Branch
Branch: 指定

– Environment
Environment image: Managed image
Operationg system: Amazon Linux2
Runtime(s): Standard
Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
Image version: Always use the latest image for this runtime
Environment type: Linux
Privileged: on
Service Role: New service role
Additional configuration
Timeout: 1 Hour 0 Minutes
Queued timeout: 8 Hour 0 Minutes
Certificate: Do not install any certificate
Compute: 3 GB memory, 2 vCPUs

– Buildspec
Use a buildspec file

– Artifact
Type: No artifacts

– Log
CloudWatch logs: CloudWatch logs

OK ここまでは出来た。次はcode deploy, codepipelineか。