あめがえるのITブログ

頑張りすぎない。ほどほどに頑張るブログ。

terraformでS3バケットを作成する際AccessDenyになる件

terraformでS3バケットを作成する際、AccessDeniedになることが何度が発生したので原因調査と対策をしてみた。

現象

terraform planでは問題は発生せず、applyを実行するとS3 policyでAccessDeniedとなる。 再度terraform applyを実行すると問題なく完了する。


原因

S3バケットにSSE-KMSを作成して設定する場合、KMS暗号化設定の完了を待たずS3バケットポリシー設定が行われ、暗号化が完了するまえにS3バケットポリシーが適用されるとS3バケットへのアクセスが拒否される可能性がある。

対策

KMS暗号化設定を待ってS3バケットポリシーの設定を行う

実践!

1.事象再現
1-1.terraformコード作成

# vi main.tf
provider "aws" {
  region = "ap-northeast-1"
  profile = "testvault"
}

resource "aws_kms_key" "kms_key" {
  description             = "KMS key"
  deletion_window_in_days = 10
}

resource "aws_s3_bucket" "kms_sample" {
  bucket = "kms-sample123456891111111"
  force_destroy = true
}

resource "aws_s3_bucket_public_access_block" "public_access_block" {
  bucket = aws_s3_bucket.kms_sample.id

  block_public_acls       = false
  ignore_public_acls      = false
  block_public_policy     = false
  restrict_public_buckets = false
}

resource "aws_s3_bucket_server_side_encryption_configuration" "kms_sample_encryption" {
  bucket = aws_s3_bucket.kms_sample.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.kms_key.arn
    }
  }
}

resource "aws_s3_bucket_policy" "bucket_policy" {
  # depends_on = [aws_s3_bucket_server_side_encryption_configuration.kms_sample_encryption]
  bucket = aws_s3_bucket.kms_sample.bucket

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = "*",
        Action = ["s3:GetObject"],
        Resource = [
          "${aws_s3_bucket.kms_sample.arn}/*"
        ]
      },
    ]
  })
}

1-2.コード実行

# terraform apply

1-3.AccessDeniedエラーが表示されることを確認

1-4.いったん削除

# terraform destroy


2.対策実行

2-1.aws_s3_bucket_policyのdepens_onのコメントアウトを解除

resource "aws_s3_bucket_policy" "bucket_policy" {
  # depends_on = [aws_s3_bucket_server_side_encryption_configuration.kms_sample_encryption]
  depends_on = [aws_s3_bucket_server_side_encryption_configuration.kms_sample_encryption]
  ### ここのコメントアウトを解除
  bucket = aws_s3_bucket.kms_sample.bucket

2-2.terraform実行

# terraform apply

2-3.エラーが出ないことを確認




感想

失敗できない場面でこんなエラーがでると泣きたくなる、、、

Amazon SageMakerノートブックを使ってみた


SageMaker ノートブックとは

Jupyter Notebookのマネージドサービス。

Jupyter Notebookとは

Webブラウザ上でソフトウェアを開発できる環境。
以前はPython専用環境だったが、現在ではRubyやR、Goなど40以上の言語がサポートされている。
※読み方はいろいろあるが総務省ではジュピターノートブックと読んでいる模様
https://www.stat.go.jp/teacher/comp-learn-04.html

やること

SageMagerノートブックを作成し、Pythonのコードを実行する。

実践!

1.ノートブックインスタンス作成
1-1.[SageMaker] - [ノートブック] - [ノートブックインスタンス]

1-2.[ノートブックインスタンスの作成]をクリック
1-3.[ノートブックインスタンス名]に任意の名前を入力 ※今回は[test]とした。

1-4.[ノートブックインスタンスの作成]をクリック
1-5.[ステータス]が[InService]になったことを確認


2.JupyterNotebookを接続
2-1.作成されたノートブックインスタンスの[Jupyter を開く]をクリック

2-2.[New]を選択し、[conda_python3]をクリック

2-3.Jupyterのコンソール画面が表示されたことを確認


3.Python実行
3-1.Jupyterのコンソールに下記を入力

print("test")


3-2.[Run]をクリック
3-3.testが出力されたことを確認




感想

次回から本格的に使ってみようと思ふ!

Amazon S3の静的Webホスティングを使ってみた



S3の静的Webホスティングを使う機会があったので検証してみた。

S3 静的Webホスティングとは
S3にHTMLファイルをアップロードし、パブリックアクセスを許可した状態でブラウザからHTMLファイルにアクセスさせればWebサイトとして動作させることが 可能。

S3 静的WebホスティングとWebサーバを比較した場合のメリット
・Webサーバーをセットアップする必要がない
・ファイルのアップロード、変更でデプロイが可能のため準備が容易

S3 静的WebホスティングとS3直接参照を比較した場合のメリット
トラフィックが増大しても高いスケーラビリティによってコンテンツ配信の問題が生じにくい

S3 静的Webサイトホスティングのデメリット
・動的コンテンツの利用ができないのでLambdaなどと組み合わせる必要がある
・独自のSSL/TLS証明書が利用できないのでCloudFrontなどの別サービスを組み合わせる必要がある。


やること

terraformでS3の静的Webホスティングサイトを作成し、アクセスしてページの表示状況を確認する。

構成

外部ブラウザ ⇒ S3(静的Webホスティング)

実践!

1.HTMLファイル作成
1-1.表示用、エラー用のHTMLファイル作成

# vi index.html
<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Welcome to My Static Site</title>
</head>
<body>
    <h1>Welcome to My Static Site</h1>
    <p>This is the main page.</p>
</body>
</html>
# vi error.html
<!-- error.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Error Page</title>
</head>
<body>
    <h1>Oops! Something went wrong.</h1>
    <p>We couldn't find the page you were looking for.</p>
</body>
</html>


2.環境作成
2-1.下記を実行

# vi main.tf
provider "aws" {
  region = "ap-northeast-1"
}

resource "aws_s3_bucket" "static_website" {
  bucket = "staticwebhosting12345689" # ユニークなものに変更
  force_destroy = true

  website {
    index_document = "index.html"
    error_document = "error.html"
  }
}

resource "aws_s3_bucket_policy" "bucket_policy" {
  bucket = aws_s3_bucket.static_website.bucket

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = "*",
        Action = ["s3:GetObject"],
        Resource = [
          "${aws_s3_bucket.static_website.arn}/*"
        ]
      },
    ]
  })
}

resource "aws_s3_bucket_object" "index_html" {
  bucket = aws_s3_bucket.static_website.id
  key    = "index.html"
  source = "index.html"
  content_type = "text/html"
}

resource "aws_s3_bucket_object" "error_html" {
  bucket = aws_s3_bucket.static_website.id
  key    = "error.html"
  source = "error.html"
  content_type = "text/html"
}

2-2.terraform実行

# terraform plan
# terraform apply


3.動作確認
3-1.[S3] - [バケット] - 作成したバケットを選択

3-2.[プロパティ]タブを選択

3-3.[静的ウェブサイトホスティング]からURLをクリックし、ページが表示されることを確認


3-4.静的ウェブサイトホスティングのURLに適当な文字をいれ、エラーページが表示されることを確認
例)http://staticwebhosting12345689.s3-website-ap-northeast-1.amazonaws.com/testtest



感想

勉強しなくてもいい人生を送りたい

AWS CloudFront+APIGateway(WAF)でカスタムヘッダーで接続元を制限してみた

CloudFront+APIGatewayの構成で送信元をCloudFrontのみに制限する場合に、APIGatewayにWAFを設定し、カスタムヘッダーで制限するのが一般的らしいのでやってみた。

やること

terraformで下記を実施。
CloudFront+APIGatewayを作り、APIGatewayにWAFをアタッチ
CloudFrontにカスタムヘッダーを設定
WAFにカスタムヘッダーを含まないHTTPリクエストはブロックするルールを設定

構成



実践!

1.環境作成
1-1.下記を実行し、コードを作成

# vi main.tf
provider "aws" {
  region = "ap-northeast-1"
}

## Lambda関数
resource "aws_lambda_function" "example" {
  function_name = "example_lambda_function"
  runtime       = "python3.8"
  handler       = "lambda_function.lambda_handler"

  filename = "lambda_function.zip"

  role = aws_iam_role.lambda_exec.arn
}

## Lambdaロール
resource "aws_iam_role" "lambda_exec" {
  name = "lambda_exec_role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Principal = {
          Service = "lambda.amazonaws.com"
        },
      },
    ],
  })
}

# WAF
resource "aws_wafv2_web_acl" "example" {
  name        = "example-acl"
  description = "An example ACL"
  scope       = "REGIONAL"

  default_action {
    allow {}
  }

  rule {
    name     = "BlockMissingHeader"
    priority = 0  # 優先度を高く設定

    action {
      block {}
    }

    statement {
      not_statement {
        statement {
          byte_match_statement {
            field_to_match {
              single_header {
                name = "testheader"
              }
            }
            positional_constraint = "EXACTLY"
            search_string         = "testvalue"
            text_transformation {
              priority = 0
              type     = "NONE"
            }
          }
        }
      }
    }
    
    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "BlockMissingHeader"
      sampled_requests_enabled   = true
    }
  }

  visibility_config {
    cloudwatch_metrics_enabled = false
    metric_name                = "example-acl"
    sampled_requests_enabled   = false
  }
}

## APIGateway
resource "aws_api_gateway_rest_api" "example" {
  name        = "ExampleAPI"
  description = "Example API"
}

resource "aws_api_gateway_resource" "example" {
  rest_api_id = aws_api_gateway_rest_api.example.id
  parent_id   = aws_api_gateway_rest_api.example.root_resource_id
  path_part   = "examplepath"
}

resource "aws_api_gateway_method" "example" {
  rest_api_id   = aws_api_gateway_rest_api.example.id
  resource_id   = aws_api_gateway_resource.example.id
  http_method   = "GET"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "example" {
  rest_api_id = aws_api_gateway_rest_api.example.id
  resource_id = aws_api_gateway_resource.example.id
  http_method = aws_api_gateway_method.example.http_method

  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = aws_lambda_function.example.invoke_arn
}

resource "aws_api_gateway_deployment" "example" {
  depends_on = [aws_api_gateway_integration.example]

  rest_api_id = aws_api_gateway_rest_api.example.id
  # stage_name  = "test"
}

resource "aws_api_gateway_stage" "example_stage" {
  stage_name    = "test"
  rest_api_id   = aws_api_gateway_rest_api.example.id
  deployment_id = aws_api_gateway_deployment.example.id

  xray_tracing_enabled = false
}

resource "aws_lambda_permission" "api_gateway" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.example.function_name
  principal     = "apigateway.amazonaws.com"

  source_arn = "${aws_api_gateway_rest_api.example.execution_arn}/*/*/*"
}

# APIGatewayにWAFを関連付け
resource "aws_wafv2_web_acl_association" "example_association" {
  resource_arn = aws_api_gateway_stage.example_stage.arn
  web_acl_arn  = aws_wafv2_web_acl.example.arn
}

# CloudFront
resource "aws_cloudfront_distribution" "example" {
  origin {
    ## domain_name = aws_api_gateway_rest_api.example.domain_name
    ## domain_name = "${aws_api_gateway_rest_api.example.id}.execute-api.ap-northeast-1.amazonaws.com"
    domain_name = "${aws_api_gateway_rest_api.example.id}.execute-api.ap-northeast-1.amazonaws.com"
    origin_id   = "APIGatewayOrigin"

    custom_header {
      name  = "testheader"
      value = "testvalue"
    }

    custom_origin_config {
      http_port                = 80
      https_port               = 443
      origin_protocol_policy   = "http-only"
      origin_ssl_protocols     = ["TLSv1", "TLSv1.1", "TLSv1.2"]
    }
  }

  enabled             = true
  default_root_object = "index.html"

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "APIGatewayOrigin"

    forwarded_values {
      query_string = false

      cookies {
        forward = "none"
      }
    }

    viewer_protocol_policy = "redirect-to-https"
    min_ttl                = 0
    default_ttl            = 3600
    max_ttl                = 86400
  }

  ordered_cache_behavior {
    path_pattern     = "/test/*"  # ステージ名をビヘイビアのパスパターンに追加
    allowed_methods  = ["GET", "HEAD", "OPTIONS"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "APIGatewayOrigin"

    forwarded_values {
      query_string = true

      cookies {
        forward = "none"
      }
    }

    viewer_protocol_policy = "redirect-to-https"  # HTTPSを強制
    min_ttl                = 0
    default_ttl            = 3600
    max_ttl                = 86400
  }

  price_class = "PriceClass_100"

  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }

  viewer_certificate {
    cloudfront_default_certificate = true
  }
}

1-2.terraform実行

# terraform plan
# terraform apply


2.動作確認(CloudFront)
2-1.[CloudFront] - ディストリビューションドメイン名確認

2-2.下記をAWS外部から実行し、200OKが返ってくることを確認

> curl https://d15xfm2x3yiw75.cloudfront.net/test/examplepath -L -v

*   Trying 143.204.130.97:443...
* Connected to d15xfm2x3yiw75.cloudfront.net (143.204.130.97) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.1
> GET /test/examplepath HTTP/1.1
> Host: d15xfm2x3yiw75.cloudfront.net
> User-Agent: curl/8.4.0
> Accept: */*
>
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 20
< Connection: keep-alive
< Date: Thu, 23 Nov 2023 23:40:41 GMT
< x-amzn-RequestId: eed3548c-6b0d-432d-9016-7cfb7b5ea3e7
< x-amz-apigw-id: O4BnAH3xNjMEHog=
< X-Amzn-Trace-Id: Root=1-655fe2f9-37eab6833bcb060519da7ff4
< Via: 1.1 0f1561546531d4bd49ef6c69e8989712.cloudfront.net (CloudFront), 1.1 05aec04162b0fed6e9762cd1edd66a72.cloudfront.net (CloudFront)
< X-Amz-Cf-Pop: SFO5-C3
< X-Cache: Hit from cloudfront
< X-Amz-Cf-Pop: SFO5-C1
< X-Amz-Cf-Id: DWcy_JqNhUQMEn3O-rKtfPbE_qs4R8PDpEjsaFh0a-JHJ7Mp3TQBLw==
< Age: 102
<
"Hello from Lambda!"* Connection #0 to host d15xfm2x3yiw75.cloudfront.net left intact


3.動作確認(APIGateway)
3-1.[API Gateway] - 作成したAPIを選択

3-2. [ステージ]

3-3.[GET]を選択し呼び出しURLを確認

3-4.下記をAWS外部から実行し、403 Forbiddenが返ってくることを確認

> curl https://i7yfsdlxxk.execute-api.ap-northeast-1.amazonaws.com/tes
t/examplepath -L -v

*   Trying 99.84.133.47:443...
* Connected to i7yfsdlxxk.execute-api.ap-northeast-1.amazonaws.com (99.84.133.47) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.1
> GET /test/examplepath HTTP/1.1
> Host: i7yfsdlxxk.execute-api.ap-northeast-1.amazonaws.com
> User-Agent: curl/8.4.0
> Accept: */*
>
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
< HTTP/1.1 403 Forbidden
< Content-Type: application/json
< Content-Length: 23
< Connection: keep-alive
< Date: Fri, 24 Nov 2023 00:09:38 GMT
< x-amzn-RequestId: 7f69ce70-ae7c-4d44-8a9b-bf4d7d681b47
< x-amzn-ErrorType: ForbiddenException
< x-amz-apigw-id: O4F2aFRQtjMEKBQ=
< X-Cache: Error from cloudfront
< Via: 1.1 591400b2958a6516fdef3d2bc0ac208e.cloudfront.net (CloudFront)
< X-Amz-Cf-Pop: NRT57-C3
< X-Amz-Cf-Id: JbsXNRIGw7dqYD_HoBqzykfb69mwmCA5X_kZXxvNP4wI2hYTL8h2dA==
<
{"message":"Forbidden"}* Connection #0 to host i7yfsdlxxk.execute-api.ap-northeast-1.amazonaws.com left intact



感想

簡単な設定なはずなのになかなかに苦戦した・・・( ̄д ̄)ハァ

AWS RDS Proxy(AuroraMySQL)+SecretManager+VPCEndpoint構成を作成してみた

前にRDS(MySQL)をやったのでAuroraもやってみた。
違いはDBのClusterがあるかないかでそれによりRDSProxyの設定項目が少し変わる程度。
amegaeru.hatenablog.jp

構成




実践!

1.環境構築
1-1.terraformコードを作成

# vi main.tf
provider "aws" {
  region = "ap-northeast-1"
}

variable "env" {
  default = {
    env_name = "test"
    vpc_cidr = "10.0.0.0/16"
    sb_az1a = "ap-northeast-1a"
    sb_az1a_cidr = "10.0.1.0/24"
    sb_az1c = "ap-northeast-1c"
    sb_az1c_cidr = "10.0.2.0/24"
    sb_az1a_p = "ap-northeast-1a"
    sb_az1a_cidr_p = "10.0.3.0/24"
    sb_az1c_p = "ap-northeast-1c"
    sb_az1c_cidr_p = "10.0.4.0/24"
  }
}

### VPC
resource "aws_vpc" "vpc" {
    cidr_block = "${var.env.vpc_cidr}"
    enable_dns_support   = true
    enable_dns_hostnames = true
    tags = {
        Name = "${var.env.env_name}_vpc"
    }
}

### Subnet
resource "aws_subnet" "public_1a" {
  vpc_id = "${aws_vpc.vpc.id}"
  availability_zone = "${var.env.sb_az1a}"
  cidr_block        = "${var.env.sb_az1a_cidr}"
  tags = {
    Name = "${var.env.env_name}_public_1a"
  }
}

resource "aws_subnet" "public_1c" {
  vpc_id = "${aws_vpc.vpc.id}"
  availability_zone = "${var.env.sb_az1c}"
  cidr_block        = "${var.env.sb_az1c_cidr}"
  tags = {
    Name = "${var.env.env_name}_public_1c"
  }
}

resource "aws_subnet" "private_1a" {
  vpc_id = "${aws_vpc.vpc.id}"
  availability_zone = "${var.env.sb_az1a_p}"
  cidr_block        = "${var.env.sb_az1a_cidr_p}"
  tags = {
    Name = "${var.env.env_name}_private_1a"
  }
}

resource "aws_subnet" "private_1c" {
  vpc_id = "${aws_vpc.vpc.id}"
  availability_zone = "${var.env.sb_az1c_p}"
  cidr_block        = "${var.env.sb_az1c_cidr_p}"
  tags = {
    Name = "${var.env.env_name}_private_1c"
  }
}

### InternetGateway
resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.vpc.id

  tags = {
    Name = "${var.env.env_name}_igw"
  }
}

### RouteTable_public
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "${var.env.env_name}_public_route_table"
  }
}

resource "aws_route_table_association" "public_1a" {
  subnet_id      = aws_subnet.public_1a.id
  route_table_id = aws_route_table.public.id
}

resource "aws_route_table_association" "public_1c" {
  subnet_id      = aws_subnet.public_1c.id
  route_table_id = aws_route_table.public.id
}

### RouteTable_private
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.vpc.id

  tags = {
    Name = "${var.env.env_name}_private_route_table"
  }
}

resource "aws_route_table_association" "private_1a" {
  subnet_id      = aws_subnet.private_1a.id
  route_table_id = aws_route_table.private.id
}

resource "aws_route_table_association" "private_1c" {
  subnet_id      = aws_subnet.private_1c.id
  route_table_id = aws_route_table.private.id
}

### VPCEndpoint SecretManager
resource "aws_vpc_endpoint" "secretsmanager" {
  vpc_id            = aws_vpc.vpc.id
  service_name      = "com.amazonaws.ap-northeast-1.secretsmanager"
  vpc_endpoint_type = "Interface"
  subnet_ids        = [aws_subnet.private_1a.id, aws_subnet.private_1c.id]

  security_group_ids = [aws_security_group.example_vpce_sg.id]
}

### SecurityGroup(VPCEndpoint)
resource "aws_security_group" "example_vpce_sg" {
  name        = "example_vpce_sg"
  description = "Allow RDS"
  vpc_id      = aws_vpc.vpc.id

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group_rule" "egress_vpce_all" {
  type        = "egress"
  from_port   = 0
  to_port     = 0
  protocol    = "-1"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_vpce_sg.id
}

### SecurityGroup(RDS Proxy)
resource "aws_security_group" "example_rdsp_sg" {
  name        = "example_rdsp_sg"
  description = "RDS Proxy"
  vpc_id      = aws_vpc.vpc.id

  ingress {
    from_port   = 3306
    to_port     = 3306
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group_rule" "egress_rdsp_all" {
  type        = "egress"
  from_port   = 0
  to_port     = 0
  protocol    = "-1"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_rdsp_sg.id
}

### SecurityGroup(RDS)
resource "aws_security_group" "example_rds_sg" {
  name        = "example_rds_sg"
  description = "RDS"
  vpc_id      = aws_vpc.vpc.id

  ingress {
    from_port   = 3306
    to_port     = 3306
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

### SecurityGroup(EC2)
resource "aws_security_group" "example_ec2_sg" {
  name        = "example_ec2_sg"
  vpc_id      = aws_vpc.vpc.id
}

resource "aws_security_group_rule" "ingress_ec2_http" {
  type        = "ingress"
  from_port   = 80
  to_port     = 80
  protocol    = "tcp"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_ec2_sg.id
}

resource "aws_security_group_rule" "ingress_ec2_ssh" {
  type        = "ingress"
  from_port   = 22
  to_port     = 22
  protocol    = "tcp"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_ec2_sg.id
}

resource "aws_security_group_rule" "egress_ec2_all" {
  type        = "egress"
  from_port   = 0
  to_port     = 0
  protocol    = "-1"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_ec2_sg.id
}

### IAM Role(RDS)
resource "aws_iam_role" "example_rds_iam_role" {
  name = "example-rds-iam-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Principal = {
          Service = "rds.amazonaws.com"
        },
      },
    ],
  })
}

# Secrets Managerへのアクセスを許可するIAMポリシー
resource "aws_iam_policy" "secretsmanager_access" {
  name        = "secretsmanager_access_policy"
  description = "Policy to allow RDS Proxy to access Secrets Manager"

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = [
          "secretsmanager:GetSecretValue",
          "secretsmanager:DescribeSecret"
        ],
        Effect   = "Allow",
        Resource = "*"
      }
    ]
  })
}

# IAMポリシーをRDS ProxyのIAMロールにアタッチ
resource "aws_iam_role_policy_attachment" "secretsmanager_access_attachment" {
  role       = aws_iam_role.example_rds_iam_role.name
  policy_arn = aws_iam_policy.secretsmanager_access.arn
}

### IAM Role(EC2)
resource "aws_iam_role" "example-ec2-ssm-role" {
  name               = "example-ec2-ssm-role"
  assume_role_policy = data.aws_iam_policy_document.example-ec2-assume-role.json
}

data "aws_iam_policy_document" "example-ec2-assume-role" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service" 
      identifiers = ["ec2.amazonaws.com"]
    }
  }
}

data "aws_iam_policy" "example-ec2-policy_ssm_managed_instance_core" {
  arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

resource "aws_iam_role_policy_attachment" "example-ec2-ssm_managed_instance_core" {
  role       = aws_iam_role.example-ec2-ssm-role.name
  policy_arn = data.aws_iam_policy.example-ec2-policy_ssm_managed_instance_core.arn
}

resource "aws_iam_instance_profile" "example-ec2-profile" {
  name = "example-ec2-profile"
  role = aws_iam_role.example-ec2-ssm-role.name
}

### EC2
resource "aws_instance" "example" {
  ami           = "ami-06180cd4edb6844d2"
  instance_type = "t2.micro"
  subnet_id = aws_subnet.public_1a.id
  security_groups = [aws_security_group.example_ec2_sg.id]
  associate_public_ip_address = true
  iam_instance_profile = aws_iam_instance_profile.example-ec2-profile.name

  user_data = <<-EOF
                #!/bin/bash
                sudo yum update -y
                sudo amazon-linux-extras install epel -y
                sudo yum install nginx -y
                sudo systemctl start nginx
                sudo systemctl enable nginx
                sudo yum install mysql -y
              EOF
              
  tags = {
    Name = "example-instance"
  }
}

### SecretManager
resource "aws_secretsmanager_secret" "example" {
  name = "mysecret"
}

resource "aws_secretsmanager_secret_version" "example" {
  secret_id     = aws_secretsmanager_secret.example.id
  secret_string = "{\"username\":\"user\",\"password\":\"mypassword\"}"
}

### RDS Proxy
resource "aws_db_proxy" "example" {
  name                   = "example"
  debug_logging          = false
  engine_family          = "MYSQL"
  idle_client_timeout    = 1800
  require_tls            = false
  role_arn               = aws_iam_role.example_rds_iam_role.arn
  vpc_security_group_ids = [aws_security_group.example_rdsp_sg.id]
  vpc_subnet_ids         = [aws_subnet.private_1a.id, aws_subnet.private_1c.id]

  auth {
    auth_scheme = "SECRETS"
    description = "example"
    iam_auth    = "DISABLED"
    secret_arn  = aws_secretsmanager_secret.example.arn
  }
}

# RDS Proxyターゲットグループ
resource "aws_db_proxy_default_target_group" "default" {
  db_proxy_name = aws_db_proxy.example.name
}

resource "aws_db_proxy_target" "example" {
  db_proxy_name = aws_db_proxy.example.name
  target_group_name = "default"
  db_cluster_identifier = aws_rds_cluster.aurora_cluster.id
  ## ここがdb_cluster_identifierに代わって値はAuroraのものに変更
}

### RDS
## ここがClusterの設定に変更
resource "aws_rds_cluster" "aurora_cluster" {
  cluster_identifier = "aurora-cluster-demo"
  engine             = "aurora-mysql"
  engine_version     = "5.7.mysql_aurora.2.07.10"
  database_name      = "mydb"
  master_username    = "user"
  master_password    = "mypassword"
  db_subnet_group_name = aws_db_subnet_group.example.name
  vpc_security_group_ids = [aws_security_group.example_rds_sg.id]
  skip_final_snapshot = true
}

## ここがClusterの設定に変更
resource "aws_rds_cluster_instance" "aurora_instances" {
  count              = 2
  identifier         = "aurora-instance-${count.index}"
  cluster_identifier = aws_rds_cluster.aurora_cluster.id
  instance_class     = "db.r4.large"
  engine             = "aurora-mysql"
  engine_version     = "5.7.mysql_aurora.2.07.10"
}

resource "aws_db_subnet_group" "example" {
  name       = "my-db-subnet-group"
  subnet_ids = [aws_subnet.private_1a.id, aws_subnet.private_1c.id]

  tags = {
    Name = "My DB Subnet Group"
  }
}

1-2.terraform実行

# terraform plan
# terraform apply


2.MySQL接続確認
2-1.[EC2] - 作成されたEC2インスタンスを選択 - [接続]

2-2.[接続]

2-3.接続できることを確認

2-4.RDS Proxyのエンドポイントへmysqlコマンドで接続し、接続できることを確認

sh-4.2$ mysql -h example.proxy-ckmgc9vjvwrf.ap-northeast-1.rds.amazonaws.com -u user -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1617414075
Server version: 5.7.12 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> exit
Bye



terraformで何度か作り直しをする場合、下記コマンドをCloudShellから実行しSecretManagerのシークレットを削除すること。
※terraformで削除しても7日間は使えない状態で残っていて、再作成した際にはすでにあるというエラーになるのでCLIでバッサリ削除する。

$ aws secretsmanager delete-secret --secret-id mysecret --force-delete-without-recovery
{
    "ARN": "arn:aws:secretsmanager:ap-northeast-1:790301748424:secret:mysecret-3Q9YYz",
    "Name": "mysecret",
    "DeletionDate": "2023-11-23T16:12:26.020000+00:00"
}



感想

半日かかった。。。

AWS RDS Proxy(MySQL)+SecretManager+VPCEndpoint構成を作成してみた

RDSProxyを使った構成を作成してみた。

RDSProxyとは

 ・接続プーリングができる:多数の同時接続をしていることに伴うオーバーヘッドを削減できる。
 ・RDS Proxy の認証:パスワード認証、IAM認証などが使える。
 ・TLS/SSL が使える:RDS Proxyとアプリケーション間の接続は TLS1.2 を利用できる。
 ・フェイルオーバーによる高可用性:元のデータベースインスタンスが使用できなくなったときに、別のインスタンスに置き換えが可能

やること

terraformでRDS(MySQL)の前段にRDSProxyを置きパスワード認証でMySQLに接続する構成を作成する。
RDSProxyのパスワード認証の場合、SecretManagerにパスワード情報を記載し、そこで認証を行うのでSecretManagerも作っていく。
さらにDBがいるSubnetはインターネットに接続できないことも多いのでVPCEndpointを使い内部通信を行えるようにする。

構成



実践!

1.環境構築
1-1.terraformコードを作成

# vi main.tf
provider "aws" {
  region = "ap-northeast-1"
}

variable "env" {
  default = {
    env_name = "test"
    vpc_cidr = "10.0.0.0/16"
    sb_az1a = "ap-northeast-1a"
    sb_az1a_cidr = "10.0.1.0/24"
    sb_az1c = "ap-northeast-1c"
    sb_az1c_cidr = "10.0.2.0/24"
    sb_az1a_p = "ap-northeast-1a"
    sb_az1a_cidr_p = "10.0.3.0/24"
    sb_az1c_p = "ap-northeast-1c"
    sb_az1c_cidr_p = "10.0.4.0/24"
  }
}

### VPC
resource "aws_vpc" "vpc" {
    cidr_block = "${var.env.vpc_cidr}"
    enable_dns_support   = true
    enable_dns_hostnames = true
    tags = {
        Name = "${var.env.env_name}_vpc"
    }
}

### Subnet
resource "aws_subnet" "public_1a" {
  vpc_id = "${aws_vpc.vpc.id}"
  availability_zone = "${var.env.sb_az1a}"
  cidr_block        = "${var.env.sb_az1a_cidr}"
  tags = {
    Name = "${var.env.env_name}_public_1a"
  }
}

resource "aws_subnet" "public_1c" {
  vpc_id = "${aws_vpc.vpc.id}"
  availability_zone = "${var.env.sb_az1c}"
  cidr_block        = "${var.env.sb_az1c_cidr}"
  tags = {
    Name = "${var.env.env_name}_public_1c"
  }
}

resource "aws_subnet" "private_1a" {
  vpc_id = "${aws_vpc.vpc.id}"
  availability_zone = "${var.env.sb_az1a_p}"
  cidr_block        = "${var.env.sb_az1a_cidr_p}"
  tags = {
    Name = "${var.env.env_name}_private_1a"
  }
}

resource "aws_subnet" "private_1c" {
  vpc_id = "${aws_vpc.vpc.id}"
  availability_zone = "${var.env.sb_az1c_p}"
  cidr_block        = "${var.env.sb_az1c_cidr_p}"
  tags = {
    Name = "${var.env.env_name}_private_1c"
  }
}

### InternetGateway
resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.vpc.id

  tags = {
    Name = "${var.env.env_name}_igw"
  }
}

### RouteTable_public
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "${var.env.env_name}_public_route_table"
  }
}

resource "aws_route_table_association" "public_1a" {
  subnet_id      = aws_subnet.public_1a.id
  route_table_id = aws_route_table.public.id
}

resource "aws_route_table_association" "public_1c" {
  subnet_id      = aws_subnet.public_1c.id
  route_table_id = aws_route_table.public.id
}

### RouteTable_private
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.vpc.id

  tags = {
    Name = "${var.env.env_name}_private_route_table"
  }
}

resource "aws_route_table_association" "private_1a" {
  subnet_id      = aws_subnet.private_1a.id
  route_table_id = aws_route_table.private.id
}

resource "aws_route_table_association" "private_1c" {
  subnet_id      = aws_subnet.private_1c.id
  route_table_id = aws_route_table.private.id
}

resource "aws_vpc_endpoint" "secretsmanager" {
  vpc_id            = aws_vpc.vpc.id
  service_name      = "com.amazonaws.ap-northeast-1.secretsmanager"
  vpc_endpoint_type = "Interface"
  subnet_ids        = [aws_subnet.private_1a.id, aws_subnet.private_1c.id]

  security_group_ids = [aws_security_group.example_vpce_sg.id]
}

### SecurityGroup(VPCEndpoint)
resource "aws_security_group" "example_vpce_sg" {
  name        = "example_vpce_sg"
  description = "Allow RDS"
  vpc_id      = aws_vpc.vpc.id

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group_rule" "egress_vpce_all" {
  type        = "egress"
  from_port   = 0
  to_port     = 0
  protocol    = "-1"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_vpce_sg.id
}

### SecurityGroup(RDS Proxy)
resource "aws_security_group" "example_rdsp_sg" {
  name        = "example_rdsp_sg"
  description = "RDS Proxy"
  vpc_id      = aws_vpc.vpc.id

  ingress {
    from_port   = 3306
    to_port     = 3306
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group_rule" "egress_rdsp_all" {
  type        = "egress"
  from_port   = 0
  to_port     = 0
  protocol    = "-1"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_rdsp_sg.id
}

### SecurityGroup(RDS)
resource "aws_security_group" "example_rds_sg" {
  name        = "example_rds_sg"
  description = "RDS"
  vpc_id      = aws_vpc.vpc.id

  ingress {
    from_port   = 3306
    to_port     = 3306
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

### SecurityGroup(EC2)
resource "aws_security_group" "example_ec2_sg" {
  name        = "example_ec2_sg"
  vpc_id      = aws_vpc.vpc.id
}

resource "aws_security_group_rule" "ingress_ec2_http" {
  type        = "ingress"
  from_port   = 80
  to_port     = 80
  protocol    = "tcp"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_ec2_sg.id
}

resource "aws_security_group_rule" "ingress_ec2_ssh" {
  type        = "ingress"
  from_port   = 22
  to_port     = 22
  protocol    = "tcp"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_ec2_sg.id
}

resource "aws_security_group_rule" "egress_ec2_all" {
  type        = "egress"
  from_port   = 0
  to_port     = 0
  protocol    = "-1"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example_ec2_sg.id
}

### IAM Role(RDS)
resource "aws_iam_role" "example_rds_iam_role" {
  name = "example-rds-iam-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Principal = {
          Service = "rds.amazonaws.com"
        },
      },
    ],
  })
}

# Secrets Managerへのアクセスを許可するIAMポリシー
resource "aws_iam_policy" "secretsmanager_access" {
  name        = "secretsmanager_access_policy"
  description = "Policy to allow RDS Proxy to access Secrets Manager"

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = [
          "secretsmanager:GetSecretValue",
          "secretsmanager:DescribeSecret"
        ],
        Effect   = "Allow",
        Resource = "*"
      }
    ]
  })
}

# IAMポリシーをRDS ProxyのIAMロールにアタッチ
resource "aws_iam_role_policy_attachment" "secretsmanager_access_attachment" {
  role       = aws_iam_role.example_rds_iam_role.name
  policy_arn = aws_iam_policy.secretsmanager_access.arn
}

### IAM Role(EC2)
resource "aws_iam_role" "example-ec2-ssm-role" {
  name               = "example-ec2-ssm-role"
  assume_role_policy = data.aws_iam_policy_document.example-ec2-assume-role.json
}

data "aws_iam_policy_document" "example-ec2-assume-role" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service" 
      identifiers = ["ec2.amazonaws.com"]
    }
  }
}

data "aws_iam_policy" "example-ec2-policy_ssm_managed_instance_core" {
  arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

resource "aws_iam_role_policy_attachment" "example-ec2-ssm_managed_instance_core" {
  role       = aws_iam_role.example-ec2-ssm-role.name
  policy_arn = data.aws_iam_policy.example-ec2-policy_ssm_managed_instance_core.arn
}

resource "aws_iam_instance_profile" "example-ec2-profile" {
  name = "example-ec2-profile"
  role = aws_iam_role.example-ec2-ssm-role.name
}

### EC2
resource "aws_instance" "example" {
  ami           = "ami-06180cd4edb6844d2"
  instance_type = "t2.micro"
  subnet_id = aws_subnet.public_1a.id
  security_groups = [aws_security_group.example_ec2_sg.id]
  associate_public_ip_address = true
  iam_instance_profile = aws_iam_instance_profile.example-ec2-profile.name

  user_data = <<-EOF
                #!/bin/bash
                sudo yum update -y
                sudo amazon-linux-extras install epel -y
                sudo yum install mysql -y
              EOF
              
  tags = {
    Name = "example-instance"
  }
}

### SecretManager
resource "aws_secretsmanager_secret" "example" {
  name = "mysecret1"
}

resource "aws_secretsmanager_secret_version" "example" {
  secret_id     = aws_secretsmanager_secret.example.id
  secret_string = "{\"username\":\"user\",\"password\":\"mypassword\"}"
}

### RDS Proxy
resource "aws_db_proxy" "example" {
  name                   = "example"
  debug_logging          = false
  engine_family          = "MYSQL"
  idle_client_timeout    = 1800
  require_tls            = false
  role_arn               = aws_iam_role.example_rds_iam_role.arn
  vpc_security_group_ids = [aws_security_group.example_rdsp_sg.id]
  vpc_subnet_ids         = [aws_subnet.private_1a.id, aws_subnet.private_1c.id]

  auth {
    auth_scheme = "SECRETS"
    description = "example"
    iam_auth    = "DISABLED"
    secret_arn  = aws_secretsmanager_secret.example.arn
  }
}

# RDS Proxyターゲットグループ
resource "aws_db_proxy_default_target_group" "default" {
  db_proxy_name = aws_db_proxy.example.name
}

resource "aws_db_proxy_target" "example" {
  db_proxy_name = aws_db_proxy.example.name
  target_group_name = "default"
  db_instance_identifier = aws_db_instance.example.identifier
}

### RDS
resource "aws_db_instance" "example" {
  allocated_storage    = 20
  storage_type         = "gp2"
  engine               = "mysql"
  engine_version       = "5.7"
  instance_class       = "db.t2.micro"
  identifier = "mydb-instance"
  db_name    = "mydb"
  username             = "user"
  password             = "mypassword"
  parameter_group_name = "default.mysql5.7"
  skip_final_snapshot  = true

  db_subnet_group_name = aws_db_subnet_group.example.name
  vpc_security_group_ids = [aws_security_group.example_rds_sg.id]
}

resource "aws_db_subnet_group" "example" {
  name       = "my-db-subnet-group"
  subnet_ids = [aws_subnet.private_1a.id, aws_subnet.private_1c.id]

  tags = {
    Name = "My DB Subnet Group"
  }
}

1-2.terraform実行

# terraform plan
# terraform apply


2.MySQL接続確認
2-1.[EC2] - 作成されたEC2インスタンスを選択 - [接続]

2-2.[接続]

2-3.接続できることを確認

2-4.RDS Proxyのエンドポイントへmysqlコマンドで接続し、接続できることを確認

sh-4.2$ mysql -h example.proxy-ckmgc9vjvwrf.ap-northeast-1.rds.amazonaws.com -u user -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 742749209
Server version: 5.7.42 Source distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> exit
Bye



感想

まぁできますわな。

AWS SecretManagerで同じ名前のシークレットが作成できない

terraformでSecretManagerの検証している際に下記エラーが発生してシークレットが再作成できない事象が発生したので調べてみた。

│ Error: creating Secrets Manager Secret: InvalidRequestException: You can't create this secret because a secret with this name is already scheduled for deletion.


Secrets Manager は、復旧期間が 7 日以上経過した後、シークレットを削除するようにスケジュールします。 つまり、リカバリウィンドウが終了するまで、AWS マネジメントコンソールで同じ名前を使用してシークレットを再作成することはできません。 repost.aws

※なるほど

◆調べてみた

1.コマンドラインで調べてみた。

$ aws secretsmanager delete-secret --secret-id mysecret
{
    "ARN": "arn:aws:secretsmanager:ap-northeast-1:xxxxxxxxxxxx:secret:mysecret-vqHjwa",
    "Name": "mysecret",
    "DeletionDate": "2023-12-23T08:49:04.550000+00:00"
}

※確かに存在する。

2.GUIでも確認
※隠れているので設定で見れるようにする。
2-1.[SecretManager] - 設定

2-2.[削除予定のシークレットを表示する]にチェックし、[保存]を押下

2-3.表示されたことを確認


3.下記コマンドで強制的に削除できるもよう。

$ aws secretsmanager delete-secret --secret-id mysecret --force-delete-without-recovery
{
    "ARN": "arn:aws:secretsmanager:ap-northeast-1:xxxxxxxxxxxx:secret:mysecret-vqHjwa",
    "Name": "mysecret",
    "DeletionDate": "2023-11-23T08:51:44.386000+00:00"
}


※消えた!


感想

確かECRのリポジトリは削除した場合、表に表示されなくなるが、アクションから表示できた記憶。表示する方法くらい統一しろよと思う今日この頃。