Add native Packer support (Atmos loves Packer) #1394

aknysh · 2025-08-06T20:08:17Z

what

Add native HashiCorp Packer support to Atmos
Add Packer docs and atmos packer CLI commands docs
- https://pr-1394.atmos-docs.ue2.dev.plat.cloudposse.org/core-concepts/projects/configuration/packer/
- https://pr-1394.atmos-docs.ue2.dev.plat.cloudposse.org/cli/commands/packer/usage/
Add tests

why

Use the power of Atmos components, stacks, imports, inheritance, templating and YAML functions to configure and provision Packer components to build machine images for multiple cloud platforms from Packer templates

description

Atmos natively supports HashiCorp Packer and lets you create identical machine images for multiple platforms from a single source template using the power of Atmos components, stacks, imports, inheritance, templating and YAML functions. It's compatible with every version of Packer and designed to work with multiple different versions of it concurrently.

Configure Packer in `atmos.yaml`

base_path: "./"

components:
  packer:
    # Can also be set using 'ATMOS_COMPONENTS_PACKER_COMMAND' ENV var, or '--packer-command' command-line argument
    command: packer
    # Can also be set using 'ATMOS_COMPONENTS_PACKER_BASE_PATH' ENV var, or '--packer-dir' command-line argument
    base_path: "components/packer"

stacks:
  base_path: "stacks"
  included_paths:
    - "deploy/**/*"
  excluded_paths:
    - "**/_defaults.yaml"
  name_template: "{{ .vars.stage }}"

Add Packer template (Packer component)

packer {
  required_plugins {
    # https://developer.hashicorp.com/packer/integrations/hashicorp/amazon
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
  }
}

variable "region" {
  type        = string
  description = "AWS Region"
}

variable "stage" {
  type    = string
  default = null
}

variable "ami_org_arns" {
  type        = list(string)
  description = "List of Amazon Resource Names (ARN) of AWS Organizations that have access to launch the resulting AMI(s). By default no organizations have permission to launch the AMI"
  default     = []
}

variable "ami_ou_arns" {
  type        = list(string)
  description = "List of Amazon Resource Names (ARN) of AWS Organizations organizational units (OU) that have access to launch the resulting AMI(s). By default no organizational units have permission to launch the AMI."
  default     = []
}

variable "ami_users" {
  type        = list(string)
  description = "List of account IDs that have access to launch the resulting AMI(s). By default no additional users other than the user creating the AMI has permissions to launch it."
  default     = []
}

# https://developer.hashicorp.com/packer/integrations/hashicorp/amazon#authentication
variable "assume_role_arn" {
  type        = string
  description = "Amazon Resource Name (ARN) of the IAM Role to assume. Refer to https://developer.hashicorp.com/packer/integrations/hashicorp/amazon#authentication"
}

variable "provisioner_shell_commands" {
  type        = list(string)
  description = "List of commands to execute on the machine that Packer builds"
  default     = []
}

source "amazon-ebs" "al2023" {
  ami_name      = var.ami_name
  source_ami    = var.source_ami
  instance_type = var.instance_type
  region        = var.region
  ssh_username  = var.ssh_username
  ami_org_arns  = var.ami_org_arns
  ami_ou_arns   = var.ami_ou_arns
  ami_users     = var.ami_users
  kms_key_id    = var.kms_key_arn
  encrypt_boot  = var.encrypt_boot

  force_deregister      = var.force_deregister
  force_delete_snapshot = var.force_delete_snapshot

  associate_public_ip_address = var.associate_public_ip_address

  ami_block_device_mappings {
    device_name           = "/dev/xvda"
    volume_size           = var.volume_size
    volume_type           = var.volume_type
    delete_on_termination = true
  }

  assume_role {
    role_arn         = var.assume_role_arn
    session_name     = var.assume_role_session_name
    duration_seconds = var.assume_role_duration_seconds
  }

  aws_polling {
    delay_seconds = 5
    max_attempts  = 100
  }

  tags = var.ami_tags
}

build {
  sources = ["source.amazon-ebs.al2023"]

  provisioner "shell" {
    inline = var.provisioner_shell_commands
  }

  # https://developer.hashicorp.com/packer/tutorials/docker-get-started/docker-get-started-post-processors
  # https://developer.hashicorp.com/packer/docs/post-processors
  # https://developer.hashicorp.com/packer/docs/post-processors/manifest
  post-processor "manifest" {
    output     = var.manifest_file_name
    strip_path = var.manifest_strip_path
  }
}

Configure defaults for the Packer component in the `catalog`

components:
  packer:
    aws/bastion:
      settings:
        packer:
          template: "main.pkr.hcl"
          source_ami: "ami-0013ceeff668b979b"
          source_ami_name: "al2023-ami-2023.7.20250527.1-kernel-6.12-arm64"
          source_ami_description: "Amazon Linux 2023 AMI 2023.7.20250527.1 arm64 HVM kernel-6.12"
          source_ami_owner_account_id: "137112412989"
          region: "us-east-2"
          org_id: "o-xxxxxxxxx"
          org_management_account_id: "xxxxxxxxxxxx"
      metadata:
        component: aws/bastion
      vars:
        # https://masterminds.github.io/sprig/date.html
        ami_name: "bastion-al2023-{{ now | unixEpoch }}"
        source_ami: "{{ .settings.packer.source_ami }}"
        region: "{{ .settings.packer.region }}"
        ami_org_arns:
          - "arn:aws:organizations::{{ .settings.packer.org_management_account_id }}:organization/{{ .settings.packer.org_id }}"
        ami_ou_arns: []
        ami_users: []
        kms_key_arn: null
        encrypt_boot: false
        ssh_username: "ec2-user"
        associate_public_ip_address: true
        volume_type: "gp3"
        skip_create_ami: false
        manifest_file_name: "manifest.json"
        manifest_strip_path: false
        assume_role_session_name: "atmos-packer"
        assume_role_duration_seconds: 1800
        force_deregister: false
        force_delete_snapshot: false
        # SSM Agent is pre-installed on AL2023 AMIs but should be enabled explicitly as done above.
        # `dnf clean all` removes cached metadata and packages to reduce AMI size.
        # `cloud-init clean` ensures the image will boot as a new instance on the next launch.
        provisioner_shell_commands:
          # Enable and start the SSM agent (already installed by default on AL2023)
          - "sudo systemctl enable --now amazon-ssm-agent"
          # Install packages, clean metadata and cloud-init
          - "sudo -E bash -c 'dnf install -y jq && dnf clean all && cloud-init clean'"
          # Install other packages
        ami_tags:
          SourceAMI: "{{ .settings.packer.source_ami }}"
          SourceAMIName: "{{ .settings.packer.source_ami_name }}"
          SourceAMIDescription: "{{ .settings.packer.source_ami_description }}"
          SourceAMIOwnerAccountId: "{{ .settings.packer.source_ami_owner_account_id }}"
          ScanStatus: pending

Define Atmos `nonprod` and `prod` stacks

vars:
  stage: nonprod

import:
  - catalog/aws/bastion/defaults

components:
  packer:
    aws/bastion:
      vars:
        # Define the variables specific to the `nonprod` account
        instance_type: "t4g.small"
        volume_size: 8
        assume_role_arn: "arn:aws:iam::NONPROD_ACCOUNT_ID:role/ROLE_NAME"
        ami_tags:
          Stage: nonprod

vars:
  stage: prod

import:
  - catalog/aws/bastion/defaults

components:
  packer:
    aws/bastion:
      vars:
        # Define the variables specific to the `prod` account
        instance_type: "t4g.medium"
        volume_size: 16
        assume_role_arn: "arn:aws:iam::PROD_ACCOUNT_ID:role/ROLE_NAME"
        ami_tags:
          Stage: prod

Execute Atmos Packer commands

> atmos packer version

Packer v1.14.1

# https://developer.hashicorp.com/packer/docs/commands/validate

> atmos packer validate aws/bastion -s nonprod

The configuration is valid.

# https://developer.hashicorp.com/packer/docs/commands/inspect

> atmos packer inspect aws/bastion -s nonprod

Packer Inspect: HCL2 mode

> input-variables:

var.ami_name: "bastion-al2023-1754457104"
var.ami_org_arns: "[\n  \"arn:aws:organizations::xxxxxxxxxxxx:organization/o-xxxxxxxxx\",\n]"
var.ami_ou_arns: "[]"
var.ami_tags: "{\n  \"ScanStatus\" = \"pending\"\n  \"SourceAMI\" = \"ami-0013ceeff668b979b\"\n  \"SourceAMIDescription\" = \"Amazon Linux 2023 AMI 2023.7.20250527.1 arm64 HVM kernel-6.12\"\n  \"SourceAMIName\" = \"al2023-ami-2023.7.20250527.1-kernel-6.12-arm64\"\n  \"SourceAMIOwnerAccountId\" = \"137112412989\"\n  \"Stage\" = \"nonprod\"\n}"
var.ami_users: "[]"

> local-variables:

> builds:

  > <0>:
    sources:
      amazon-ebs.al2023

    provisioners:
      shell

    post-processors:
      0:
        manifest

# https://developer.hashicorp.com/packer/docs/commands/init

> atmos packer init aws/bastion -s nonprod

Installed plugin github.com/hashicorp/amazon v1.3.9 in "~/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.3.9_x5.0_darwin_arm64"

# https://developer.hashicorp.com/packer/docs/commands/build

> atmos packer build aws/bastion -s nonprod

amazon-ebs.al2023:

==> amazon-ebs.al2023: Prevalidating any provided VPC information
==> amazon-ebs.al2023: Prevalidating AMI Name: bastion-al2023-1754025080
==> amazon-ebs.al2023: Found Image ID: ami-0013ceeff668b979b
==> amazon-ebs.al2023: Setting public IP address to true on instance without a subnet ID
==> amazon-ebs.al2023: No VPC ID provided, Packer will use the default VPC
==> amazon-ebs.al2023: Inferring subnet from the selected VPC "vpc-xxxxxxx"
==> amazon-ebs.al2023: Set subnet as "subnet-xxxxxxx"
==> amazon-ebs.al2023: Creating temporary keypair: packer_688c4c79-f14a-b77e-ca1e-b5b4c17b4581
==> amazon-ebs.al2023: Creating temporary security group for this instance: packer_688c4c7b-3f16-69f9-0c39-88a3fcbe94fd
==> amazon-ebs.al2023: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs.al2023: Launching a source AWS instance...
==> amazon-ebs.al2023: changing public IP address config to true for instance on subnet "subnet-xxxxxxx"
==> amazon-ebs.al2023: Instance ID: i-0b621ca091aa4c240
==> amazon-ebs.al2023: Waiting for instance (i-0b621ca091aa4c240) to become ready...
==> amazon-ebs.al2023: Using SSH communicator to connect: 18.222.63.67
==> amazon-ebs.al2023: Waiting for SSH to become available...
==> amazon-ebs.al2023: Connected to SSH!
==> amazon-ebs.al2023: Provisioning with shell script: /var/folders/rt/fqmt0tmx3fs1qfzbf3qxxq700000gn/T/packer-shell653292668
==> amazon-ebs.al2023: Waiting for process with pid 2085 to finish.
==> amazon-ebs.al2023: Amazon Linux 2023 Kernel Livepatch repository   154 kB/s |  16 kB     00:00
==> amazon-ebs.al2023: Package jq-1.7.1-49.amzn2023.0.2.aarch64 is already installed.
==> amazon-ebs.al2023: Dependencies resolved.
==> amazon-ebs.al2023: Nothing to do.
==> amazon-ebs.al2023: Complete!
==> amazon-ebs.al2023: 17 files removed
==> amazon-ebs.al2023: Stopping the source instance...
==> amazon-ebs.al2023: Stopping instance
==> amazon-ebs.al2023: Waiting for the instance to stop...
==> amazon-ebs.al2023: Creating AMI bastion-al2023-1754025080 from instance i-0b621ca091aa4c240
==> amazon-ebs.al2023: Attaching run tags to AMI...
==> amazon-ebs.al2023: AMI: ami-0b2b3b68aa3c5ada8
==> amazon-ebs.al2023: Waiting for AMI to become ready...
==> amazon-ebs.al2023: Skipping Enable AMI deprecation...
==> amazon-ebs.al2023: Skipping Enable AMI deregistration protection...
==> amazon-ebs.al2023: Modifying attributes on AMI (ami-0b2b3b68aa3c5ada8)...
==> amazon-ebs.al2023: Modifying: ami org arns
==> amazon-ebs.al2023: Modifying attributes on snapshot (snap-09ad35550e1438fb2)...
==> amazon-ebs.al2023: Adding tags to AMI (ami-0b2b3b68aa3c5ada8)...
==> amazon-ebs.al2023: Tagging snapshot: snap-09ad35550e1438fb2
==> amazon-ebs.al2023: Creating AMI tags
==> amazon-ebs.al2023: Adding tag: "Stage": "nonprod"
==> amazon-ebs.al2023: Adding tag: "ScanStatus": "pending"
==> amazon-ebs.al2023: Adding tag: "SourceAMI": "ami-0013ceeff668b979b"
==> amazon-ebs.al2023: Adding tag: "SourceAMIDescription": "Amazon Linux 2023 AMI 2023.7.20250527.1 arm64 HVM kernel-6.12"
==> amazon-ebs.al2023: Adding tag: "SourceAMIName": "al2023-ami-2023.7.20250527.1-kernel-6.12-arm64"
==> amazon-ebs.al2023: Adding tag: "SourceAMIOwnerAccountId": "137112412989"
==> amazon-ebs.al2023: Creating snapshot tags
==> amazon-ebs.al2023: Terminating the source AWS instance...
==> amazon-ebs.al2023: Cleaning up any extra volumes...
==> amazon-ebs.al2023: No volumes to clean up, skipping
==> amazon-ebs.al2023: Deleting temporary security group...
==> amazon-ebs.al2023: Deleting temporary keypair...
==> amazon-ebs.al2023: Running post-processor:  (type manifest)
Build 'amazon-ebs.al2023' finished after 3 minutes 39 seconds.

==> Wait completed after 3 minutes 39 seconds

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs.al2023: AMIs were created:
us-east-2: ami-0b2b3b68aa3c5ada8

--> amazon-ebs.al2023: AMIs were created:
us-east-2: ami-0b2b3b68aa3c5ada8

# `atmos packer output` command is specific to Atmos (Packer itself does not have an `output` command)
# The command is used to get an output from a Packer manifest
# The manifest is generated by Packer when executing a `packer build` command

> atmos packer output aws/bastion -s nonprod

builds:
- artifact_id: us-east-2:ami-0c2ca16b7fcac7529
  build_time: 1.753281956e+09
  builder_type: amazon-ebs
  custom_data: null
  files: null
  name: al2023
  packer_run_uuid: 5114a723-92f6-060f-bae4-3ac2d0324557
- artifact_id: us-east-2:ami-0b2b3b68aa3c5ada8
  build_time: 1.7540253e+09
  builder_type: amazon-ebs
  custom_data: null
  files: null
  name: al2023
  packer_run_uuid: a57874d1-c478-63d7-cfde-9d91e513eb9a
  last_run_uuid: a57874d1-c478-63d7-cfde-9d91e513eb9a

# `atmos packer output` command is specific to Atmos (Packer itself does not have an `output` command)
# The command is used to get an output from a Packer manifest
# The manifest is generated by Packer when executing a `packer build` command

# Use a YQ expression to get a specific section or attribute from the Packer manifest,
# in this case, the `artifact_id` from the first build.

> atmos packer output aws/bastion -s nonprod --query '.builds[0].artifact_id'

us-east-2:ami-0c2ca16b7fcac7529

# `atmos packer output` command is specific to Atmos (Packer itself does not have an `output` command).
# The command is used to get an output from a Packer manifest.
# The manifest is generated by Packer when executing a `packer build` command.

# Use a YQ expression to get a specific section or attribute from the Packer manifest,
# in this case, the AMI (second part after the `:`) from the `artifact_id` from the first build.

> atmos packer output aws/bastion -s nonprod -q '.builds[0].artifact_id | split(":")[1]'

ami-0c2ca16b7fcac7529

internal/exec/stack_processor_utils.go

RoseSecurity · 2025-08-07T21:24:12Z

This is awesome! Excited to test this out!

github-actions · 2025-08-07T23:00:59Z

These changes were released in v1.186.0.

aknysh added 30 commits July 18, 2025 16:04

updates

7b848c7

updates

29cf1ef

updates

465ad8b

updates

33b62ae

updates

bdd7efc

updates

d86d657

updates

08aa0c0

updates

ec97666

updates

1f6c3bf

updates

4f28846

updates

2fd07ab

updates

d825923

updates

4683c39

updates

c606d61

updates

e56a32d

updates

d4f5691

updates

c8db645

updates

49831ec

updates

fa99a37

updates

90a466e

updates

733f496

updates

2498aec

updates

0975994

updates

2cd514d

updates

46f3bf2

updates

709d9a5

updates

06f7b77

updates

41dc2ec

updates

20743a2

updates

cfaf1c4

updates

d93c868

github-actions bot deployed to preview August 7, 2025 05:18 View deployment

updates

5793129

github-actions bot deployed to preview August 7, 2025 14:37 View deployment

updates

ffcac4d

github-actions bot deployed to preview August 7, 2025 15:02 View deployment

updates

f31bab4

github-actions bot deployed to preview August 7, 2025 15:38 View deployment

aknysh and others added 2 commits August 7, 2025 12:12

updates

9ccb10a

[autofix.ci] apply automated fixes

6f2f39e

github-actions bot deployed to preview August 7, 2025 16:16 View deployment

aknysh added 2 commits August 7, 2025 13:04

Merge remote-tracking branch 'origin/main' into add-packer

7dde59e

updates

2e8b52c

github-actions bot deployed to preview August 7, 2025 17:07 View deployment

updates

968b24f

github-actions bot deployed to preview August 7, 2025 17:40 View deployment

updates

9abbb2d

github-actions bot deployed to preview August 7, 2025 18:12 View deployment

updates

d30e826

github-actions bot deployed to preview August 7, 2025 18:43 View deployment

github-advanced-security bot found potential problems Aug 7, 2025

View reviewed changes

internal/exec/stack_processor_utils.go Dismissed Show dismissed Hide dismissed

aknysh and others added 2 commits August 7, 2025 15:35

updates

ac6743c

[autofix.ci] apply automated fixes

11fa612

github-actions bot temporarily deployed to preview August 7, 2025 19:38 Destroyed

osterman approved these changes Aug 7, 2025

View reviewed changes

aknysh merged commit 9179bd8 into main Aug 7, 2025
52 checks passed

aknysh deleted the add-packer branch August 7, 2025 21:42

mergify bot removed the needs-cloudposse Needs Cloud Posse assistance label Aug 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add native Packer support (Atmos loves Packer) #1394

Add native Packer support (Atmos loves Packer) #1394

Uh oh!

aknysh commented Aug 6, 2025 •

edited

Loading

Uh oh!

Uh oh!

RoseSecurity commented Aug 7, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 7, 2025

Uh oh!

Uh oh!

Uh oh!

Add native Packer support (Atmos loves Packer) #1394

Add native Packer support (Atmos loves Packer) #1394

Uh oh!

Conversation

aknysh commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

what

why

description

Configure Packer in atmos.yaml

Add Packer template (Packer component)

Configure defaults for the Packer component in the catalog

Define Atmos nonprod and prod stacks

Execute Atmos Packer commands

Uh oh!

Uh oh!

RoseSecurity commented Aug 7, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 7, 2025

Uh oh!

Uh oh!

aknysh commented Aug 6, 2025 •

edited

Loading

Configure Packer in `atmos.yaml`

Configure defaults for the Packer component in the `catalog`

Define Atmos `nonprod` and `prod` stacks