Backport of Datasource logic cleanup into release/1.9.x

.github/workflows/build.yml

@@ -222,7 +222,7 @@ jobs:
           instructions: |-
             go build -o "$BIN_PATH" -ldflags="$LD_FLAGS" -tags netcgo -trimpath -buildvcs=false
 
-  build-docker:
+  build-docker-light:
     name: Docker light ${{ matrix.arch }} build
     needs:
       - set-product-version
@@ -251,3 +251,31 @@ jobs:
           dev_tags: |
             docker.io/hashicorppreview/${{ env.REPO_NAME }}:${{ env.version }}
             docker.io/hashicorppreview/${{ env.REPO_NAME }}:${{ env.version }}-${{ github.sha }}
+
+  build-docker-full:
+    name: Docker full ${{ matrix.arch }} build
+    needs:
+      - set-product-version
+      - build-linux
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        arch: [ "arm", "arm64", "386", "amd64" ]
+    env:
+      version: ${{ needs.set-product-version.outputs.product-version }}
+    steps:
+      - uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
+      - name: Docker Build (Action)
+        uses: hashicorp/actions-docker-build@v1
+        with:
+          version: ${{ env.version }}
+          target: release-full
+          arch: ${{ matrix.arch }}
+          tags: |
+            docker.io/hashicorp/${{ env.REPO_NAME }}:full
+            docker.io/hashicorp/${{ env.REPO_NAME }}:full-${{ env.version }}
+            public.ecr.aws/hashicorp/${{ env.REPO_NAME }}:full
+            public.ecr.aws/hashicorp/${{ env.REPO_NAME }}:full-${{ env.version }}
+          dev_tags: |
+            docker.io/hashicorppreview/${{ env.REPO_NAME }}:full-${{ env.version }}
+            docker.io/hashicorppreview/${{ env.REPO_NAME }}:full-${{ env.version }}-${{ github.sha }}

.go-version

@@ -1 +1 @@
-1.20.4
+1.20.7

.release/docker/README.md

@@ -1,37 +1,46 @@
 # Packer Docker Container
 
-The root of this repository contains the officially supported HashiCorp Dockerfile to build the hashicorp/packer docker image. The `dev` docker image should be built for local dev and testing, while the production docker image, `release`, is built in CI and makes use of CI-built binaries. The `official` docker image is built using the official binaries from releases.hashicorp.com. 
+The root of this repository contains the officially supported HashiCorp Dockerfile to build the hashicorp/packer docker image. The `dev` docker image should be built for local dev and testing, while the production docker image, `release`, is built in CI and makes use of CI-built binaries. The `light` and `full` docker images are built using the official binaries from releases.hashicorp.com.
 
 ## Build
 
-See the Makefile targets in the root of this repository for building Packer images in either
-development or release modes:
-
-  - `make docker-dev`
-  - `make docker-official`
-  - `make docker-multiarch-official`
-  - `make docker`
+Refer to the Makefile of this repository, especially the `docker` and `docker-dev` targets to build a local version of the dev image based on the sources available.
 
 ### Usage
 
 This repository automatically builds containers for using the
-[`packer`](https://packer.io) command line program. It contains two distinct
-varieties of build, an `official` version, aka `light`, which just contains the binary.
-It also contains a `dev` version, aka `full`, which compiles the binary from source 
+[`packer`](https://developer.hashicorp.com/packer) command line program. It contains three distinct
+varieties of build: a `light` version, which just contains the binary,
+a `full` build, which contains the Packer binary with pre-installed plugins,
+and a `dev` version, which compiles the binary from source
 inside the container before exposing it for use.
 
-##### `official`
+##### `light`
 
-The `official` version of this container will copy the current stable version of
+The `light` version of this container will copy the current stable version of
 the binary, taken from releases.hashicorp.com, into the container. It will also
-set it for use as the default entrypoint. This will be the best option for most uses, 
-especially if you are just looking to run the binary from a container. This image
-is tagged as both `official` and `light`.
+set it for use as the default entrypoint. This will be the best option for most uses,
+especially if you are just looking to run the binary from a container.
 The `latest` tag on DockerHub also points to this version.
 
 You can use this version with the following:
 ```shell
-docker run <args> hashicorp/packer:official <command>
+docker run <args> hashicorp/packer:light <command>
+```
+
+##### `full`
+
+The `full` version of the container builds upon `light` and pre-installs
+the plugins officially maintained by HashiCorp.
+
+You can use this version with the following:
+```shell
+docker run <args> hashicorp/packer:full <command>
+```
+
+You can view the list of pre-installed plugins with the following:
+```shell
+docker run <args> hashicorp/packer:full plugins installed
 ```
 
 ##### `dev`
@@ -41,7 +50,7 @@ the current ref of this [repository](https://github.com/hashicorp/packer). Using
 official `golang` image](https://hub.docker.com/_/golang/) as a base, this
 container will copy the source from the current branch, build the binary, and
 expose it for running. Because all build artifacts are included, it should be quite a bit larger than
-the `official` image. This version of the container is most useful for development or
+the `light` image. This version of the container is most useful for development or
 debugging.
 
 You can use this version with the following:
@@ -64,8 +73,8 @@ docker run \
 
 ~> **Note**: packer init is available from Packer v1.7.0 and later
 
-The command will mount the working directory (`pwd`) to `workspace`, which is the working directory (`-w`) inside the container. 
-Any plugin installed with `packer init` will be installed under the directory specified under the `PACKER_PLUGIN_PATH` environment variable. `PACKER_PLUGIN_PATH` must be set to a path inside the volume mount so that plugins can become available at `packer build`. 
+The command will mount the working directory (`pwd`) to `workspace`, which is the working directory (`-w`) inside the container.
+Any plugin installed with `packer init` will be installed under the directory specified under the `PACKER_PLUGIN_PATH` environment variable. `PACKER_PLUGIN_PATH` must be set to a path inside the volume mount so that plugins can become available at `packer build`.
 
 Running `packer build`
 ```shell

CHANGELOG.md

@@ -1,6 +1,126 @@
-## 1.9.0 (Unreleased)
+## 1.9.5 (Upcoming)
+
+## 1.9.4 (August 18, 2023)
+
+### BUG FIXES:
+
+* core: When invoking Packer with the CHECKPOINT_DISABLE environment variable the telemetry
+    reporter is left uninitialized in order to disable telemetry reporting.
+    Any method calls on the nil reporter is expected to check if the reporter is active or in
+    NOOP mode. The SetBundledUsage function, introduced in Packer 1.9.2, failed to perform a nil
+    check before attempting to modify an attribute, causing Packer to fail when telemetry is
+    disabled. This release fixes this issue by introducing such a check.
+
+## 1.9.3 (August 17, 2023)
+
+### NOTES:
+* **New Docker Image**: As part of the bundled plugin removal effort, a new
+     Docker target called `release-full` has been added to the Packer release
+     artifacts. The release-full image includes Packer and all the official
+     plugins pre-installed in its environment. This image is being offered as an
+     alternative option for those users who may still be relying on the plugin
+     versions bundled into the Packer binary.
+     [GH-12532](https://github.com/hashicorp/packer/pull/12532)
+
+### IMPROVEMENTS:
+* core/docs: Clarify the expected usage of the `packer init` command for HCL2
+     template builds.[GH-12535](https://github.com/hashicorp/packer/pull/12535)
+* core/hcp: Add support for project-level service principals. A user connecting
+     with a project level service principals must provide a valid HCP_PROJECT_ID
+     in order to connect.
+     [GH-12520](https://github.com/hashicorp/packer/pull/12520)
+     [GH-12576](https://github.com/hashicorp/packer/pull/12576)
+* core: A new Docker image `packer:release-full` has been added for all
+     supported architectures. The release-full image includes Packer and all the
+     official plugins pre-installed in its environment.
+     [GH-12532](https://github.com/hashicorp/packer/pull/12532)
+* core: Add enhanced support to Packer telemetry for bundle plugins usage.
+     [GH-12536](https://github.com/hashicorp/packer/pull/12536)
+
+### BUG FIXES:
+
+* core: Bump golang.org/x/net to v0.13.0 to address CVE GO-2023-1988. Packer
+     itself is not vulnerable to the CVE as we don't render web pages, but
+     security checks do point it as an issue that needs to be addressed.
+     [GH-12561](https://github.com/hashicorp/packer/pull/12561)
+* core: Fix custom plugin loading in current working directory regression.
+     [GH-12544](https://github.com/hashicorp/packer/pull/12544)
+
+
+## 1.9.2 (July 19, 2023)
 
 ### NOTES:
+
+* Vendored plugins within Packer have not been updated. Plugin releases occur on
+     a regular basis to address issues and feature requests.
+     Please note that in an upcoming version of Packer, we will remove the last
+     bundled plugins from Packer. Users are encouraged to use `packer init` for HCL2 templates or
+     `packer plugins install` with legacy JSON templates for installing external
+     plugins.
+
+* Packer will now warn when using bundled plugins. This feature will be removed in
+    a future version of the tool, so this warning is meant to bring awareness of the
+    upcoming change, and help users update their templates.
+    [GH-12495](https://github.com/hashicorp/packer/pull/12495)
+
+### BUG FIXES:
+
+* Fixed a bug with how Packer was discovering plugins: in order to load
+     plugins, Packer would recursively scan all the known directories in which
+     we could have installed plugins. This caused unnecessary directory
+     walks and slowdowns upon invocation. Packer will now only check
+     for nested plugins within the directories used by commands such as `packer
+     init`, or `packer plugins install`, or as defined in PACKER_PLUGIN_PATH.
+     Refer to
+     [Packer's plugin directory documentation](https://developer.hashicorp.com/packer/docs/configure#packer-s-plugin-directory)
+     for details on how loading works.
+     [GH-12414](https://github.com/hashicorp/packer/pull/12414)
+
+* The `packer init` subcommand now bundles all the missing installed plugins into one
+    condensed warning, as opposed to one warning per missing plugin.
+    [GH-12506](https://github.com/hashicorp/packer/pull/12506)
+
+### PLUGINS:
+
+* packer-plugin-parallels: The Parallels plugin has been handed over to the Parallels
+     team. New releases for this plugin are available at
+     https://github.com/parallels/packer-plugin-parallels. This plugin is no longer
+     being bundled in the Packer binary release. Existing references to the
+     plugin will continue to work but users are advised to update the
+     `required_plugins` block to use the new plugin source address.
+    [GH-12476](https://github.com/hashicorp/packer/pull/12476)
+    ```
+     required_plugins {
+       parallels = {
+         source  =  "github.com/parallels/parallels"
+         version =  "~> 1"
+        }
+    }
+    ```
+
+### IMPROVEMENTS:
+
+* The `hcl2_upgrade` sub-command will now add `required_plugins` to the template
+    generated from JSON for [official plugins](https://developer.hashicorp.com/packer/plugins#tiers-and-namespaces).
+    [GH-12504](https://github.com/hashicorp/packer/pull/12504)
+
+## 1.9.1 (June 1, 2023)
+
+### BUG FIXES:
+
+* On May 16th 2023, HCP introduced multi-project support to the platform.
+    In order to use multiple projects in your organization, you will need to update Packer
+    to version 1.9.1 or above. Starting with 1.9.1, you may specify a project ID to push
+    builds to with the `HCP_PROJECT_ID` environment variable. If no project ID is specified,
+    Packer will pick the project with the oldest creation date. Older versions of Packer are
+    incompatible with multi-project support on HCP, and builds will fail for HCP
+    organizations with multiple projects on versions before 1.9.1.
+    [GH-12453](https://github.com/hashicorp/packer/pull/12453)
+
+## 1.9.0 (May 31, 2023)
+
+### NOTES:
+
 * **Breaking Change**: Iteration fingerprints used to be computed from the Git SHA of the
      repository where the template is located when running packer build. This
      changes with this release, and now fingerprints are automatically generated
@@ -12,20 +132,60 @@
      these builds will work exactly as they did before.
      [GH-12172](https://github.com/hashicorp/packer/pull/12172)
 
+* **Breaking Change**: Community-maintained plugins bundled with the Packer binary have been removed.
+     These external plugin components are released independently of Packer core and can be installed
+     directly by the user. Users relying on the external plugin components listed below should refer
+     to the `packer plugins` sub-command and, if using HCL2, a `required_plugins` block to define a
+     list of plugins for building a template.
+
 ### PLUGINS
+
 * Remove provisioner plugins for Chef, Converge, Puppet, Salt, and Inspec as
      vendored plugins. These plugins have been previously archived and not
      updated in release since being archived. These plugins can be installed
      using `packer init` or with the Packer plugins sub-command `packer plugins install github.com/hashicorp/chef`.
      [GH-12374](https://github.com/hashicorp/packer/pull/12374)
 
+* The following community plugins won't be bundled with Packer anymore:
+
+    * [Alicloud](https://github.com/hashicorp/packer-plugin-alicloud)
+    * [CloudStack](https://github.com/hashicorp/packer-plugin-cloudstack)
+    * [HCloud](https://github.com/hashicorp/packer-plugin-hcloud)
+    * [HyperOne](https://github.com/hashicorp/packer-plugin-hyperone)
+    * [Hyper-V](https://github.com/hashicorp/packer-plugin-hyperv)
+    * [JDCloud](https://github.com/hashicorp/packer-plugin-jdcloud)
+    * [LXC](https://github.com/hashicorp/packer-plugin-lxc)
+    * [LXD](https://github.com/hashicorp/packer-plugin-lxd)
+    * [NCloud](https://github.com/hashicorp/packer-plugin-ncloud)
+    * [OpenStack](https://github.com/hashicorp/packer-plugin-openstack)
+    * [Proxmox](https://github.com/hashicorp/packer-plugin-proxmox)
+    * [TencentCloud](https://github.com/hashicorp/packer-plugin-tencentcloud)
+    * [Triton](https://github.com/hashicorp/packer-plugin-triton)
+    * [Yandex](https://github.com/hashicorp/packer-plugin-yandex)
+
+    [GH-12436](https://github.com/hashicorp/packer/pull/12436)
+
+Users relying on these external plugin components should refer to the `packer plugins` sub-command and,
+if using HCL2, a `required_plugins` block to define a list of plugins to use for building a template.
+
 ### IMPROVEMENTS:
 
 * core/hcp: Now, fingerprints used by HCP Packer are randomly generated ULIDs
      instead of a Git SHA, and a new one is always generated, unless one is
-     already specified in the environment.
+     specified in the environment.
      [GH-12172](https://github.com/hashicorp/packer/pull/12172)
 
+### BUG FIXES:
+
+* Fix LDFLAGS for release pipelines: Between Packer 1.8.5 and Packer 1.8.7, changes
+    to the LDFLAGS in use for building the binaries for Packer had mistakenly
+    removed some compilation flags, leading to the final binaries not being stripped.
+    This change raised the size of the built binaries by as much as 45%.
+    In this release, we fixed the LDFLAGS during compilation, yielding leaner binaries.
+
+* Bumped gopsutil to v3. This fixes a macOS intermittent crash reported by the community
+    [GH-12430](https://github.com/hashicorp/packer/issues/12430)
+
 ## 1.8.7 (May 4, 2023)
 
 ### NOTES: