Local output omits platform specific folder

[ethrgeist]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Hi,

if there are multiple platforms defined, e.g.

  platforms = [
    "linux/amd64",
    "linux/arm64",
  ]

and output local is used

output = ["type=local,dest=dist"]

then the dist folder looks like this:

dist
├── linux_amd64
           │  (folders with build artifacts)
├── linux_arm64
           │  (folders with build artifacts)
...

Expected behaviour

Expected behavior would be to find the arch specific folder even where there is only a single platform to have predictable paths no matter how many platforms are enabled for the build.

dist
├── linux_amd64
           │  (folders with build artifacts)

Actual behaviour

If there is only 1 platform

  platforms = [
    "linux/amd64",
  ]

the dist folder looks like this, the arch specific folder is omitted

dist
├── (folder with build artifacts)

Buildx version

github.com/docker/buildx v0.23.0-desktop.1 503f948aadbddb6de3ec5581f766e1d27f6975a1

Docker info

Client:
 Version:    28.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v1.1.7
    Path:     /<snip>/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.23.0-desktop.1
    Path:     /<snip>/.docker/cli-plugins/docker-buildx
  cloud: Docker Cloud (Docker Inc.)
    Version:  v0.3.0
    Path:     /<snip>/.docker/cli-plugins/docker-cloud
  compose: Docker Compose (Docker Inc.)
    Version:  v2.36.0-desktop.1
    Path:     /<snip>/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.38
    Path:     /<snip>/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Docker Inc.)
    Version:  v0.1.8
    Path:     /<snip>/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /<snip>/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /<snip>/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /<snip>/.docker/cli-plugins/docker-init
  mcp: Docker MCP Plugin (Docker Inc.)
    Version:  dev
    Path:     /<snip>/.docker/cli-plugins/docker-mcp
  model: Docker Model Runner (Docker Inc.)
    Version:  v0.1.23
    Path:     /<snip>/.docker/cli-plugins/docker-model
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /<snip>/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.18.0
    Path:     /<snip>/.docker/cli-plugins/docker-scout
WARNING: Plugin "/<snip>/.docker/cli-plugins/docker-feedback" is not valid: failed to fetch metadata: fork/exec /<snip>/.docker/cli-plugins/docker-feedback: no such file or directory

Server:
 Containers: 4
  Running: 0
  Paused: 0
  Stopped: 4
 Images: 14
 Server Version: 28.1.1
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.14-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 12
 Total Memory: 7.653GiB
 Name: docker-desktop
 ID: 16d8175c-9303-4345-8ea3-a382a33e46ce
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///<snip>/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW is set
WARNING: daemon is not using the default seccomp profile

Builders list

NAME/NODE           DRIVER/ENDPOINT     STATUS    BUILDKIT   PLATFORMS
default             docker                                   
 \_ default          \_ default         running   v0.21.0    linux/amd64 (+2), linux/arm64, linux/ppc64le, linux/s390x, (2 more)
desktop-linux*      docker                                   
 \_ desktop-linux    \_ desktop-linux   running   v0.21.0    linux/amd64 (+2), linux/arm64, linux/ppc64le, linux/s390x, (2 more)

Configuration

target "binaries" {
  target     = "binary"
  platforms = [
    # "linux/386",
    "linux/amd64",
    # "linux/arm64",
  ]
  output = ["type=local,dest=dist"]
}

Build logs

Additional info

No response

[crazy-max]

You can use the platform-split attribute: https://github.com/moby/buildkit?tab=readme-ov-file#local-directory

target "binaries" {
  target     = "binary"
  platforms = [
    # "linux/386",
    "linux/amd64",
    # "linux/arm64",
  ]
  output = ["type=local,dest=dist,platform-split=false"]
}

I see this attribute is not documented on docs website: https://docs.docker.com/build/exporters/local-tar/ (cc @ArthurFlag)

[ethrgeist]

Thanks for the quick response, i ran a quick local test and the parameter seems to not do anything (either true or false), the arch specific folder still gets omited if there is only 1 platform.

And i would make a case that the plattform specific folder should be the default behavior to have consistent paths independend of the amount of different platforms.

e: Reading https://github.com/moby/buildkit/blob/master/README.md#local-directory

This looks like it would do the opposite of what i need:

to merge files from all platforms together into same directory

[crazy-max]

This looks like it would do the opposite of what i need:

Oh my bad I got confused. And indeed if not a multiplatform build we don't create a platform dir: https://github.com/moby/buildkit/blob/ab80544b39204300712c4415a0f8722ceee033c3/exporter/local/export.go#L151

But for your use case I still think platform-split=false is what you need if you want a predictable path. Although when doing a multiplatform build you need to make sure outputted files are unique: https://github.com/moby/buildkit/blob/ab80544b39204300712c4415a0f8722ceee033c3/exporter/local/export.go#L140

[ethrgeist]

And indeed if not a multiplatform build we don't create a platform dir: https://github.com/moby/buildkit/blob/ab80544b39204300712c4415a0f8722ceee033c3/exporter/local/export.go#L151

Okay, but that makes CICD scripting way harder since paths would depend on settings in the HCL file that are bound to change over time, maybe this behavior could change or atleast get a parameter that always forces the platform directory?

But for your use case I still think platform-split=false is what you need if you want a predictable path

I understand and i tested this parameter with false and true, in both cases it does not change the output structure within the local dist directory.

Although when doing a multiplatform build you need to make sure outputted files are unique

Always having the platform specific folder would solve this without touching artifact chains.

[ethrgeist]

I played around with platform-split=false again and i was able to achive the behavior of "forced" platform specific folders in the dist output, so i can atleast confirm this setting works. I had to change compiler output paths for that but it works. So adding a bit of documentation might help others.

Since $TARGETPLATFORM uses a / as seperator there is a lot of parsing and tr involved, so i'd still would prefer if i could achive this behavior more direct with buildx at a "higher" level and not have to touch project specific artifact chains.

[tonistiigi]

It looks like indeed platform-split=true does not work to reverse the default behavior. You can define --build-arg BUILDKIT_MULTI_PLATFORM=true to force multi-platform output with separators, documented in https://docs.docker.com/reference/dockerfile/#buildkit-built-in-build-args