Network dropped connection on reset error for running go binary from mapped volume on container on Apple Silicon Mac #14142
Description
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
Running a go binary built from container via podman, but running it from a mapped path (using path on Mac host, mapped into container) results in Network dropped connection on reset error with application aborting what it was supposed to do.
Did some online search regarding this error for golang and for podman and haven't found anything useful. I did notice in search results the error is a generic platform/framework error message mapped into the golang language, but am unclear when or how this gets triggered, and why it would trigger from podman container when running the go code from mapped path.
Hoping the community here can provide some insight. I used a debian based container image, not sure if actual container image used really matters here. For reference used debian:bullseye-slim as base image and installed go tooling on top of it.
It seems I have read, write, execute access to mapped path in container, but executing the compiled go code from that path just errors out.
If not a bug, or not a podman bug, wondering if it is some podman machine vs container configuration issue where something isn't configured correctly to result in this kind of problem. Unfortunately the error message is vague and online search doesn't yield much additional information.
Haven't tried hooking up a debugger/profiler yet to see if that can help trace the problem.
Steps to reproduce the issue:
-
podman run --rm -it -v /mac_host_path:/data debian_based_image_with_golang_set_up bash -
compile some go code into binary from that container, binary is written & stored in within
mac_host_paththat is mapped into container at/dataThe go code could be a simply print of "hello world", e.g. https://gobyexample.com/hello-world. -
Execute go-based binary from the mapped path. For comparison, can then copy the go-based binary to a local path on the container and execute. Can also execute a custom shell script from the mapped path to compare as well. Can also compare with running go code from source e.g.
go run main.go, in the mapped path.
Describe the results you received:
Running binary from mapped path results in error Network dropped connection on reset. Running binary from local path works fine. Running shell script from mapped path works fine. Running the uncompiled go code with go run in the mapped path works without error.
Describe the results you expected:
Running binary from mapped path should run normally without error.
Additional information you deem important (e.g. issue happens only occasionally):
Output of podman version:
Client: Podman Engine
Version: 4.0.3
API Version: 4.0.3
Go Version: go1.18
Built: Fri Apr 1 08:28:59 2022
OS/Arch: darwin/arm64
Server: Podman Engine
Version: 4.0.3
API Version: 4.0.3
Go Version: go1.18
Built: Fri Apr 1 11:22:39 2022
OS/Arch: linux/arm64
Output of podman info --debug:
host:
arch: arm64
buildahVersion: 1.24.3
cgroupControllers:
- cpu
- io
- memory
- pids
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: conmon-2.1.0-2.fc36.aarch64
path: /usr/bin/conmon
version: 'conmon version 2.1.0, commit: '
cpus: 1
distribution:
distribution: fedora
variant: coreos
version: "36"
eventLogger: journald
hostname: localhost.localdomain
idMappings:
gidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 100000
size: 1000000
uidmap:
- container_id: 0
host_id: 502
size: 1
- container_id: 1
host_id: 100000
size: 1000000
kernel: 5.17.3-300.fc36.aarch64
linkmode: dynamic
logDriver: journald
memFree: 604237824
memTotal: 2052624384
networkBackend: netavark
ociRuntime:
name: crun
package: crun-1.4.4-1.fc36.aarch64
path: /usr/bin/crun
version: |-
crun version 1.4.4
commit: 6521fcc5806f20f6187eb933f9f45130c86da230
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
os: linux
remoteSocket:
exists: true
path: /run/user/502/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: true
serviceIsRemote: true
slirp4netns:
executable: /usr/bin/slirp4netns
package: slirp4netns-1.2.0-0.2.beta.0.fc36.aarch64
version: |-
slirp4netns version 1.2.0-beta.0
commit: 477db14a24ff1a3de3a705e51ca2c4c1fe3dda64
libslirp: 4.6.1
SLIRP_CONFIG_VERSION_MAX: 3
libseccomp: 2.5.3
swapFree: 0
swapTotal: 0
uptime: 2h 21m 49.07s (Approximately 0.08 days)
plugins:
log:
- k8s-file
- none
- passthrough
- journald
network:
- bridge
- macvlan
volume:
- local
registries:
search:
- docker.io
store:
configFile: /var/home/core/.config/containers/storage.conf
containerStore:
number: 2
paused: 0
running: 2
stopped: 0
graphDriverName: overlay
graphOptions: {}
graphRoot: /var/home/core/.local/share/containers/storage
graphStatus:
Backing Filesystem: xfs
Native Overlay Diff: "true"
Supports d_type: "true"
Using metacopy: "false"
imageCopyTmpDir: /var/tmp
imageStore:
number: 70
runRoot: /run/user/502/containers
volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
APIVersion: 4.0.3
Built: 1648837359
BuiltTime: Fri Apr 1 11:22:39 2022
GitCommit: ""
GoVersion: go1.18
OsArch: linux/arm64
Version: 4.0.3
Package info (e.g. output of rpm -q podman or apt list podman):
$ brew list podman
/opt/homebrew/Cellar/podman/4.0.3/bin/podman
/opt/homebrew/Cellar/podman/4.0.3/bin/podman-mac-helper
/opt/homebrew/Cellar/podman/4.0.3/bin/podman-remote
/opt/homebrew/Cellar/podman/4.0.3/etc/bash_completion.d/podman
/opt/homebrew/Cellar/podman/4.0.3/libexec/gvproxy
/opt/homebrew/Cellar/podman/4.0.3/share/fish/vendor_completions.d/podman.fish
/opt/homebrew/Cellar/podman/4.0.3/share/man/ (161 files)
/opt/homebrew/Cellar/podman/4.0.3/share/zsh/site-functions/_podman
Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)
Yes, I think this is the latest version? Installed from brew. Yes looked over the troubleshooting guide.
Additional environment details (AWS, VirtualBox, physical, etc.):
Default install of podman for Mac OS (Apple silicon) & using default machine image, default settings, with exception of adding -v $HOME:$HOME although not sure if that was necessary in latest podman.