Skip to content

[fix included] dockerd[753]: http: panic serving @: runtime error: invalid memory address or nil pointer dereference #43034

New issue
New issue
Open
Open
[fix included] dockerd[753]: http: panic serving @: runtime error: invalid memory address or nil pointer dereference#43034
Labels
area/networkingkind/bugBugs are bugs. The cause may or may not be known at triage time so debugging may be needed.version/20.10
@CRCinAU

Description

@CRCinAU
CRCinAU
opened on Nov 19, 2021

Description

Describe the results you received:

Nov 19 02:01:03  systemd[1]: Started Start docker-compose on boot.
Nov 19 02:01:04  docker-compose[3379]: Creating network "docker_default" with the default driver
Nov 19 02:01:04  dockerd[753]: http: panic serving @: runtime error: invalid memory address or nil pointer dereference
                                      goroutine 794 [running]:
                                      net/http.(*conn).serve.func1(0x4000bff2c0)
                                              /usr/local/go/src/net/http/server.go:1804 +0x108
                                      panic(0x5590289c40, 0x559158a3e0)
                                              /usr/local/go/src/runtime/panic.go:971 +0x3f4
                                      github.com/docker/docker/vendor/github.com/vishvananda/netlink.parseAddr(0x4000276864, 0x40, 0x40, 0x0, 0x40015a46ec, 0x4, 0x280, 0x0, 0x0, 0x4000276878, ...)
                                              /go/src/github.com/docker/docker/vendor/github.com/vishvananda/netlink/addr_linux.go:274 +0x174
                                      github.com/docker/docker/vendor/github.com/vishvananda/netlink.(*Handle).AddrList(0x40008b8260, 0x55906788f0, 0x4000e25d40, 0x2, 0x0, 0x0, 0x400013c8d0, 0x0, 0x4000de87e8)
                                              /go/src/github.com/docker/docker/vendor/github.com/vishvananda/netlink/addr_linux.go:199 +0x1a0
                                      github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.(*bridgeInterface).addresses(0x40006471a0, 0x0, 0x5590647cd8, 0x4000de0a30, 0x31, 0x40010c7140, 0x0, 0x40015a45f0, 0x4000de86b8)
                                              /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/interface.go:57 +0x44
                                      github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.setupBridgeIPv4(0x4000c7eea0, 0x40006471a0, 0x0, 0x0)
                                              /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/setup_ipv4.go:31 +0xac
                                      github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.(*bridgeSetup).apply(0x4000de8928, 0x4000eb0c60, 0x4)
                                              /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/setup.go:17 +0x74
                                      github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.(*driver).createNetwork(0x40003f2280, 0x4000c7eea0, 0x0, 0x0)
                                              /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/bridge.go:809 +0x6a8
                                      github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.(*driver).CreateNetwork(0x40003f2280, 0x40010c6f40, 0x40, 0x4000eeb7d0, 0x55906775e0, 0x4000961180, 0x4000eebef0, 0x1, 0x1, 0x5591667850, ...)
                                              /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/bridge.go:648 +0x3fc
                                      github.com/docker/docker/vendor/github.com/docker/libnetwork.(*controller).addNetwork(0x400090e100, 0x4000961180, 0x0, 0x40003f2280)
                                              /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/controller.go:1011 +0x100
                                      github.com/docker/docker/vendor/github.com/docker/libnetwork.(*controller).NewNetwork(0x400090e100, 0x558fcb2091, 0x6, 0x40015a4100, 0xe, 0x40010c6f40, 0x40, 0x4000eeb6e0, 0x6, 0x6, ...)
                                              /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/controller.go:828 +0x92c
                                      github.com/docker/docker/daemon.(*Daemon).createNetwork(0x4000cb2000, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x100, 0x0, ...)
                                              /go/src/github.com/docker/docker/daemon/network.go:365 +0x454
                                      github.com/docker/docker/daemon.(*Daemon).CreateNetwork(0x4000cb2000, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x100, 0x0, ...)
                                              /go/src/github.com/docker/docker/daemon/network.go:286 +0x54
                                      github.com/docker/docker/api/server/router/network.(*networkRouter).postNetworkCreate(0x4000141580, 0x559069a4b8, 0x4000eea840, 0x559068b530, 0x4000489420, 0x4000b03400, 0x4000eea720, 0x55901c1660, 0x1)
                                              /go/src/github.com/docker/docker/api/server/router/network/network_routes.go:229 +0x2a8
                                      github.com/docker/docker/api/server/middleware.ExperimentalMiddleware.WrapHandler.func1(0x559069a4b8, 0x4000eea840, 0x559068b530, 0x4000489420, 0x4000b03400, 0x4000eea720, 0x559069a4b8, 0x4000eea840)
                                              /go/src/github.com/docker/docker/api/server/middleware/experimental.go:26 +0x140
                                      github.com/docker/docker/api/server/middleware.VersionMiddleware.WrapHandler.func1(0x559069a4b8, 0x4000eea810, 0x559068b530, 0x4000489420, 0x4000b03400, 0x4000eea720, 0x7f8439eb00, 0x40)
                                              /go/src/github.com/docker/docker/api/server/middleware/version.go:62 +0x4cc
                                      github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1(0x559069a4b8, 0x4000eea810, 0x559068b530, 0x4000489420, 0x4000b03400, 0x4000eea720, 0x559069a4b8, 0x4000eea810)
                                              /go/src/github.com/docker/docker/pkg/authorization/middleware.go:59 +0x5e4
                                      github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1(0x559068b530, 0x4000489420, 0x4000b03300)
                                              /go/src/github.com/docker/docker/api/server/server.go:141 +0x1bc
                                      net/http.HandlerFunc.ServeHTTP(0x400000f158, 0x559068b530, 0x4000489420, 0x4000b03300)
                                              /usr/local/go/src/net/http/server.go:2049 +0x40
                                      github.com/docker/docker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0x4000dc8e40, 0x559068b530, 0x4000489420, 0x4000b03100)
                                              /go/src/github.com/docker/docker/vendor/github.com/gorilla/mux/mux.go:210 +0x9c
                                      net/http.serverHandler.ServeHTTP(0x40002782a0, 0x559068b530, 0x4000489420, 0x4000b03100)
                                              /usr/local/go/src/net/http/server.go:2867 +0xbc
                                      net/http.(*conn).serve(0x4000bff2c0, 0x559069a4b8, 0x4000814540)
                                              /usr/local/go/src/net/http/server.go:1932 +0x71c
                                      created by net/http.(*Server).Serve
                                              /usr/local/go/src/net/http/server.go:2993 +0x308
Nov 19 02:01:04  docker-compose[3379]: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?
Nov 19 02:01:04  docker-compose[3379]: If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
Nov 19 02:01:04  systemd-udevd[3393]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Nov 19 02:01:04  NetworkManager[551]: <info>  [1637287264.6040] manager: (br-f060ca033c96): new Bridge device (/org/freedesktop/NetworkManager/Devices/34)
Nov 19 02:01:04  NetworkManager[551]: <info>  [1637287264.6199] devices added (path: /sys/devices/virtual/net/br-f060ca033c96, iface: br-f060ca033c96)
Nov 19 02:01:04  NetworkManager[551]: <info>  [1637287264.6199] device added (path: /sys/devices/virtual/net/br-f060ca033c96, iface: br-f060ca033c96): no ifupdown configuration found.
Nov 19 02:01:04  systemd[1]: docker-compose.service: Main process exited, code=exited, status=1/FAILURE
Nov 19 02:01:04  systemd[1]: docker-compose.service: Failed with result 'exit-code'.

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):
This seems to be an ongoing issue - and we can only get things running by rebooting the machine... Trying to do a systemctl restart docker will continue to crash.

Docker is installed on arm via:

wget -qO - https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
     $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-install docker-ce docker-ce-cli containerd.io docker-compose

Output of docker version:

Client: Docker Engine - Community
 Version:           20.10.10
 API version:       1.41
 Go version:        go1.16.9
 Git commit:        b485636
 Built:             Mon Oct 25 07:42:07 2021
 OS/Arch:           linux/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.10
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.9
  Git commit:       e2f740d
  Built:            Mon Oct 25 07:40:42 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.11
  GitCommit:        5b46e404f6b9f661a205e28d59c982d3634148f8
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

# docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 20.10.10
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.4.194-g50f0a60241aa-dirty
 Operating System: Ubuntu 18.04.6 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 1.912GiB
 Name: faceway
 ID: 3U46:J6HX:SMDX:7QBL:URFY:56YQ:NRGD:4HY3:AY5Z:BXVR:RRL6:COKH
 Docker Root Dir: /userdata/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No kernel memory TCP limit support

Additional environment details (AWS, VirtualBox, physical, etc.):

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/networkingkind/bugBugs are bugs. The cause may or may not be known at triage time so debugging may be needed.version/20.10

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions