Skip to content

regression - Panic on jaeger-query 1.2.0 startup using expvar metrics and cassandra - worked in 1.1.0 #716

@the-cloud-source

Description

@the-cloud-source

Since 1.2.0 query won't start if using expvar backend, it panics with:

{"level":"info","ts":1519836326.7820027,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":16687,"status":"unavailable"}
2018/02/28 16:45:26 Reuse of exported var name: jaeger-query.ServiceNames.attempts
panic: Reuse of exported var name: jaeger-query.ServiceNames.attempts


goroutine 1 [running]:
log.Panicln(0xc4203327e0, 0x2, 0x2)
        /usr/local/go/src/log/log.go:340 +0xc0
expvar.Publish(0xc4203fb320, 0x22, 0x106b160, 0xc4204236e0)
        /usr/local/go/src/expvar/expvar.go:261 +0x2ee
expvar.NewFloat(0xc4203fb320, 0x22, 0xc420176a20)
        /usr/local/go/src/expvar/expvar.go:287 +0x60
github.com/jaegertracing/jaeger/vendor/github.com/go-kit/kit/metrics/expvar.NewCounter(0xc4203fb320, 0x22, 0xc4203fb320)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/vendor/github.com/go-kit/kit/metrics/expvar/expvar.go:23 +0x39
github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics/go-kit/expvar.factory.Counter(0xa, 0xc4203fb320, 0x22, 0xc4204ba390, 0xa)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics/go-kit/expvar/factory.go:37 +0x35
github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics/go-kit/expvar.(*factory).Counter(0xc420015d68, 0xc4203fb320, 0x22, 0xc4204ba360, 0xc4203fb320)
        <autogenerated>:1 +0x50
github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics/go-kit.(*factory).Counter(0xc420176a20, 0xa983cb, 0x8, 0xc4204ba360, 0x0, 0x0)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics/go-kit/factory.go:108 +0xa9
github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics.initMetrics(0xb230a0, 0xc420208640, 0x1075760, 0xc420176a20, 0x0, 0x19, 0xc4204041c0)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics/metrics.go:72 +0x447
github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics.Init(0xb230a0, 0xc420208640, 0x1075760, 0xc420176a20, 0x0)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/vendor/github.com/uber/jaeger-lib/metrics/metrics.go:25 +0x57
github.com/jaegertracing/jaeger/pkg/cassandra/metrics.NewTable(0x1075760, 0xc4200510e0, 0xc22ab3, 0xc, 0xc1c7a8)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/pkg/cassandra/metrics/table.go:36 +0xa3
github.com/jaegertracing/jaeger/plugin/storage/cassandra/spanstore.NewServiceNamesStorage(0x1072620, 0xc420146c80, 0x0, 0x1075760, 0xc4200510e0, 0xc420050cc0, 0x0)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/plugin/storage/cassandra/spanstore/service_names.go:56 +0x54
github.com/jaegertracing/jaeger/plugin/storage/cassandra/spanstore.NewSpanReader(0x1072620, 0xc420146c80, 0x1075760, 0xc4200510e0, 0xc420050cc0, 0xb42620)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/plugin/storage/cassandra/spanstore/reader.go:117 +0xc9
github.com/jaegertracing/jaeger/plugin/storage/cassandra.(*Factory).CreateSpanReader(0xc420050b40, 0xc42013d4a0, 0xc20ec5, 0x9, 0xc420089288)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/plugin/storage/cassandra/factory.go:94 +0x53
github.com/jaegertracing/jaeger/plugin/storage.(*Factory).CreateSpanReader(0xc42013d440, 0xc420332fa0, 0x410bb6, 0xc420176960, 0x60)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/plugin/storage/factory.go:96 +0x6f
main.archiveOptions(0x10754e0, 0xc42013d440, 0xc420050cc0, 0x0, 0x0, 0x0)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/cmd/query/main.go:184 +0x4c
main.main.func1(0xc4200878c0, 0xc42014f4c0, 0x0, 0x2, 0x0, 0x0)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/cmd/query/main.go:116 +0xf69
github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).execute(0xc4200878c0, 0xc42000e0a0, 0x2, 0x2, 0xc4200878c0, 0xc42000e0a0)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:698 +0x47a
github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc4200878c0, 0x0, 0x4, 0xc420050ba0)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:783 +0x30e
github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).Execute(0xc4200878c0, 0xc4200878c0, 0xc420109f48)
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:736 +0x2b
main.main()
        /var/cvs/open-tracing/src/github.com/jaegertracing/jaeger/cmd/query/main.go:165 +0x70f

config file used cfg.json

{
  "log-level":          "info",
  "metrics-backend":    "expvar",
  "metrics-http-route": "/expvar",
  "span-storage":       {"type": "cassandra"},
  "dependency-storage": {"data-frequency" : "24h0m0s"},

  "query": {
    "port":                   16686,
    "health-check-http-port": 16687,
    "prefix":                 "api",
    "static-files":           "/var/lib/jaeger/web",
    "ui-config":              ""
  },

  "cassandra": {
    "keyspace": "jaeger_v1_dc",
    "max-retry-attempts": 3,
    "connections-per-host": 2,
    "proto": 4,
    "servers": "cassandra01,cassandra02,cassandra03",
    "port": 9042,
    "socket-keep-alive": "10s",
    "timeout": "0s"
  },
  "cassandra-archive": {
    "enabled": false
  }
}

panic goes if not using expvar metrics

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions