What did you do?

Ran Prometheus 3.1 and ran workload i.e. otel collector contrib docker run otel/opentelemetry-collector-contrib:0.118.0 (older version 0.117.0, don't have this behavior)
Try to scrape metrics (about the collector itself) from the otel collector contrib endpoint.

What did you expect to see?

Successful scrape of the metrics of the otel collector contrib instance, with the dots translated to underscores.

What did you see instead? Under which circumstances?

Scrape being rejected because prometheus still negotiated dots in labels, but validation after rejecting the scrape.

System information

Linux 5.15.167.4-microsoft-standard-WSL2 x86_6

Prometheus version

prometheus, version 3.1.0 (branch: HEAD, revision: 7086161a93b262aa0949dbf2aba15a5a7b13e0a3)
  build user:       root@74c225e2044f
  build date:       20250102-13:52:43
  go version:       go1.23.4
  platform:         linux/amd64
  tags:             netgo,builtinassets,stringlabels

Prometheus configuration file

global:
  evaluation_interval: 30s
  scrape_interval: 5s
  metric_name_validation_scheme: legacy
scrape_configs:
  - job_name: otel-collector
    scrape_interval: 5s
    static_configs:
      - targets: [ otel-instancel:8888 ]

Suspected bug in https://github.com/prometheus/prometheus/blob/main/scrape/scrape.go#L780