Assuming you're using the Prometheus Operator in the Management Cluster, metrics can be scraped via Service Monitors like in this case:

apiVersion: v1
kind: Service
metadata:
  labels:
    kamaji.clastix.io/name: charlie-metrics
  name: charlie-metrics
  namespace: default
spec:
  ports:
  - name: kube-apiserver-metrics
    port: 6443
    protocol: TCP
    targetPort: 6443
  - name: kube-controller-manager-metrics
    port: 10257
    protocol: TCP
    targetPort: 10257
  - name: kube-scheduler-metrics
    port: 10259
    protocol: TCP
    targetPort: 10259
  selector:
    kamaji.clastix.io/name: charlie
  type: ClusterIP

---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    release: kube-prometheus-stack
  name: kube-prometheus-stack-tcp-charlie
  namespace: default
spec:
  endpoints:
  # API Server endpoint
  - port: kube-apiserver-metrics
    scheme: https
    path: /metrics
    interval: 15s
    scrapeTimeout: 10s
    tlsConfig:
      # skip certificate verification
      insecureSkipVerify: true
      # Client certificate for authentication
      cert:
        secret:
          name: charlie-api-server-kubelet-client-certificate
          key: apiserver-kubelet-client.crt
      # Client key for authentication
      keySecret:
        name: charlie-api-server-kubelet-client-certificate
        key: apiserver-kubelet-client.key
    metricRelabelings:
    - action: drop
      regex: apiserver_request_duration_seconds_bucket;(0.15|0.2|0.3|0.35|0.4|0.45|0.6|0.7|0.8|0.9|1.25|1.5|1.75|2|3|3.5|4|4.5|6|7|8|9|15|25|40|50)
      sourceLabels:
      - __name__
      - le
    relabelings:
    - action: replace
      targetLabel: cluster
      replacement: charlie
    - action: replace
      targetLabel: job
      replacement: apiserver
  # Controller Manager endpoint
  - port: kube-controller-manager-metrics
    scheme: https
    path: /metrics
    interval: 15s
    scrapeTimeout: 10s
    tlsConfig:
      # skip certificate verification
      insecureSkipVerify: true
      # Client certificate for authentication
      cert:
        secret:
          name: charlie-api-server-kubelet-client-certificate
          key: apiserver-kubelet-client.crt
      # Client key for authentication
      keySecret:
        name: charlie-api-server-kubelet-client-certificate
        key: apiserver-kubelet-client.key
    relabelings:
    - action: replace
      targetLabel: cluster
      replacement: charlie
    - action: replace
      targetLabel: job
      replacement: kube-controller-manager
  # Scheduler endpoint
  - port: kube-scheduler-metrics
    scheme: https
    path: /metrics
    interval: 15s
    scrapeTimeout: 10s
    tlsConfig:
      # skip certificate verification
      insecureSkipVerify: true
      # Client certificate for authentication
      cert:
        secret:
          name: charlie-api-server-kubelet-client-certificate
          key: apiserver-kubelet-client.crt
      # Client key for authentication
      keySecret:
        name: charlie-api-server-kubelet-client-certificate
        key: apiserver-kubelet-client.key
    relabelings:
    - action: replace
      targetLabel: cluster
      replacement: charlie
    - action: replace
      targetLabel: job
      replacement: kube-scheduler
  selector:
    matchLabels:
      kamaji.clastix.io/name: charlie-metrics
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: prometheus-secret-access
  namespace: default
subjects:
- kind: ServiceAccount
  name: kube-prometheus-stack-prometheus
  namespace: monitoring-system
roleRef:
  kind: Role
  name: prometheus-secret-reader
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: prometheus-secret-reader
  namespace: default
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch"]

Provide Prometheus metrics for monitoring/alerting and related Grafana dashboards #28

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions