Skip to content

CFP: Convert Hubble into a hive/cell based architecture #34501

@chancez

Description

@chancez

Cilium Feature Proposal

Thanks for taking time to make a feature proposal for Cilium! If you have usage questions, please try the slack channel and see the FAQ first.

Is your proposed feature related to a problem?

  • It's hard to integrate with other Cilium sub-systems and features that are provided through the cell/hive framework
  • launchHubble is huge and fails when any component is misconfigured (eg: metrics). This makes Hubble less reliable.
  • It's hard to debug why Hubble is not working, and the hive framework provides easier mechanisms to report health/status information

Describe the feature you'd like

Re-architect hubble to use hive/cells

(Optional) Describe your proposed solution

Break-up Hubble components incrementally into cells.

---
title: launchHubble()
---
  graph TD;
      monEventsFilter["monitor events filter"]-->|OnMonitorEvent|observer;

      k8sClientset((k8sClient.Clientset))-->k8sDrop;
      k8sWatcher((watchers.K8sWatcher))-->k8sDrop;
      k8sDrop["k8s drop events notifier"]-->|OnDecodedFlowFunc|observer;

      localNodeStore((node.LocalNodeStore))-->localNodeWatcher;
      localNodeWatcher["local node watcher"]-->|OnDecodedFlow|observer;

      gRPCmetrics["gRPC Metrics"]-->Metrics;
      gRPCmetrics["gRPC Metrics"]-->|serveroption.WithGRPC|localServer;
      Metrics{{Metrics}}-->metricsServer;
      metricsTLS["Metrics TLS"]-->metricsServer;
      Metrics-->|OnDecodedFlowFunc|observer;
      metricsServer["Metrics server"];

      redact["l7 redact"]-->|parserOpts|parsers;
      epManager((endpointmanager.EndpointManager))-->parsers;
      idAllocator((identitycell.CachingIdentityAllocator))-->parsers;
      IPCache((ipcache.IPCache))-->parsers;
      svcManager((service.ServiceManager))-->parsers;
      linkCache((link.LinkCache))-->|"link.NewLinkCache()"|parsers;
      cgroupManager((manager.CGroupManager))-->parsers;
      parsers-->observer;
      
      Exporter-->|OnDecodedEvent|observer;
      dynamicExporter["Dynamic Exporter"]-->|OnDecodedEvent|observer;
     
      nsManager["Namespace Manager"]-->observer;
      monAgent((monitorAgent.Agent))-->observer;
      observer["Observer service"]-->localServer;
      observer-->tcpServer;

      nodeManager((nodeManager.NodeManager))-->peerSvc;
      peerSvc["Peer service"]-->localServer;
      peerSvc-->tcpServer;

      recorder((recorder.Recorder))-->recSvc;
      monAgent-->recSvc;
      recSvc["Recorder service"]-->localServer;

      serverTLS["Server TLS"]-->tcpServer;
      tcpServer["TCP Server"];

style monEventsFilter fill:#FFD5E4
style k8sDrop fill:#FFD5E4
style localNodeWatcher fill:#FFD5E4
style nsManager fill:#FFD5E4
style observer fill:#FFD5E4
style parsers fill:#FFD5E4,stroke-width:2px,stroke-dasharray: 5 5
style peerSvc fill:#FFD5E4
style recSvc fill:#FFD5E4
style tcpServer fill:#FFD5E4
style localServer fill:#FFD5E4
Loading

In the diagram above, rectangle are hubble components. The pink colored one are yet to be converted to cells and the neutral blue ones already done:

  1. Refactor Hubble as a cell #35206 created a top-level cell, enabling Hubble to use the cell/hive dependency injection mechanism for its dependencies.
  2. daemon: refactor Hubble Exporters as a cell #35596 refactor of the Hubble export facilities (i.e. static and dynamic exporters) as cells.
  3. feat(hubble): decouple the payloadparser from hubble control plane. #38368 refactor of the Hubble top-level parser as a cell (+linkcache)
  4. hubble: Metrics server cell #39549 refactor of the Hubble metric subsystem (+certloader)

Sub-issues

Metadata

Metadata

Assignees

Labels

area/hubbleImpacts hubble server or relaykind/cfpCilium Feature Proposalkind/featureThis introduces new functionality.kind/metaMeta-task for co-ordination.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions