feat: add hello world deployment example #1854
Conversation
WalkthroughA new Kubernetes deployment manifest for the hello_world example was added, defining a custom resource with three services: Frontend, Middle, and Backend. Additionally, the Backend service's worker count in the configuration file was changed from 2 to 1. No other resource or parameter changes were made. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Frontend
participant Middle
participant Backend
User->>Frontend: Send request
Frontend->>Middle: Forward request
Middle->>Backend: Forward request
Backend-->>Middle: Process and respond
Middle-->>Frontend: Process and respond
Frontend-->>User: Respond
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
examples/hello_world/deploy/k8s/hello_world.yaml (2)
48-73: Explicitly setcomponentTypefor Middle to avoid implicit default.Only
FrontenddeclarescomponentType: main. The CRD currently defaults missing values to"worker"but being explicit improves readability and guards against future default changes.Middle: + componentType: worker
75-99: Backend section mirrors Frontend—consider YAML anchors or kustomize for DRYness.All three service blocks repeat identical resource and container specs (except the service name). Using YAML anchors/aliases or templating (Helm/kustomize) would trim ~60 lines and make future edits less error-prone.
Example with anchors:
x-common: &common replicas: 1 resources: requests: cpu: "1" memory: "2Gi" limits: cpu: "1" memory: "2Gi" extraPodSpec: # … Frontend: <<: *common componentType: main # … Middle: <<: *common componentType: worker # … Backend: <<: *common componentType: worker # …
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
examples/hello_world/config.yaml(1 hunks)examples/hello_world/deploy/k8s/hello_world.yaml(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
examples/hello_world/config.yaml (3)
Learnt from: nnshah1
PR: ai-dynamo/dynamo#1444
File: tests/fault_tolerance/configs/agg_tp_1_dp_8.yaml:31-38
Timestamp: 2025-07-01T15:33:53.262Z
Learning: In fault tolerance test configurations, the `resources` section under `ServiceArgs` specifies resources per individual worker, not total resources for all workers. So `workers: 8` with `gpu: '1'` means 8 workers × 1 GPU each = 8 GPUs total.
Learnt from: fsaady
PR: ai-dynamo/dynamo#1730
File: examples/sglang/slurm_jobs/scripts/worker_setup.py:230-244
Timestamp: 2025-07-03T10:14:30.570Z
Learning: In examples/sglang/slurm_jobs/scripts/worker_setup.py, background processes (like nats-server, etcd) are intentionally left running even if later processes fail. This design choice allows users to manually connect to nodes and debug issues without having to restart the entire SLURM job from scratch, providing operational flexibility for troubleshooting in cluster environments.
Learnt from: GuanLuo
PR: ai-dynamo/dynamo#1371
File: examples/llm/benchmarks/vllm_multinode_setup.sh:18-25
Timestamp: 2025-06-05T01:46:15.509Z
Learning: In multi-node setups with head/worker architecture, the head node typically doesn't need environment variables pointing to its own services (like NATS_SERVER, ETCD_ENDPOINTS) because local processes can access them via localhost. Only worker nodes need these environment variables to connect to the head node's external IP address.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Build and Test - vllm
🔇 Additional comments (2)
examples/hello_world/config.yaml (1)
31-36: Worker count reduction brings CPU request back in line – double-check remaining services.Changing
Backend.ServiceArgs.workersfrom 2 → 1 prevents the previous over-subscription (recall that thecpu: "1"entry is per worker, per learning #1444). Good catch.
While you’re here, confirm thatMiddle(stillworkers: 2,cpu: "1") is intentional; at present that pod will ask the scheduler for 1 CPU yet try to run two worker processes.examples/hello_world/deploy/k8s/hello_world.yaml (1)
49-57: CPU request may be insufficient forMiddle’s two workers.
Middlepod requests only 1 CPU butconfig.yamllaunches 2 workers. Either raise the CPU request/limit (e.g.,"2") or lowerworkersto 1 to avoid throttling.
Overview:
Add hello world k8s deployment example
Closes: DYN-695
Fixes: nvbug: 5383701
Summary by CodeRabbit