Skip to content

Conversation

nschmeller
Copy link
Contributor

This commit adds support for specifying Kubernetes container image pull secrets in the kubernetes step decorator.

As an example:

@kubernetes(
    image='docker.io/some-private-repo/image',
    image_pull_secrets=['regcred']
)
@step
def some_step:
    ...

Example output from kubectl describe pod <some-pod> on the metaflow-dev stack where metaflow was installed via pip install -e </path/to/repo/checkout>:

Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  1m   default-scheduler  Successfully assigned default/t-f03abf3d-2rpgp-j72q8 to minikube
  Normal  Pulling    1m   kubelet            Pulling image "docker.io/some-private-repo/image"
  Normal  Pulled     1m   kubelet            Successfully pulled image "docker.io/some-private-repo/image" in 669ms (670ms including waiting)
  Normal  Created    1m   kubelet            Created container some-step
  Normal  Started    1m   kubelet            Started container some-step

Resolves #1261

@nschmeller nschmeller marked this pull request as ready for review May 8, 2025 16:08
This commit adds support for specifying Kubernetes container
image pull secrets in the `kubernetes` step decorator.

As an example:

```python
@kubernetes(
    image='docker.io/some-private-repo/image',
    image_pull_secrets=['regcred']
)
@step
def some_step:
    ...
```

Example output from `kubectl describe pod <some-pod>` on the
`metaflow-dev` stack where `metaflow` was installed via
`pip install -e </path/to/repo/checkout>`:

```console
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  1m   default-scheduler  Successfully assigned
default/t-f03abf3d-2rpgp-j72q8 to minikube
  Normal  Pulling    1m   kubelet            Pulling image
"docker.io/some-private-repo/image"
  Normal  Pulled     1m   kubelet            Successfully pulled image
"docker.io/some-private-repo/image" in 669ms (670ms including waiting)
  Normal  Created    1m   kubelet            Created container some-step
  Normal  Started    1m   kubelet            Started container some-step
```

Resolves Netflix#1261
@savingoyal savingoyal enabled auto-merge (squash) June 13, 2025 16:57
@savingoyal savingoyal merged commit 6cba5f3 into Netflix:master Jun 13, 2025
27 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Introduce support for container secrets for @kubernetes
2 participants