Skip to content

Avoid build error messages on okteto test execution #4736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 23, 2025

Conversation

ifbyol
Copy link
Member

@ifbyol ifbyol commented Jun 20, 2025

Proposed changes

On okteto test, when it fails due to some problem with BuildKit (bad Dockerfile for example) we are showing the error coming from the buildkit underlying layers, which usually mention build operations or similar, that doesn't make sense for test execution.

For that reason, I'm changing to show a generic error when okteto test execution fails unexpectedly indicating that the test execution of the test container failed, with a hint to review the image and artifacts, and a reminder for using log-level flag to get additional output.

In the case of getting a CommandErr, we just return that error, as that represents an error in the test execution itself.

As part of this, I also added some changes in pkg/build/buildkit/errors.go to not consider the error failed to calculate checksum of ref xxxx: "xxx": not found as some lack of permissions getting the image. I added an extra check to return the error as it is in that case instead of completely replace it.

In the of having artifacts defined in the test container, this is the error we return now

Screenshot 2025-06-20 at 13 38 52

If there is no artifacts defined, this is the error we display

Screenshot 2025-06-20 at 13 40 10

In case of a failure in the test itself, we keep returning the same:

Screenshot 2025-06-20 at 13 41 05

How to validate

Define a test container with an artifact that is not generated by the commands, for example:

test:
  backend:
    image: okteto/golang:1
    context: backend
    caches:
      - /go/
      - /root/.cache/
    commands:
      - echo "foo"
    artifacts:
      - foo.txt

It should fail with the generic error I'm adding. Before, it was failing with a misleading error: The image '' is not accessible or it does not exist

Define a test container with a non-existent image, for example:

test:
  backend:
    image: okteto/foo:1
    context: backend
    caches:
      - /go/
      - /root/.cache/
    commands:
      - echo "foo"

It should fail with the generic error I'm adding.

Define a test container with a failed command to make sure we are failing in the same way we used to do:

test:
  backend:
    image: okteto/golang:1
    context: backend
    caches:
      - /go/
      - /root/.cache/
    commands:
      - asdgasfga

It should fail with the same error than without my changes.

ifbyol added 3 commits June 20, 2025 13:17
Signed-off-by: Nacho Fuertes <nacho@okteto.com>
Signed-off-by: Nacho Fuertes <nacho@okteto.com>
…hat error

Signed-off-by: Nacho Fuertes <nacho@okteto.com>
@ifbyol ifbyol requested a review from a team as a code owner June 20, 2025 11:47
Copy link

codecov bot commented Jun 20, 2025

Codecov Report

Attention: Patch coverage is 20.00000% with 12 lines in your changes missing coverage. Please review.

Project coverage is 48.86%. Comparing base (08807a2) to head (3488d03).

❌ Your patch check has failed because the patch coverage (20.00%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4736      +/-   ##
==========================================
- Coverage   48.90%   48.86%   -0.04%     
==========================================
  Files         355      355              
  Lines       29752    29766      +14     
==========================================
- Hits        14550    14546       -4     
- Misses      14043    14059      +16     
- Partials     1159     1161       +2     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ifbyol ifbyol merged commit 5362ff5 into master Jun 23, 2025
16 of 18 checks passed
@ifbyol ifbyol deleted the ifbyol/avoid-build-errors-on-test-execution branch June 23, 2025 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants