Skip to content

Concourse resource pull is flakey and requires a web VM restart #2150

@kkallday

Description

@kkallday

Bug Report

Hi Concourse folks,

My team has ran into some weird issues with resources not checking/updating and resulting in jobs not triggering.

Specifically, today I ran into an issue in which a resource wasn't listing artifacts. There were two different pipelines that have the same resource that pull releases using the pivnet resource. Both resources have the same definition in the pipeline yaml however one was able to list artifacts successfully and the other one wasn't. Furthermore, the fly check-resource command hung indefinitely when trying to update the resource.

Although restarting the web vms fixed the issue, I'd like to know if there's a specific place I can look into for logs to help diagnose the problem. We have a fairly large concourse deployment (~10 workers) so its kind of hard to track down the which worker might have hosted the check container.

As a stopgap, we are now restarting our vms every day but this feels like a hack.

Thanks,

Kevin

  • Concourse version: v3.9.2
  • Deployment type (BOSH/Docker/binary): BOSH
  • Infrastructure/IaaS: GCP
  • Did this used to work? We previously didn't have to restart our web vms so often

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions