What challenge are you facing?

We are heavy users of Concourse and have four different instances running pipelines for our dev workflows. All four of these instances run on Concourse 2.6.0. Recently we upgraded to 3.3.4 on one of these instances and began to see "unknown handle" errors on some of the resources used by the pipelines. However, these errors only occurred on the day of the upgrade and did not appear again for five days. After upgrading a second instance to 3.3.4, we saw the same "unknown handle" errors at a significantly greater frequency on pipelines in both Concourse instances, several times per day. Given enough time, some of these errors seem to resolve themselves when running a job that uses the resource. But most of the time we have to recreate the worker associated to the lost handle in order to resolve, else the job stays in a pending state for a long period of time.

So far we have observed that the error consistently occurs on the same two resource types. One an OOTB git resource type that triggers on changes to the master branch from different repositories. The other is a custom resource type that triggers on pull requests from different repositories. It is also worth mentioning that there is a custom developed git hook that triggers jobs. Overall the issue seems similar to this: #1255. However, we need help tracking what the exact issue is from the Concourse side and what specifically we should do to rectify it.

I have logs from each worker captured during the time of the failures, an example pipeline configuration that is representative of the pipeline configurations that seem to be affected, and the source code from the custom resource type mentioned above; all ready to share with someone who can help me figure out what the issue is.

• Concourse version: 3.3.4
• Deployment type (BOSH/Docker/binary): BOSH
• Infrastructure/IaaS: Infrastructure
• Browser (if applicable): N/A
• Did this used to work? Yes, we saw no "unknown handle" errors while running 2.6.0