Skip to content

fix (Tiltfile) stuck in pending state #6493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 4, 2025

Conversation

knisbet
Copy link
Contributor

@knisbet knisbet commented Jan 28, 2025

Fixes #5902

I was having the same issue described in #5902 where when using filesystem triggers to add or remove resources, it would sometimes race and leave the Tiltfile in the pending status without the most recent state.

I believe I successfully tracked this down to the filewatch controller when reconciled can replace the watcher object with a new configuration, but doesn't preserve previous file events when doing so. This can then race with the Reconcile on the tiltfile to miss that there is a pending filesystem change and instead see an empty list of filesystem changes (

filesChanged = trigger.FilesChanged(tf.Spec.RestartOn, fileWatches, lastStartTime)
if len(filesChanged) > 0 {
reason = reason.With(model.BuildReasonFlagChangedFiles)
} else if timecmp.After(lastRestartEvent, lastStartTime) {
reason = reason.With(model.BuildReasonFlagTriggerUnknown)
}
). The Tiltfile reconciler then does not trigger a subsequent build.

@knisbet
Copy link
Contributor Author

knisbet commented Jan 28, 2025

I will note I'm not that familiar with the code base, so Sorry if there is an implication I'm missing with this change, or a better way to address the issue.

I will also note I think there is another race still present, but only impacting the Status as reported to the UI. The WebUI will show the (Tiltfile) status as pending, even though it has actually processed all the pending filesystem events. I don't believe it's caused by the same problem as this PR, but wanted to mention it just in case I'm missing something.

Signed-off-by: Kevin Nisbet <kevin@spirl.com>
@knisbet knisbet force-pushed the kevin/5902-watched-file-pending-fix branch from 27fc8de to 668d93e Compare January 28, 2025 17:56
@nicks nicks self-requested a review January 29, 2025 13:30
@nicks
Copy link
Member

nicks commented Jan 31, 2025

Thanks for sending this change - I think this is the right change conceptually, but is also a fairly low-level change, and need some time to test out some cases I'm worried about.

@knisbet
Copy link
Contributor Author

knisbet commented Feb 4, 2025

Thanks for sending this change - I think this is the right change conceptually, but is also a fairly low-level change, and need some time to test out some cases I'm worried about.

Sure thing. Feel free to tag me if I need to revise anything, and feel free to throw this out if you see a better way to fix the issue.

Copy link
Member

@nicks nicks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@nicks nicks merged commit 6d9d99f into tilt-dev:master Feb 4, 2025
6 of 7 checks passed
@knisbet knisbet deleted the kevin/5902-watched-file-pending-fix branch February 4, 2025 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug report: Watched file tiltfile reload leaves (Tiltfile) pending until manual reload
2 participants