Skip to content

Conversation

florolf
Copy link
Contributor

@florolf florolf commented Oct 5, 2016

unibi_format() calls out() multiple times for a given format string.
When data->buf fills up during this process, flush_buf() gets called,
which possibly calls unibi_out() again to toggle the cursor visibility.
However, if we were halfway through outputting an escape sequence, doing
this will clobber it, resulting in junk being displayed.

Fix this by not toggling the cursor visibility when draining a full
buffer in out().

@florolf florolf changed the title tui/flush_buf: don't toggle cursor when called from out() [RFC] tui/flush_buf: don't toggle cursor when called from out() Oct 5, 2016
@marvim marvim added the RFC label Oct 5, 2016
static void flush_buf(UI *ui)
{
TUIData *data = ui->data;

if (!data->busy) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the cursor fiddling is already guarded by a boolean check, we could add a bool toggle_cursor parameter to flush_buf. IMO that's less awkward than having a "raw" variant of the function.

@justinmk
Copy link
Member

justinmk commented Oct 5, 2016

Thanks a lot for looking into this! Similar problem at #3875.

@justinmk
Copy link
Member

justinmk commented Oct 6, 2016

LGTM.

unibi_format() calls out() multiple times for a given format string.
When data->buf fills up during this process, flush_buf() gets called,
which possibly calls unibi_out() again to toggle the cursor visibility.
However, if we were halfway through outputting an escape sequence, doing
this will clobber it, resulting in junk being displayed.

Fix this by not toggling the cursor visibility when draining a full
buffer in out().
@florolf florolf force-pushed the dont-clobber-escapes branch from d4928c9 to e6054cf Compare October 6, 2016 07:10
@florolf florolf changed the title [RFC] tui/flush_buf: don't toggle cursor when called from out() [RDY] tui/flush_buf: don't toggle cursor when called from out() Oct 6, 2016
@marvim marvim added RDY and removed RFC labels Oct 6, 2016
@justinmk
Copy link
Member

justinmk commented Oct 6, 2016

@florolf can you describe the circumstances/ symptoms that you observed before this fix? IOW when did you see junk being displayed?

@justinmk
Copy link
Member

justinmk commented Oct 6, 2016

I.e. which terminal did you see this on? Did any specific settings, usage, or plugins make the symptoms more likely? Or was it just random?

@florolf
Copy link
Contributor Author

florolf commented Oct 6, 2016

I'm using termite, which uses libvte3-ng. For me, the problem only occured when a sufficient amount of text was being displayed on the screen (i.e. small font in a full screen terminal on a high resolution screen), which is not surprising given that we have to fill up a 64k buffer. The corruption I observed was pretty random overall, but deterministic for a given file/terminal size/cursor position; running :redraw! made it go away.

I think the problem is exacerbated when using a true color colorscheme (I am), which uses many long escape codes.

Other than that, I didn't really debug my config a lot since this was obviously a lower-level problem.

@florolf
Copy link
Contributor Author

florolf commented Oct 6, 2016

FWIW, I've been running a nvim version with this patch applied the whole day and it fixed all instances of the corruption I've been seeing. Not sure if this also means that this fix applies to #3875.

justinmk pushed a commit that referenced this pull request Oct 6, 2016
unibi_format() calls out() multiple times for a given format string.
When data->buf fills up during this process, flush_buf() gets called,
which possibly calls unibi_out() again to toggle the cursor visibility.
However, if we were halfway through outputting an escape sequence, doing
this will clobber it, resulting in junk being displayed.

Fix this by not toggling the cursor visibility when draining a full
buffer in out().
@justinmk
Copy link
Member

justinmk commented Oct 6, 2016

Merged, thanks @florolf!

Would mind sharing how you narrowed this down? What tools/steps did you use?

@justinmk justinmk closed this Oct 6, 2016
@justinmk justinmk removed the RDY label Oct 6, 2016
@florolf
Copy link
Contributor Author

florolf commented Oct 6, 2016

Thanks for merging!

As suggested on IRC, I used script to capture the screen output. Looking at the capture made it clear that this was a nvim issue and not a terminal issue, because the escape codes were already clobbered in the dump. Then I added debugging statements in various places in tui.c, which made it clear that the out() sequence of the true color escape sequences (which were almost 100% of those that leaked into the terminal for me) was being interrupted. (Another possibility would have been that unibilium was generating invalid data, but this turned out not to be the case). I remembered that the cursor toggling looked suspicious and that turned out to be the problem. :)

justinmk added a commit to justinmk/neovim that referenced this pull request Oct 27, 2016
FEATURES:
0b5a7e4 neovim#4432 API: external UIs can render custom popupmenu
c6ac4f8 neovim#4934 API: call any API method from vimscript
31df051 neovim#4568 API: nvim_call_atomic(): multiple calls in a single request
b268ba3 neovim#5424 API: nvim_win_get_number(), nvim_tabpage_get_number()
e7e2844 has("nvim-1.2.3") checks for a specific Nvim version
522b885 neovim#5295, neovim#5493 `:CheckHealth` checks tmux, terminfo, performance
719dae2 neovim#5384 events: allow event processing in getchar()
f25797f neovim#5386 API: metadata: Nvim version & API level
22dfe69 neovim#5389 API: metadata: "since", "deprecated_since"
605e743 Added QuickFixLine highlight group

CHANGES:
4af6ec7 neovim#5253 perf: Disable clipboard in do_cmdline()
6e9f329 neovim#5299 perf: Skip foldUpdate() in insert-mode.
9d4fcec neovim#5426 perf: Do not auto-update folds for some foldmethods.
eeec0ca neovim#5419 tui: Default to normal-mode cursor shape.

FIXES:
e838452 neovim#5436 tui: Fix "weird characters" / "bleeding termcodes"
10a54ad neovim#5243 signal_init: Always unblock SIGCHLD.
bccb49b neovim#5316 eval.c: Fix memory leak for detached pty job
626065d neovim#5227 tchdir: New tab should inherit CWD.
cd321b7 neovim#5292 getcwd(): Return empty string if CWD is invalid.
6127eae shada: Fix non-writeable ShaDa directory handling
ca65514 neovim#2789 system(): Respect shellxescape, shellxquote
2daf54e neovim#4874 Restore vim-like tab dragging
0c536b5 neovim#5319 syntax.c: Support bg/fg special color-names.
3c53371 neovim#4972 from justinmk/schedule-ui_refresh
68bcb32 neovim#4789 tui.c: Do not wait for tui loop on teardown.
c8b6ec2 neovim#5409 v:count broken in command-line window
6bc3bce neovim#5461 fix emoji display
51937e1 neovim#5470 fix :terminal with :argadd, :argu
79d77da neovim#5481 external UIs: opening multiple files from command-line
657ba62 neovim#5501 rplugin: resolve paths in manifest file
6a6f188 neovim#5502 system('foo &', 'bar'): Show error, don't crash.
1ff162c neovim#5515 os_nodetype: open fd with O_NONBLOCK
2a6c5bb neovim#5450 modeline: Handle version number overflow.
0ade1bb neovim#5225 CI tests now run against Windows!
justinmk added a commit to justinmk/neovim that referenced this pull request Oct 28, 2016
FEATURES:
0b5a7e4 neovim#4432 API: external UIs can render custom popupmenu
c6ac4f8 neovim#4934 API: call any API method from vimscript
31df051 neovim#4568 API: nvim_call_atomic(): multiple calls in a single request
b268ba3 neovim#5424 API: nvim_win_get_number(), nvim_tabpage_get_number()
e7e2844 has("nvim-1.2.3") checks for a specific Nvim version
522b885 neovim#5295, neovim#5493 `:CheckHealth` checks tmux, terminfo, performance
719dae2 neovim#5384 events: allow event processing in getchar()
f25797f neovim#5386 API: metadata: Nvim version & API level
22dfe69 neovim#5389 API: metadata: "since", "deprecated_since"
605e743 Added QuickFixLine highlight group

CHANGES:
4af6ec7 neovim#5253 perf: Disable clipboard in do_cmdline()
6e9f329 neovim#5299 perf: Skip foldUpdate() in insert-mode.
9d4fcec neovim#5426 perf: Do not auto-update folds for some foldmethods.
eeec0ca neovim#5419 tui: Default to normal-mode cursor shape.

FIXES:
e838452 neovim#5436 tui: Fix "weird characters" / "bleeding termcodes"
10a54ad neovim#5243 signal_init: Always unblock SIGCHLD.
bccb49b neovim#5316 eval.c: Fix memory leak for detached pty job
626065d neovim#5227 tchdir: New tab should inherit CWD.
cd321b7 neovim#5292 getcwd(): Return empty string if CWD is invalid.
6127eae shada: Fix non-writeable ShaDa directory handling
ca65514 neovim#2789 system(): Respect shellxescape, shellxquote
2daf54e neovim#4874 Restore vim-like tab dragging
0c536b5 neovim#5319 syntax.c: Support bg/fg special color-names.
3c53371 neovim#4972 from justinmk/schedule-ui_refresh
68bcb32 neovim#4789 tui.c: Do not wait for tui loop on teardown.
c8b6ec2 neovim#5409 v:count broken in command-line window
6bc3bce neovim#5461 fix emoji display
51937e1 neovim#5470 fix :terminal with :argadd, :argu
79d77da neovim#5481 external UIs: opening multiple files from command-line
657ba62 neovim#5501 rplugin: resolve paths in manifest file
6a6f188 neovim#5502 system('foo &', 'bar'): Show error, don't crash.
1ff162c neovim#5515 os_nodetype: open fd with O_NONBLOCK
2a6c5bb neovim#5450 modeline: Handle version number overflow.
0ade1bb neovim#5225 CI tests now run against Windows!
@justinmk justinmk added tui termcodes, terminfo, termcap debug labels Nov 30, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tui termcodes, terminfo, termcap
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants