Skip to content

startup: Remove special case of TUI initialization #9829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 4, 2019
Merged

startup: Remove special case of TUI initialization #9829

merged 6 commits into from
Jun 4, 2019

Conversation

erw7
Copy link
Contributor

@erw7 erw7 commented Apr 1, 2019

fixes #7967

Implemented according to #9825 (comment).

  • Move ui_builtin_start() to the same position as embedded_mode remote_ui_wait_for_attch().
  • If stdin is redirected, save the original stdin and replace fd 0 with tty before calling ui_builtin_start().
  • Remove unnecessary code such as start_input(), stop_input() etc.
  • Fix the problem that the display of nvim -u NORC -c "echo 'Hello'" is cleared.
  • Examine the behavior of exmode_active and correct if necessary.
  • Write a test.

@justinmk

This comment has been minimized.

@marvim marvim added the RFC label Apr 1, 2019
@erw7
Copy link
Contributor Author

erw7 commented Apr 2, 2019

This method has problems for the following reasons when stdin is redirected. stdin is changed from a pipe to a tty(console) in a readfile() called from read_stdin(). Therefore, if we call ui_builtin_start() before calling read_stdin(), the input of tui will be a pipe.

This problem occurs because we can not pass a tty handle to uv_pipe_init() on Windows. I think that it is good to solve by #9825.

The easiest way to solve this problem is #9825, but does this make sense to implement it as it may reduce code?

@erw7 erw7 changed the title [RFC] Fix stall before starting tui when something is output in Windows [WIP] Fix stall before starting tui when something is output in Windows Apr 2, 2019
@marvim marvim added WIP and removed RFC labels Apr 2, 2019
@justinmk
Copy link
Member

justinmk commented Apr 9, 2019

The easiest way to solve this problem is #9825, but does this make sense to implement it as it may reduce code?

stream.c change in 10f703d seems separate from #9825. Maybe we should merge that sooner than the rest.

See https://github.com/neovim/neovim/pull/9825/files#r273467992.
Another idea: on Windows, we could simply cancel any/all prompts if the UI is not ready, as a workaround for #7967.

@erw7
Copy link
Contributor Author

erw7 commented Apr 10, 2019

Is there any possibility that the following code will be executed?

neovim/src/nvim/tui/input.c

Lines 445 to 460 in 21b108f

if (eof) {
if (input->in_fd == 0 && !os_isatty(0) && os_isatty(2)) {
// Started reading from stdin which is not a pty but failed. Switch to
// stderr since it is a pty.
//
// This is how we support commands like:
//
// echo q | nvim -es
//
// and
//
// ls *.md | xargs nvim
input->in_fd = 2;
stream_close(&input->read_stream, NULL, NULL);
multiqueue_put(input->loop->fast_events, tinput_restart_reading, 1,
input);

@erw7
Copy link
Contributor Author

erw7 commented Apr 10, 2019

stream.c change in 10f703d seems separate from #9825. Maybe we should merge that sooner than the rest.

@justinmk Does that mean that it separates 10f703d into another pull request? I will ask you one more point. "Hello" displayed by nvim-qt--u NORC -c "echo 'Hello'" disappears immediately after startup. This seems to be the cause of the failure of the #9829 (comment) test, but is it better to create an issue?

@justinmk
Copy link
Member

justinmk commented Apr 10, 2019

Is there any possibility that the following code will be executed?

I think that's only relevant for -es/-Es. It happens when the stdin stream is closed.

@justinmk Does that mean that it separates 10f703d into another pull request?

I think so.

"Hello" displayed by nvim-qt--u NORC -c "echo 'Hello'" disappears immediately after startup. This seems to be the cause of the failure of the #9829 (comment) test,

By "disappears", do you mean it displayed in the shell, not the GUI?
That shouldn't happen if nvim-qt is using --embed without --headless, with Nvim 0.3.4. Because Nvim waits for the UI.

For all cases except --headless/-es/-Es, we should not attempt to display a message until we have a UI. That simplifies the system.

but is it better to create an issue?

Let's try to deal with it here, since it is related.

@erw7
Copy link
Contributor Author

erw7 commented Apr 11, 2019

I think that's only relevant for -es/-Es. It happens when the stdin stream is closed.

In the case of -es/-Es, the following code is not executed because silent_mode is true. I think that TUI will not execute and the corresponding code will not.

neovim/src/nvim/main.c

Lines 479 to 482 in b358054

if (!headless_mode && !embedded_mode && !silent_mode) {
input_stop(); // Stop reading input, let the UI take over.
ui_builtin_start();
}

If the code in question is not executed, I can use to the handle of TTY instead of stdin on tinput_init() when redirecting stdin, so I would like to know that.

@erw7
Copy link
Contributor Author

erw7 commented Apr 11, 2019

@justinmk Does that mean that it separates 10f703d into another pull request?

I think so.

@justinmk I created a separate pull request #9884.

@erw7
Copy link
Contributor Author

erw7 commented Apr 11, 2019

By "disappears", do you mean it displayed in the shell, not the GUI?

No. It looks like the GUI is cleared after launching.

After remote_ui_wait_for_attach() is called in main.c:370, exe_pre_commands() and exe_commands() are called in main.c:381 and main.c:548. Therefore, is not what displayed by --cmd/-c is displayed in the GUI?

@bfredl
Copy link
Member

bfredl commented Apr 11, 2019

Therefore, is not what displayed by --cmd/-c is displayed in the GUI?

It is displayed in --embed GUI but not TUI, but it could be a worthwhile change to make TUI also start early. Though it could have some effects like making env vars set in init.vim not take effect in TUI initialization.

@justinmk
Copy link
Member

In the case of -es/-Es, the following code is not executed because silent_mode is true. I think that TUI will not execute and the corresponding code will not.

My mistake, I was thinking of this:

neovim/src/nvim/os/input.c

Lines 111 to 114 in 9daa7d9

if (read_stream.closed && silent_mode) {
// Drained eventloop & initial input; exit silent/batch-mode (-es/-Es).
read_error_exit();
}

Though it could have some effects like making env vars set in init.vim not take effect in TUI initialization.

Good point, let's just document that. init.vim should not be munging TUI-related env vars.

@erw7
Copy link
Contributor Author

erw7 commented Apr 12, 2019

Good point, let's just document that. init.vim should not be munging TUI-related env vars.

Create tinit.vim to munging TUI-related env vars, and do we use it like ginit.vim in GUI?

@erw7
Copy link
Contributor Author

erw7 commented Apr 12, 2019

Comment said "Don't clear the screen", but CLEAR is assigned to must_redraw. Is the following code correct?

neovim/src/nvim/main.c

Lines 486 to 489 in d08692a

if (exmode_active || early_ui) {
// Don't clear the screen when starting in Ex mode, or when an
// embedding UI might have displayed messages
must_redraw = CLEAR;

@justinmk
Copy link
Member

justinmk commented Apr 12, 2019

Create tinit.vim to munging TUI-related env vars, and do we use it like ginit.vim in GUI?

Neither is needed. We will introduce a UIEnter event in a separate PR.

@bfredl
Copy link
Member

bfredl commented Apr 12, 2019

@erw7 Yes. The point is we can't clear right now, but the clear should then be done next time an ordinary redraw is done.

@justinmk
Copy link
Member

This looks really good so far!

@erw7
Copy link
Contributor Author

erw7 commented Apr 14, 2019

@bfredl The position where "Hello" displayed by nvim -u NORC -c "echo 'Hello'" is at the center of the column, but could you give me some hints to fix it?

@bfredl
Copy link
Member

bfredl commented Apr 14, 2019

@erw7 cannot reproduce, does this happen only on windows?

@erw7
Copy link
Contributor Author

erw7 commented Apr 14, 2019

@bfredl I have now confirmed on Linux. It seems like a problem that occurs only with Windows.

@erw7
Copy link
Contributor Author

erw7 commented Apr 17, 2019

@bfredl I understand the cause of #9829 (comment). It seems that the msg_col changed by the call of scriptnames() by execute() of the following code is not restored. This problem seems to be solved by #9338.

\ : matchstr(get(split(execute('scriptnames'), '\n'), 0, ''), '\f\+$')

@justinmk justinmk added this to the 0.4 milestone Apr 28, 2019
@erw7 erw7 changed the title [WIP] Fix stall before starting tui when something is output in Windows [RFC] Remove the special case of TUI initialization May 30, 2019
@marvim marvim added RFC and removed WIP labels May 30, 2019
@justinmk
Copy link
Member

justinmk commented Jun 3, 2019

#9338 is merged now, would you mind rebasing this?

@erw7
Copy link
Contributor Author

erw7 commented Jun 4, 2019

#9338 is merged now, would you mind rebasing this?

@justinmk I rebased this.

GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE,
(LPSECURITY_ATTRIBUTES)NULL,
OPEN_EXISTING, 0, (HANDLE)NULL);
Copy link
Member

@justinmk justinmk Jun 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is FILE_SHARE_WRITE needed? We do not specify it in

CreateFile("CONIN$", GENERIC_READ | GENERIC_WRITE,

setmouse(); // may start using the mouse

if (exmode_active || early_ui) {
// Don't clear the screen when starting in Ex mode, or when an
// embedding UI might have displayed messages
must_redraw = CLEAR;
redraw_later(VALID);
} else {
screenclear(); // clear screen
TIME_MSG("clearing screen");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we pretty much never hit this case, now :)

@justinmk justinmk merged commit 4719fdb into neovim:master Jun 4, 2019
justinmk added a commit that referenced this pull request Jun 4, 2019
fixes #7967
fixes #9959

Historically Vim/Nvim does backflips to handle input and show messages
before a UI is available. This logical contradiction was already fixed
for remote UIs (#9024 c236e80). Fixing it also for the TUI avoids
problems on Windows, simplifies the logic, and avoids races like #9959.

- Move ui_builtin_start() to the same position as embedded_mode
  remote_ui_wait_for_attach().
- If stdin is redirected, save the original `stdin` and replace fd
  0 with tty before calling `ui_builtin_start()`.
@erw7 erw7 deleted the fix-stall-before-tui-start-on-windows3 branch September 11, 2019 05:20
justinmk added a commit that referenced this pull request Sep 15, 2019
This release represents ~2700 commits since v0.3.4, the previous
non-maintenance release.  Besides the highlights listed below, this
release features vast improvements to documentation, internal subsystems
and test/CI infrastructure, and 700+ patches merged from Vim.

FEATURES:

New API functions:
  nvim_create_buf: create various kinds of buffers
  nvim_get_context, nvim_load_context
    8e6b0a7 #10619 API: Context: save/restore/inspect editor state
  nvim_input_mouse: perform mouse actions
  nvim_open_win: create floating windows (and external, for supporting UIs)
  nvim_paste: paste text at cursor
  nvim_put: put text at cursor
  nvim_select_popupmenu_item: perform popupmenu actions
  nvim_set_keymap: create/delete mappings
  nvim_set_vvar: set v: variables
  nvim_ui_pum_set_height
  nvim_ui_try_resize_grid
  nvim_win_close: close windows
  nvim_win_get_config: get window configuration
  nvim_win_set_config: reconfigure windows

New UI events:
  redraw.grid_destroy
  redraw.hl_group_set
    8a3f858 #10504 UI/highlight: expose builtin highlight groups using hl_group_set event
  redraw.msg_clear
  redraw.msg_history_show
  redraw.msg_ruler
  redraw.msg_set_pos
  redraw.msg_show
  redraw.msg_showcmd
  redraw.msg_showmode
  redraw.win_close
  redraw.win_external_pos
  redraw.win_float_pos
  redraw.win_hide
  redraw.win_pos

API
f5c56f0 #9170 API/Lua: nvim_buf_attach: support Lua callback
82d48c0 #9896 API: emit nvim_error_event on failed async request
b9ad12e #9992 UI/nvim_ui_attach(): add `override` option
3d1ed7c #9993 UI/ext_messages: learn more message kinds
8ed54bb #9547 proper multiline error message for rpcrequest, API wrappers

Lua
This release introduces "Nvim-Lua standard library". See ":help lua-stdlib".
89d7e24 #9463 Lua stdlib: vim.inspect, string functions
8e941c5 #9740 Lua: generate documentation from docstrings
1cbe014 #9301 lua/stdlib: Introduce vim.shared
c83926c #10123 Lua: introduce vim.loop (expose libuv event-loop)
81e1dbc #10120 Lua: vim.schedule(cb)
1f54f68 #10688 Lua: minimal UTF-16 support needed for LSP
6fb0020 #10513 Lua encoding support
    c0993ed Lua: support getting UTF-32 and UTF-16 sizes of replaced text
    b0e2619 Lua: add {old_byte_size} to on_lines buffer change event

UI:
- The Nvim 0.3.4 UI protocol introduced line-based updates instead of
  legacy char-based updates. Nvim 0.4 continues to evolve the UI
  protocol. See ":help ui". Legacy UI clients are supported. See
  ":help api-contract".
9a1675b #6619 Floating windows
  - Can be (re)positioned, anchored, external.
  - Are real windows showing real buffers. No shortcuts, hacks, or compromises.
  - Support all features and API of normal windows, plus more.
6427894 #8455 Multigrid: "windows drawn on separate grids"
  - Windows are logically isolated internally.
  - Windows are sent to UIs as distinct objects, so that UIs can control
    layout instead of being stuck with the classic TUI layout.
  - Per-window font-size, dimenions, line-spacing.
  - Compositor: Internal subsystem for composing grids.
3855204 #6917 UIEnter, UILeave
788bcbb #9923 ui: ":syn blend=", 'winblend'
7cf7c0a #9575 ui: 'redrawdebug' option for flexible debugging of redrawing
5c836d2 #9607 wildoptions=pum (enabled by default)
37f8df8 #9571 UI: 'pumblend' option for semi-transparent popupmenu
c403a95 #9446 Visual: highlight char-at-cursor
  - Traditionally Vim's visual selection does "reverse mode", which
    perhaps conflicts with the non-blinking block cursor. But
    'guicursor' defaults to a vertical bar for selection=exclusive, and
    this confuses users who expect to see the text highlighted.

:terminal
fc27dc9 #8550 autocmds: TermEnter, TermLeave
d13803f #9810 keymap, terminal: more keycodes
3b56f59 #9535 :terminal : Fix F1-F4 key codes
2d4a37e #10370 :ls : show "R", "F" for terminal-jobs
fd0fd75 #9966 terminal: swap priority of terminal, editor highlights
7bb858c #9494 libvterm 0.1

TUI
3afb397 syntax, TUI: support "strikethrough"
ccbcd39 #9408 TUI: "title stacking" unconditionally
298608f #9509 TUI: detect background color, set bg=dark/light
42f492a #9097 TUI: handle Smulx extension capability (extended underline)
424ddd0 #10205 TUI: support rgba background detection
9b43832 #9601 TUI: italics in tmux, Terminal.app
f6fb370 #9793 keymap: support more (keypad) keycodes
3340e08 #9423 TUI: Konsole DECSCUSR fixup

:checkhealth
d0fd66b health/provider.vim: check curl HTTPS support
c38862a #10490 checkhealth: try yarn if npm is missing
43356a4 #9929 health: check if tmux enabled true colors
ec5a4d8 #9548 checkhealth: validate locale

providers (clipboard, python, etc.):
96be8a2 #10161 Allow reloading providers (useful for UIs/clients)
db3c797 #9487 provider: improve error message if provider is missing

Various:
36762a0 #9295 signs: support multiple columns
801fe79 #10382 eval: wait() (wait for any condition)
9df3a67 #10400 MsgArea highlight; message grid
a9bea8c #10790 keymap: allow modifiers to multibyte chars, like <m-ä>
25e0a44 #10878 #4448 paste: redesign (10x+ faster pasting; extensible vim.paste Lua hook)
ef5037e #9706 autocmd: introduce "++once" feature
175398f #9616 add CompleteChanged autocmd
7fcf2f9 #9717 TextYankPost: add v:event["inclusive"]
3a699a7 #8364 termdebug.vim plugin
ca1ce59 #9709 performance: use os_copy to create backups
ed0e96c man.vim: set 'linebreak'
70f6939 #9564 events: add "Signal" event
f89d0d8 #9568 inccommand: auto-disable if folding is slow

FIXES:

41bb68b #10584 process_stop: uv: do not close stdin first/explicitly
e50aa2a #10117 normal: Don't exit CTRL-O mode after processing K_EVENT
95fa71c #9504 :recover : Fix crash on non-existent *.swp
5a836d4 #9507 screen: don't unconditionally clear messages on window scroll
149dcbf #10021 channel: refactor events, prevent recursive invocation of events
d19ff73 #10107 Fix multiple c_CTRL-D showing statusline
b65a7b7 #10103 Fix wildmode=list,full and display+=msgsep interaction
0be6d3c #9634 fsync: Ignore ENOTSUP. Fix writing to SMB.
b247c6f #10025 kbtree: pointer UB and unitialized value fixes
018e0d5 #9643 API/buffer-updates: always detach on buf-reload
400ee59 #9961 API: fix cursor position when lines are added
769f44e #9911 win/defaults: Use "…/nvim-data/site" in 'runtimepath'
83d5716 #9911 spellfile.vim: store files in stdpath('data')
8dbf231 #9887 RPC: conform message-id type to msgpack-RPC spec
5f996e3 #9894 options: properly reset directories on 'autochdir'
4c4a570 #9807 various CursorMoved fixes
943bedf #9853 event-loop: do not set CA_COMMAND_BUSY
9d207fd #9693 dictwatcheradd(): support b:changedtick
2d50bf3 #9789 mac: fix locale detection
c563133 #9754 :mksession : restore tab-local working directories
092e7e6 #9703 #9703 executable(): return false if user is not owner
11a481f #9686 env var fixes/improvements
8e54847 #9666 #7920 os/env: Fix completion of multibyte env var names
5193826 #10468 Fix is_executable_in_path() on Windows
8eaa452 #9516 win: exepath(), executable() fixes
f55c1e4 #10544 reltimefloat(): allow negative result
b08dc3e #10561 win: jobstart(), system(): $PATHEXT-resolve exe
7cc2b72 #10392 TextYankPost: spurious/too-early dispatch during delete
6e01ed6 OpenBSD: stop jobs/processes properly
58dd5fc #10522 jobstop(): close channel before process_stop()
8363202 #10959 improved resize behavior (all UIs)
c6eb1f4 #10830 API: fix nvim_command_output buffer overflow
cbfd18c #10763 startup: handle 'guicursor' after user config
b8f2436 #10915 jobwait(): fix race if job exits quickly
2fafed6 #10765 clipboard: handle/avoid SIGTERM with previous owner
8aca932 #9954 clipboard: setreg("*") with clipboard=unnamed
3f10c5b #9480 performance: clipboard/macOS: assume that pbcopy works
48efafc #10398 screen: disable redrawing inside VimResized
5e4b93a #10389 API/Lua: make nvim_execute_lua use native lua floats, not special tables
8c6f5b7 #9934 Spurious quote mark in command line when typing <C-R>
a8a38f3 Lua 5.2/5.3 compat

:terminal
47b7b47 #10700 :terminal : update buffer when switching tabpage
5225c1e #9605 terminal: Fix potential invalid local 'scrollback'
894f6be #8325 :terminal : set topline based on window height
8171e96 #9551 Improve :terminal resize
d928b03 #9856 :stopinsert should leave terminal-mode
3f71218 #9926 :terminal : fix: Using `:stopinsert` while in normal mode
5020daa #9883 ui/terminal: make terminal state redraw like any other state

TUI:
9f19e8d #9443 TUI: Do not disable BCE for builtin terminfos
a4076e5 #9474 win/TUI: fix text overrides line numbers
533d4a3 #9645 TUI: do not resize host-terminal on startup
b51e5d8 #9688 tui_tk_ti_getstr: handle weird value
1f5eac1 #10785 TUI: fix data-race during resize

CHANGES:

9697c7f #8194 fix menu_get()
7f2e43c #9520 improve Lua error messages
c234318 #9526 Remove jemalloc
baf93d9 #9581 UI: always use concrete colors for default_colors_set
91688b4 #9563 defaults: set 'scrollback' to -1 by default
bb24fec #10136 defaults: exclude "S" from 'shortmess'
ddd0eb6 #8540 startup: -es/-Es (silent/batch mode): skip swapfile
3536249 #9805 jumplist: avoid extra tail entry
939d905 #10573 channels: reflect exit due to signals in exit status code
45c34bd #10689 :doautocmd : Never show "No matching autocommands"
fb19aee #9110 API: make nvim_win_set_option() set window-global, not buffer-local
abfc8b3 #10778 emsg_multiline: log Vim errors
06d9cc7 #10657 exists("$FOO"): return false for empty env var
6616d1d #10743 win/env: Vim-compat: Empty string deletes env var
7d66483 #10662 win: expand nested env var in $HOME
2816bc8 #8349 edit.c: Disable indent during completion
58f505d #9829 startup: remove TUI init special-case
    Historically Vim/Nvim does backflips to handle input and show messages
    before a UI is available. This logical contradiction was already fixed
    for remote UIs (#9024 c236e80). Fixing it also for the TUI avoids
    problems on Windows, simplifies the logic, and avoids races like #9959.
@justinmk justinmk added the lifecycle startup, shutdown, attach, detach label Nov 28, 2019
@justinmk justinmk changed the title [RFC] Remove the special case of TUI initialization startup: Remove special case of TUI initialization Oct 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lifecycle startup, shutdown, attach, detach
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Windows/TUI: Printing to stdout during initialization prevents starting normally
4 participants