needed after neovim#9024 to receive options from init.vim

needed after neovim#9024 to receive options from init.vim

needed after neovim#9024 to receive options from init.vim

needed after neovim#9024 to receive options from init.vim

Besides the "visible" improvements, this release features numerous
internal improvements to the UI/screen code and test infrastructure.

Numerous patches were merged from Vim, which are not mentioned below.

FEATURES:

07ad5d7 clipboard: Support custom VimL functions neovim#9304
725da1f neovim#9401 win/TUI: Improve terminal/console support
7a8dadb neovim#9077 startup: Use $XDG_CONFIG_DIRS/nvim/sysinit.vim if it exists
feec926 neovim#9299 support <cmd> mapping in more places
0653ed6 neovim#9028 diff/highlight: Show underline for low-priority CursorLine
bddcbbb signs: Add "numhl" argument neovim#9113
05f9c7c clipboard: support Wayland (neovim#9230)
14ae394 neovim#9052 TUI: add support for undercurl and underline color
4fa3492 neovim#9023 man.vim: soft (dynamic) wrap neovim#9023

API:

8b39e4e neovim#6920 API: implement object namespaces
b1aaa0a API: Implement nvim_win_set_buf() neovim#9100
8de87c7 neovim#8180 API: virtual text annotations (nvim_buf_set_virtual_text)
2b9fc9a neovim#8660 API: add nvim_buf_is_loaded()
    API: buf_get_lines, buf_line_count handle unloaded buffers
88f77c2 API: nvim_buf_get_offset_for_line
94841e5 API/UI: neovim#8221 ext_newgrid, ext_hlstate
    (use line-based rather than char-based updates)

UI

b5cfac0 neovim#8806 TUI: use BCE again more often, (smoother resizes/scrolling)
77b5e9a neovim#9315 screen: add missing status redraw when redraw_later(CLEAR) was used
5f15788 TUI: clip invalid regions on resize (neovim#8779), fixes neovim#8774
c936ae0 neovim#9193 TUI: improvements for scrolling and clearing
f204274 neovim#9143 UI: disable clearing almost everywhere
f4b2b66 neovim#9079 TUI: always use safe cursor movement after resize
d36afaf neovim#9211 ui_options: also send when starting or from OptionSet
67f80d4 TUI: Avoid reset_cursor_color in old VTE neovim#9191
e55ebae neovim#9021 don't erase screen on `:hi Normal` during startup
c5790d9 neovim#8915 TUI: Hint wrapped lines to terminals.

FIXES:

231de72 RPC: turn errors from async calls into notifications
907ad92 TUI: Restore terminal title via "title stacking" (neovim#9407)
cb76a8a genappimage: Unset $ARGV0 at invocation neovim#9376
b48efd9 neovim#9347 TUI: FreeBSD: Improve support for BSD vt console
c16529a TUI: Konsole 18.07.70 supports DECSCUSR (neovim#9364)
aec096f os/lang: use the correct LC_NUMERIC also for OS X
5fee0be provider: improve error message (neovim#9344)
3c42d7a TUI: alacritty supports set_cursor_color neovim#9353
7bff9a5 TUI: Alacritty supports DECSCUSR (neovim#9048)
57acfce macOS: infer primary language if $LANG is empty neovim#9345
bc132ae runtime/syntax: Fix highlighting of augroup contents (neovim#9328)
715fdfe neovim#9297 VimL/confirm(): Show dialog even if :silent
799d9c3 clipboard: Prefer xclip (neovim#9302)
6dae777 provider/nodejs: fix npm,yarn detection
16bc1e9 neovim#9218 channel: avoid buffering output when only terminal and no callbacks are active
72fecad neovim#8804 Fix crash in lang_init() on macOS if lang_region = NULL
d581398 ruby: detect rbenv shims for other versions (neovim#8733)
e568ac7 neovim#9123 third-party/unibilium: Fix parsing of extended capability entries
c4c74c3 jobstart(): Fix hang on non-executable cwd neovim#9204
1cf50cb provider/nodejs: Simultaneously query npm and yarn neovim#9054
6c496db undo: Fix infinite loop if undo_read_byte returns EOF neovim#2880
f8f8357 neovim#9034 'swapfile: always show dialog'

CHANGES:

c236e80 neovim#9024 --embed: wait for UI unless --headless
180b50d neovim#9248 python: 'neovim' module was renamed to 'pynvim'
2000b6a neovim#8589 VimL: Remove legacy aliases "v:errmsg", "v:shell_error", "v:this_session"
deb18a0 defaults: background=dark neovim#2894 (neovim#9205)
c1187d4 defaults: win: 'shellpipe' for cmd.exe (neovim#8827)

Besides the "visible" improvements, this release features numerous
internal improvements to the UI/screen code and test infrastructure.

Numerous patches were merged from Vim, which are not mentioned below.

FEATURES:

07ad5d7 clipboard: Support custom VimL functions neovim#9304
725da1f neovim#9401 win/TUI: Improve terminal/console support
7a8dadb neovim#9077 startup: Use $XDG_CONFIG_DIRS/nvim/sysinit.vim if it exists
feec926 neovim#9299 support <cmd> mapping in more places
0653ed6 neovim#9028 diff/highlight: Show underline for low-priority CursorLine
bddcbbb signs: Add "numhl" argument neovim#9113
05f9c7c clipboard: support Wayland (neovim#9230)
14ae394 neovim#9052 TUI: add support for undercurl and underline color
4fa3492 neovim#9023 man.vim: soft (dynamic) wrap neovim#9023

API:

8b39e4e neovim#6920 API: implement object namespaces
b1aaa0a API: Implement nvim_win_set_buf() neovim#9100
8de87c7 neovim#8180 API: virtual text annotations (nvim_buf_set_virtual_text)
2b9fc9a neovim#8660 API: add nvim_buf_is_loaded()
    API: buf_get_lines, buf_line_count handle unloaded buffers
88f77c2 API: nvim_buf_get_offset_for_line
94841e5 API/UI: neovim#8221 ext_newgrid, ext_hlstate
    (use line-based rather than char-based updates)

UI

b5cfac0 neovim#8806 TUI: use BCE again more often, (smoother resizes/scrolling)
77b5e9a neovim#9315 screen: add missing status redraw when redraw_later(CLEAR) was used
5f15788 TUI: clip invalid regions on resize (neovim#8779), fixes neovim#8774
c936ae0 neovim#9193 TUI: improvements for scrolling and clearing
f204274 neovim#9143 UI: disable clearing almost everywhere
f4b2b66 neovim#9079 TUI: always use safe cursor movement after resize
d36afaf neovim#9211 ui_options: also send when starting or from OptionSet
67f80d4 TUI: Avoid reset_cursor_color in old VTE neovim#9191
e55ebae neovim#9021 don't erase screen on `:hi Normal` during startup
c5790d9 neovim#8915 TUI: Hint wrapped lines to terminals.

FIXES:

231de72 RPC: turn errors from async calls into notifications
907ad92 TUI: Restore terminal title via "title stacking" (neovim#9407)
cb76a8a genappimage: Unset $ARGV0 at invocation neovim#9376
b48efd9 neovim#9347 TUI: FreeBSD: Improve support for BSD vt console
c16529a TUI: Konsole 18.07.70 supports DECSCUSR (neovim#9364)
aec096f os/lang: use the correct LC_NUMERIC also for OS X
5fee0be provider: improve error message (neovim#9344)
3c42d7a TUI: alacritty supports set_cursor_color neovim#9353
7bff9a5 TUI: Alacritty supports DECSCUSR (neovim#9048)
57acfce macOS: infer primary language if $LANG is empty neovim#9345
bc132ae runtime/syntax: Fix highlighting of augroup contents (neovim#9328)
715fdfe neovim#9297 VimL/confirm(): Show dialog even if :silent
799d9c3 clipboard: Prefer xclip (neovim#9302)
6dae777 provider/nodejs: fix npm,yarn detection
16bc1e9 neovim#9218 channel: avoid buffering output when only terminal and no callbacks are active
72fecad neovim#8804 Fix crash in lang_init() on macOS if lang_region = NULL
d581398 ruby: detect rbenv shims for other versions (neovim#8733)
e568ac7 neovim#9123 third-party/unibilium: Fix parsing of extended capability entries
c4c74c3 jobstart(): Fix hang on non-executable cwd neovim#9204
1cf50cb provider/nodejs: Simultaneously query npm and yarn neovim#9054
6c496db undo: Fix infinite loop if undo_read_byte returns EOF neovim#2880
f8f8357 neovim#9034 'swapfile: always show dialog'

CHANGES:

c236e80 neovim#9024 --embed: wait for UI unless --headless
180b50d neovim#9248 python: 'neovim' module was renamed to 'pynvim'
2000b6a neovim#8589 VimL: Remove legacy aliases "v:errmsg", "v:shell_error", "v:this_session"
deb18a0 defaults: background=dark neovim#2894 (neovim#9205)
c1187d4 defaults: win: 'shellpipe' for cmd.exe (neovim#8827)

After neovim/neovim#9024, "nvim --embed" waits for a UI to attach.
"nvim --embed --headless" is required if there is not going to be a UI.

Since Nvim 0.3.2 the meaning of --headless was changed to 
more-strictly imply "no UI", and the --embed was changed to
wait until UI attach before continuing startup.

- :help --headless
- :help --embed
- https://github.com/neovim/neovim/wiki/Following-HEAD#20180922
- neovim/neovim#9024

fixes #7967
fixes #9959

Historically Vim/Nvim does backflips to handle input and show messages
before a UI is available. This logical contradiction was already fixed
for remote UIs (#9024 c236e80). Fixing it also for the TUI avoids
problems on Windows, simplifies the logic, and avoids races like #9959.

- Move ui_builtin_start() to the same position as embedded_mode
  remote_ui_wait_for_attach().
- If stdin is redirected, save the original `stdin` and replace fd
  0 with tty before calling `ui_builtin_start()`.

This release represents ~2700 commits since v0.3.4, the previous
non-maintenance release.  Besides the highlights listed below, this
release features vast improvements to documentation, internal subsystems
and test/CI infrastructure, and 700+ patches merged from Vim.

FEATURES:

New API functions:
  nvim_create_buf: create various kinds of buffers
  nvim_get_context, nvim_load_context
    8e6b0a7 #10619 API: Context: save/restore/inspect editor state
  nvim_input_mouse: perform mouse actions
  nvim_open_win: create floating windows (and external, for supporting UIs)
  nvim_paste: paste text at cursor
  nvim_put: put text at cursor
  nvim_select_popupmenu_item: perform popupmenu actions
  nvim_set_keymap: create/delete mappings
  nvim_set_vvar: set v: variables
  nvim_ui_pum_set_height
  nvim_ui_try_resize_grid
  nvim_win_close: close windows
  nvim_win_get_config: get window configuration
  nvim_win_set_config: reconfigure windows

New UI events:
  redraw.grid_destroy
  redraw.hl_group_set
    8a3f858 #10504 UI/highlight: expose builtin highlight groups using hl_group_set event
  redraw.msg_clear
  redraw.msg_history_show
  redraw.msg_ruler
  redraw.msg_set_pos
  redraw.msg_show
  redraw.msg_showcmd
  redraw.msg_showmode
  redraw.win_close
  redraw.win_external_pos
  redraw.win_float_pos
  redraw.win_hide
  redraw.win_pos

API
f5c56f0 #9170 API/Lua: nvim_buf_attach: support Lua callback
82d48c0 #9896 API: emit nvim_error_event on failed async request
b9ad12e #9992 UI/nvim_ui_attach(): add `override` option
3d1ed7c #9993 UI/ext_messages: learn more message kinds
8ed54bb #9547 proper multiline error message for rpcrequest, API wrappers

Lua
This release introduces "Nvim-Lua standard library". See ":help lua-stdlib".
89d7e24 #9463 Lua stdlib: vim.inspect, string functions
8e941c5 #9740 Lua: generate documentation from docstrings
1cbe014 #9301 lua/stdlib: Introduce vim.shared
c83926c #10123 Lua: introduce vim.loop (expose libuv event-loop)
81e1dbc #10120 Lua: vim.schedule(cb)
1f54f68 #10688 Lua: minimal UTF-16 support needed for LSP
6fb0020 #10513 Lua encoding support
    c0993ed Lua: support getting UTF-32 and UTF-16 sizes of replaced text
    b0e2619 Lua: add {old_byte_size} to on_lines buffer change event

UI:
- The Nvim 0.3.4 UI protocol introduced line-based updates instead of
  legacy char-based updates. Nvim 0.4 continues to evolve the UI
  protocol. See ":help ui". Legacy UI clients are supported. See
  ":help api-contract".
9a1675b #6619 Floating windows
  - Can be (re)positioned, anchored, external.
  - Are real windows showing real buffers. No shortcuts, hacks, or compromises.
  - Support all features and API of normal windows, plus more.
6427894 #8455 Multigrid: "windows drawn on separate grids"
  - Windows are logically isolated internally.
  - Windows are sent to UIs as distinct objects, so that UIs can control
    layout instead of being stuck with the classic TUI layout.
  - Per-window font-size, dimenions, line-spacing.
  - Compositor: Internal subsystem for composing grids.
3855204 #6917 UIEnter, UILeave
788bcbb #9923 ui: ":syn blend=", 'winblend'
7cf7c0a #9575 ui: 'redrawdebug' option for flexible debugging of redrawing
5c836d2 #9607 wildoptions=pum (enabled by default)
37f8df8 #9571 UI: 'pumblend' option for semi-transparent popupmenu
c403a95 #9446 Visual: highlight char-at-cursor
  - Traditionally Vim's visual selection does "reverse mode", which
    perhaps conflicts with the non-blinking block cursor. But
    'guicursor' defaults to a vertical bar for selection=exclusive, and
    this confuses users who expect to see the text highlighted.

:terminal
fc27dc9 #8550 autocmds: TermEnter, TermLeave
d13803f #9810 keymap, terminal: more keycodes
3b56f59 #9535 :terminal : Fix F1-F4 key codes
2d4a37e #10370 :ls : show "R", "F" for terminal-jobs
fd0fd75 #9966 terminal: swap priority of terminal, editor highlights
7bb858c #9494 libvterm 0.1

TUI
3afb397 syntax, TUI: support "strikethrough"
ccbcd39 #9408 TUI: "title stacking" unconditionally
298608f #9509 TUI: detect background color, set bg=dark/light
42f492a #9097 TUI: handle Smulx extension capability (extended underline)
424ddd0 #10205 TUI: support rgba background detection
9b43832 #9601 TUI: italics in tmux, Terminal.app
f6fb370 #9793 keymap: support more (keypad) keycodes
3340e08 #9423 TUI: Konsole DECSCUSR fixup

:checkhealth
d0fd66b health/provider.vim: check curl HTTPS support
c38862a #10490 checkhealth: try yarn if npm is missing
43356a4 #9929 health: check if tmux enabled true colors
ec5a4d8 #9548 checkhealth: validate locale

providers (clipboard, python, etc.):
96be8a2 #10161 Allow reloading providers (useful for UIs/clients)
db3c797 #9487 provider: improve error message if provider is missing

Various:
36762a0 #9295 signs: support multiple columns
801fe79 #10382 eval: wait() (wait for any condition)
9df3a67 #10400 MsgArea highlight; message grid
a9bea8c #10790 keymap: allow modifiers to multibyte chars, like <m-ä>
25e0a44 #10878 #4448 paste: redesign (10x+ faster pasting; extensible vim.paste Lua hook)
ef5037e #9706 autocmd: introduce "++once" feature
175398f #9616 add CompleteChanged autocmd
7fcf2f9 #9717 TextYankPost: add v:event["inclusive"]
3a699a7 #8364 termdebug.vim plugin
ca1ce59 #9709 performance: use os_copy to create backups
ed0e96c man.vim: set 'linebreak'
70f6939 #9564 events: add "Signal" event
f89d0d8 #9568 inccommand: auto-disable if folding is slow

FIXES:

41bb68b #10584 process_stop: uv: do not close stdin first/explicitly
e50aa2a #10117 normal: Don't exit CTRL-O mode after processing K_EVENT
95fa71c #9504 :recover : Fix crash on non-existent *.swp
5a836d4 #9507 screen: don't unconditionally clear messages on window scroll
149dcbf #10021 channel: refactor events, prevent recursive invocation of events
d19ff73 #10107 Fix multiple c_CTRL-D showing statusline
b65a7b7 #10103 Fix wildmode=list,full and display+=msgsep interaction
0be6d3c #9634 fsync: Ignore ENOTSUP. Fix writing to SMB.
b247c6f #10025 kbtree: pointer UB and unitialized value fixes
018e0d5 #9643 API/buffer-updates: always detach on buf-reload
400ee59 #9961 API: fix cursor position when lines are added
769f44e #9911 win/defaults: Use "…/nvim-data/site" in 'runtimepath'
83d5716 #9911 spellfile.vim: store files in stdpath('data')
8dbf231 #9887 RPC: conform message-id type to msgpack-RPC spec
5f996e3 #9894 options: properly reset directories on 'autochdir'
4c4a570 #9807 various CursorMoved fixes
943bedf #9853 event-loop: do not set CA_COMMAND_BUSY
9d207fd #9693 dictwatcheradd(): support b:changedtick
2d50bf3 #9789 mac: fix locale detection
c563133 #9754 :mksession : restore tab-local working directories
092e7e6 #9703 #9703 executable(): return false if user is not owner
11a481f #9686 env var fixes/improvements
8e54847 #9666 #7920 os/env: Fix completion of multibyte env var names
5193826 #10468 Fix is_executable_in_path() on Windows
8eaa452 #9516 win: exepath(), executable() fixes
f55c1e4 #10544 reltimefloat(): allow negative result
b08dc3e #10561 win: jobstart(), system(): $PATHEXT-resolve exe
7cc2b72 #10392 TextYankPost: spurious/too-early dispatch during delete
6e01ed6 OpenBSD: stop jobs/processes properly
58dd5fc #10522 jobstop(): close channel before process_stop()
8363202 #10959 improved resize behavior (all UIs)
c6eb1f4 #10830 API: fix nvim_command_output buffer overflow
cbfd18c #10763 startup: handle 'guicursor' after user config
b8f2436 #10915 jobwait(): fix race if job exits quickly
2fafed6 #10765 clipboard: handle/avoid SIGTERM with previous owner
8aca932 #9954 clipboard: setreg("*") with clipboard=unnamed
3f10c5b #9480 performance: clipboard/macOS: assume that pbcopy works
48efafc #10398 screen: disable redrawing inside VimResized
5e4b93a #10389 API/Lua: make nvim_execute_lua use native lua floats, not special tables
8c6f5b7 #9934 Spurious quote mark in command line when typing <C-R>
a8a38f3 Lua 5.2/5.3 compat

:terminal
47b7b47 #10700 :terminal : update buffer when switching tabpage
5225c1e #9605 terminal: Fix potential invalid local 'scrollback'
894f6be #8325 :terminal : set topline based on window height
8171e96 #9551 Improve :terminal resize
d928b03 #9856 :stopinsert should leave terminal-mode
3f71218 #9926 :terminal : fix: Using `:stopinsert` while in normal mode
5020daa #9883 ui/terminal: make terminal state redraw like any other state

TUI:
9f19e8d #9443 TUI: Do not disable BCE for builtin terminfos
a4076e5 #9474 win/TUI: fix text overrides line numbers
533d4a3 #9645 TUI: do not resize host-terminal on startup
b51e5d8 #9688 tui_tk_ti_getstr: handle weird value
1f5eac1 #10785 TUI: fix data-race during resize

CHANGES:

9697c7f #8194 fix menu_get()
7f2e43c #9520 improve Lua error messages
c234318 #9526 Remove jemalloc
baf93d9 #9581 UI: always use concrete colors for default_colors_set
91688b4 #9563 defaults: set 'scrollback' to -1 by default
bb24fec #10136 defaults: exclude "S" from 'shortmess'
ddd0eb6 #8540 startup: -es/-Es (silent/batch mode): skip swapfile
3536249 #9805 jumplist: avoid extra tail entry
939d905 #10573 channels: reflect exit due to signals in exit status code
45c34bd #10689 :doautocmd : Never show "No matching autocommands"
fb19aee #9110 API: make nvim_win_set_option() set window-global, not buffer-local
abfc8b3 #10778 emsg_multiline: log Vim errors
06d9cc7 #10657 exists("$FOO"): return false for empty env var
6616d1d #10743 win/env: Vim-compat: Empty string deletes env var
7d66483 #10662 win: expand nested env var in $HOME
2816bc8 #8349 edit.c: Disable indent during completion
58f505d #9829 startup: remove TUI init special-case
    Historically Vim/Nvim does backflips to handle input and show messages
    before a UI is available. This logical contradiction was already fixed
    for remote UIs (#9024 c236e80). Fixing it also for the TUI avoids
    problems on Windows, simplifies the logic, and avoids races like #9959.