[RDY] new highlight group Whitespace for 'listchars' #6367

zhou13 · 2017-03-26T10:24:13Z

This pull request creates a new highlight group Whitespace in 'listchars' for "nbsp", "space", "tab" and "trail". The reason is that "SpecialKey" is designed to show the special characters such as ^M and <M-x>, which is usually rendered with a high-contrast color. On the other hand, users usually wish to use a low-contrast color to show the whitespace.

justinmk · 2017-03-26T17:17:42Z

src/nvim/syntax.c

@@ -5912,6 +5912,7 @@ static char *highlight_init_both[] =
  "default link EndOfBuffer NonText",
  "default link QuickFixLine Search",
  "default link Substitute Search",
+  "default link WhiteSpace NonText",


shouldn't this link to SpecialKey?

Link to SpecialKey gives maximum compatibity to VIM. But I think it is more reasonable to render whitesace in the same color as EOL as most other editors do by default. Maybe this is just my personal taste. If someone prefers using SpecialKey, I could change it back.

Doesn't matter to me. We can change it if anyone complains.

I agree with @zhou13 even if someone complains. I never will think "I want to see invisible characters, but I want them to be highlighted to look like editable, opaque text because I want to confuse myself and I also want :map to be difficult to read."

justinmk · 2017-03-26T17:21:10Z

src/nvim/globals.h

@@ -481,7 +482,7 @@ typedef enum {
 #define HL_FLAGS { '8', '~', 'z', 'Z', '@', 'd', 'e', 'i', 'l', 'm', 'M', 'n', \
                   'N', 'r', 's', 'S', 'c', 't', 'v', 'V', 'w', 'W', 'f', 'F', \
                   'A', 'C', 'D', 'T', '-', '>', 'B', 'P', 'R', 'L', '+', '=', \
-                   'x', 'X', '*', '#', '_', '!', '.', 'o', 'q' }
+                   'x', 'X', '*', '#', '_', '!', '.', 'o', 'q', 'H'}


maybe 0 is a closer mnemonic?

justinmk · 2017-03-26T17:25:56Z

I've no objections. Might want to wait for objections/suggestions from others before writing tests.

"whitespace" in technical parlance is one word, so the name should be Whitespace.

test/functional/ui/quickfix_spec.lua tests "QuickFixLine" highlight group which might be a good starting point. Tests should go in the describe('Default highlight groups' block of test/functional/ui/highlight_spec.lua.

tweekmonster · 2017-03-28T02:25:43Z

runtime/doc/vim_diff.txt

@@ -245,6 +245,10 @@ coerced to strings. See |id()| for more details, currently it uses

 |c_CTRL-R| pasting a non-special register into |cmdline| omits the last <CR>.

+"nbsp", "space", "tab" and "trail" in 'listchars' are now highlighted in a new
+group |hl-WhiteSpace| rather than ||hl-SpecialKey|.  By default,


There's an extra | here.

tweekmonster · 2017-03-28T02:36:01Z

src/nvim/syntax.c

@@ -5912,6 +5912,7 @@ static char *highlight_init_both[] =
  "default link EndOfBuffer NonText",
  "default link QuickFixLine Search",
  "default link Substitute Search",
+  "default link WhiteSpace NonText",


I agree with @zhou13 even if someone complains. I never will think "I want to see invisible characters, but I want them to be highlighted to look like editable, opaque text because I want to confuse myself and I also want :map to be difficult to read."

justinmk · 2017-03-30T23:13:13Z

LGTM after a test is added.

justinmk · 2017-04-03T12:26:26Z

Merged. Thanks @zhou13

adriaanzon · 2017-04-06T16:24:09Z

Should this link to SpecialKey by default? A lot of existing color schemes intend to highlight the listchars when specifying a color for SpecialKey.

zhou13 · 2017-04-06T22:33:51Z

@adriaanzon Please provide the link to the theme that is not working, a screenshot of your :map and a screenshot of a editor with set listchars=eol:¬,tab:»\ ,extends:>,precedes:<,nbsp:- list

adriaanzon · 2017-04-07T19:48:16Z

I'm using the seoul256 colorscheme.

I ran the following command so the trailing spaces are also highlighted:

set listchars& listchars+=eol:¬ list

Here's a before and after updating neovim:

Before	After

The :map colors didn't change. The colors of trail, tab, and nbsp became darker.

zhou13 · 2017-04-07T22:06:33Z

@adriaanzon Thanks for your input. For seoul256, the problem is that it uses a high contrast color for NonText. Although not shown in your screenshot, I think user cannot easily distinguish between normal text and a EOL if they are immediately adjacent, but that maybe just my personal taste.

It seems that for most themes on Internet, the SpecialKey and NonText are the same. So linking to either should work for most people. But these themes make your :map and real special chars difficult to read.

Anyway, some people must put hi link Whitespace XXXX into their vimrc or in their theme. I still hold my opinion that Whitespace should be linked to NonText as most people want the same color for EOL and whitespace and therefore they don't need to change it. New themes need change SpecialKey to a high contrast color for better user experience. The final decision will be made by @justinmk.

justinmk · 2017-04-07T23:23:30Z

@tweekmonster Made the case:

I never think: "I want to see invisible characters, but I want them to be highlighted to look like editable, opaque text because I want to confuse myself and I also want :map to be difficult to read."

But it looks like seoul256 highlights NonText rather boldly. @adriaanzon why is it ok for "eol" to be highlighted boldly but not ok for tab characters? They're both whitespace. It's surprising for a colorscheme to highlight "eol" boldly.

And with the Whitespace highlight it's now possible to make a finer-grained distinction.

adriaanzon · 2017-04-08T00:21:11Z

Yeah, I was actually surprised to see eol highlighted boldly as well, because I have it turned off normally. I only wanted to point out the difference between the before/after of trail, tab, and nbsp.

And with the Whitespace highlight it's now possible to make a finer-grained distinction.

Agreed. I do like the addition; it allows me to highlight the whitespace characters even lighter than before, without changing the highlighting of things like :map.

tweekmonster · 2017-04-08T14:39:38Z

I personally stopped using listchars a long time ago because it made :map hard to read when I used a SpecialKey color that made sense for invisible characters.

I think the use of SpecialKey for listchars is one of those conservatist decisions from upstream because of a strong desire to maximize the reuse of the existing highlights. nbsp, space, tab, and trail are invisible characters that take up screen space. If anything, they should've been using the Conceal highlight.

They're both whitespace.

@justinmk IMO \n is whitespace in a text stream, but not in Vim. In Vim, it's not a character you can select and replace, overtype, etc. It's more of a phantom character. So, I somewhat agree that it should be highlighted differently from invisibles. Definitely not boldly, though.

This is how I define these highlights:

Whitespace: Invisible characters. They can be selected and manipulated. I feel that calling these "unprintable" isn't very correct in terms of screen display, at least not in the way that \a and \r are unprintable. They simply don't have glyphs to display by design and function.
NonText: Assistive text displayed by Vim. They cannot be selected or manipulated. Basically a HUD. In the case of eol, it informs you that the line actually ended and isn't just a string of invisible characters that pushed the rest of the line off screen.
Conceal: Characters hidden by Vim, but made visible with one character.
SpecialKey: Characters translated by Vim because they have an internal representation that only makes sense to Vim (e.g. <Left> is internally \x80kl). <tab> just barely meets this criteria because it has a variable width.

justinmk · 2017-04-08T14:53:34Z

I don't disagree with anything here. Is there anything actionable? Linking to Conceal may cause similar imbalances in some colorschemes, I guess.

So far the only report (re seoul256) doesn't seem like a terrible result.

tweekmonster · 2017-04-08T14:57:47Z

Is there anything actionable?

Not for Neovim. I was commenting for posterity. I guess an issue can be raised for seoul256 to use the Whitespace highlight.

FEATURES: bc4a2e1 help, man.vim: "outline" (TOC) feature neovim#5169 58422f1 'guicursor' works in the TUI (and sends info to UIs) neovim#6423 129f107 api: nvim_get_mode() neovim#6247 0b59f98 api/ui: externalize tabline neovim#6583 bc6d868 'listchars': `Whitespace` highlight group neovim#6367 6afa7d6 writefile() obeys 'fsync' option neovim#6427 c60e409 eval.c refactor (also improves some error messages) neovim#5119 9d200cd getcompletion("cmdline") neovim#6376 2ea7bfc terminal: Support extra arguments in 'shell'. neovim#4504 bf51102 DirChanged autocmd neovim#5928 neovim#6262 1743df8 'cpoptions': "_" flag to toggle `cw` behaviour neovim#6235 22337b1 CTRL-R omits trailing ^M when pasting to cmdline neovim#6137 0e44916 :edit allows unescaped spaces in filename neovim#6119 abdbfd2 eval: Add id() function and make printf("%p") useful neovim#6095 bdfa147 findfile(), :find, gf work in :terminal. neovim#6009 2f38ed1 providers: Disable if `g:loaded_*` exists. b5560a6 setpos() can set lowercase marks in other buffers neovim#5753 7c513d6 Throttle :! output, pulse "..." message. neovim#5396 d2e8c76 v:exiting neovim#5651 :terminal improvements neovim#6185 neovim#6142 - cursor keeps position after leaving insert-mode. - 4ceec30 Follows output only if cursor is at end of buffer. - e7bbd35 new option: 'scrollback' - fedb844 quasi-support for undo and 'modifiable' - b45ddf7 disables 'list' by default - disables 'relativenumber' by default :help now contains full API documentation at `:help api`. man.vim saw numerous improvements. Windows support: - Windows is no longer "experimental", it is fully supported. - Windows package includes a GUI, curl.exe and other utilities. "Vim 8" features: partials, lambdas, packages. FIXES: 12fc1de ops: fix i<c-r> with multi-byte text neovim#6524 dd391bf Windows: system() and friends neovim#6497 13352c0 Windows: os_get_hostname() neovim#6413 16babc6 tui: Less-noisy mouse seqs neovim#6411 3a9dd13 (vim bug) folding edge-cases neovim#6207 f6946c6 job-control: set CLOEXEC on pty processes. neovim#5986 d1afd43 rplugin: Call s:LoadRemotePlugins() on startup. 1215084 backtick-expansion works with `shell=fish` neovim#6224 e32ec03 tui: Improved behavior after resize. neovim#6202 86c2adc edit.c: CTRL-SPC: Insert previously-inserted text. neovim#6090 c318d8e b:changedtick now follows VimL rules neovim#6112 34e24cb terminal: Initialize colors in reverse order neovim#6160 e889917 undo: Don't set b_u_curhead in ex_undojoin() neovim#5869 d25649f undo: :earlier, g-: Set b_u_seq_cur correctly. (neovim#6016) 043d8ba 'Visual-mode put from @. register' neovim#5782 42c922b open_buffer(): Do `BufEnter` for directories. 50d0d89 inccommand: Preview :sub commands only after delimiter neovim#5932 1420e10 CheckHealth improvements neovim#5519 c8d5e92 jobstart(): Return -1 if cmd is not executable. neovim#5671 CHANGES: NVIM_TUI_ENABLE_CURSOR_SHAPE was removed. Use 'guicursor' instead. See https://github.com/neovim/neovim/wiki/Following-HEAD#20170402 81525dc 'mouse=a' is no longer the default. (This will probably change again after it is improved.) neovim#6022 0c1f783 defaults: 'showcmd', 'belloff', 'ruler' neovim#6087 eb0e94f api: {get,set}_option update local options as appropriate neovim#6405 bdcb2a3 "Reading from stdin..." message was removed. neovim#6298

FEATURES: bc4a2e1 help, man.vim: "outline" (TOC) feature neovim#5169 58422f1 'guicursor' works in the TUI (and sends info to UIs) neovim#6423 129f107 api: nvim_get_mode() neovim#6247 0b59f98 api/ui: externalize tabline neovim#6583 bc6d868 'listchars': `Whitespace` highlight group neovim#6367 6afa7d6 writefile() obeys 'fsync' option neovim#6427 c60e409 eval.c refactor (also improves some error messages) neovim#5119 9d200cd getcompletion("cmdline") neovim#6376 2ea7bfc terminal: Support extra arguments in 'shell'. neovim#4504 bf51102 DirChanged autocmd neovim#5928 neovim#6262 1743df8 'cpoptions': "_" flag to toggle `cw` behaviour neovim#6235 22337b1 CTRL-R omits trailing ^M when pasting to cmdline neovim#6137 0e44916 :edit allows unescaped spaces in filename neovim#6119 abdbfd2 eval: Add id() function and make printf("%p") useful neovim#6095 bdfa147 findfile(), :find, gf work in :terminal. neovim#6009 2f38ed1 providers: Disable if `g:loaded_*` exists. b5560a6 setpos() can set lowercase marks in other buffers neovim#5753 7c513d6 Throttle :! output, pulse "..." message. neovim#5396 d2e8c76 v:exiting neovim#5651 :terminal improvements neovim#6185 neovim#6142 - cursor keeps position after leaving insert-mode. - 4ceec30 Follows output only if cursor is at end of buffer. - e7bbd35 new option: 'scrollback' - fedb844 quasi-support for undo and 'modifiable' - b45ddf7 disables 'list' by default - disables 'relativenumber' by default :help now contains full API documentation at `:help api`. man.vim saw numerous improvements. Windows support: - Windows is no longer "experimental", it is fully supported. - Windows package includes a GUI, curl.exe and other utilities. "Vim 8" features: partials, lambdas. SECURITY FIXES: CVE-2017-5953 CVE-2017-6349 CVE-2017-6350 neovim#6485 CHANGES: NVIM_TUI_ENABLE_CURSOR_SHAPE was removed. Use 'guicursor' instead. See https://github.com/neovim/neovim/wiki/Following-HEAD#20170402 81525dc 'mouse=a' is no longer the default. (This will probably change again after it is improved.) neovim#6022 0c1f783 defaults: 'showcmd', 'belloff', 'ruler' neovim#6087 eb0e94f api: {get,set}_option update local options as appropriate neovim#6405 bdcb2a3 "Reading from stdin..." message was removed. neovim#6298 FIXES: 12fc1de ops: fix i<c-r> with multi-byte text neovim#6524 dd391bf Windows: system() and friends neovim#6497 13352c0 Windows: os_get_hostname() neovim#6413 16babc6 tui: Less-noisy mouse seqs neovim#6411 3a9dd13 (vim bug) folding edge-cases neovim#6207 f6946c6 job-control: set CLOEXEC on pty processes. neovim#5986 d1afd43 rplugin: Call s:LoadRemotePlugins() on startup. 1215084 backtick-expansion works with `shell=fish` neovim#6224 e32ec03 tui: Improved behavior after resize. neovim#6202 86c2adc edit.c: CTRL-SPC: Insert previously-inserted text. neovim#6090 c318d8e b:changedtick now follows VimL rules neovim#6112 34e24cb terminal: Initialize colors in reverse order neovim#6160 e889917 undo: Don't set b_u_curhead in ex_undojoin() neovim#5869 d25649f undo: :earlier, g-: Set b_u_seq_cur correctly. (neovim#6016) 043d8ba 'Visual-mode put from @. register' neovim#5782 42c922b open_buffer(): Do `BufEnter` for directories. 50d0d89 inccommand: Preview :sub commands only after delimiter neovim#5932 1420e10 CheckHealth improvements neovim#5519 c8d5e92 jobstart(): Return -1 if cmd is not executable. neovim#5671

Neovim recently added a group `Whitespace` to differentiate from group `SpecialKey` that usually has a high-contrast color. As a result, we lose the highlighting of tabs/trail in Solarized. See neovim/neovim#6367.

xeyownt · 2017-05-28T21:22:11Z

For info, this also breaks vim-colors-solarized. trail and tabs for instance are no longer displayed with the slightly darker background. I fixed this issue in my fork.

I appreciate the possibility to highlight Whitespace and SpecialKey differently, but since Vim/gVim use SpecialKey, NeoVim should by default link Whitespace to SpecialKey since it claims compatibility with most Vim plugins.

Neovim recently added a group `Whitespace` to differentiate from group `SpecialKey` that usually has a high-contrast color. As a result, we lose the highlighting of tabs/trail in Solarized. See neovim/neovim#6367. This is backported from xeyownt/vim-colors-solarized.

This is a Neovim only feature, see neovim/neovim#6367 When using set list In Vim, the whitespace characters would be drawn in the same color as the text which is intrusive IMHO. This commit changes it to use the same color as comments. In order to make this compatible with regular Vim, one would probably have to use `NonText`, instead.

* Introduce color for whitespace markers in Neovim. This is a Neovim only feature, see neovim/neovim#6367 When using set list In Vim, the whitespace characters would be drawn in the same color as the text which is intrusive IMHO. This commit changes it to use the same color as comments. In order to make this compatible with regular Vim, one would probably have to use `NonText`, instead.

marvim added the RFC label Mar 26, 2017

justinmk reviewed Mar 26, 2017

View reviewed changes

zhou13 added 2 commits March 27, 2017 16:38

new highlight group WhiteSpace for 'listchars'

efb64c6

using HLF_0 rather than HLF_WS

3e615be

tweekmonster reviewed Mar 28, 2017

View reviewed changes

WhiteSpace to Whitespace

2c291b2

zhou13 force-pushed the whitespace branch from 6b3d6af to 797f76f Compare March 28, 2017 05:41

zhou13 changed the title ~~[RFC] new highlight group WhiteSpace for 'listchars'~~ [RFC] new highlight group Whitespace for 'listchars' Mar 29, 2017

Fix lint and test cases

cbd6558

zhou13 changed the title ~~[RFC] new highlight group Whitespace for 'listchars'~~ [RDY] new highlight group Whitespace for 'listchars' Mar 31, 2017

Add test cases

c800c7f

zhou13 force-pushed the whitespace branch from fcd11a8 to b3ef0f1 Compare March 31, 2017 05:49

marvim added RDY and removed RFC labels Mar 31, 2017

zhou13 force-pushed the whitespace branch from b3ef0f1 to c800c7f Compare March 31, 2017 08:24

justinmk closed this Apr 3, 2017

justinmk removed the RDY label Apr 3, 2017

justinmk pushed a commit that referenced this pull request Apr 3, 2017

'listchars': Whitespace highlight group #6367

bc6d868

zhou13 deleted the whitespace branch April 7, 2017 21:18

adriaanzon mentioned this pull request Apr 27, 2017

Highlight Neovim's Whitespace group junegunn/seoul256.vim#43

Closed

justinmk mentioned this pull request May 1, 2017

NVIM v0.2.0 #6634

Merged

Vintodrimmer mentioned this pull request Nov 9, 2017

Neovim Whitespace (listchars) color joshdick/onedark.vim#111

Closed

kdungs mentioned this pull request Oct 2, 2020

Introduce color for whitespace markers in Neovim. sonph/onehalf#94

Merged

justinnhli mentioned this pull request Dec 11, 2020

match SpecialKey to Comment instead of NonText cocopon/iceberg.vim#81

Merged

Passw mentioned this pull request May 31, 2023

highlight SpecialKey and NonText vim/vim#12472

Closed

Uh oh!

[RDY] new highlight group Whitespace for 'listchars' #6367

[RDY] new highlight group Whitespace for 'listchars' #6367

Uh oh!

Conversation

zhou13 commented Mar 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justinmk Mar 26, 2017

Choose a reason for hiding this comment

Uh oh!

zhou13 Mar 26, 2017

Choose a reason for hiding this comment

Uh oh!

justinmk Mar 26, 2017

Choose a reason for hiding this comment

Uh oh!

tweekmonster Mar 28, 2017

Choose a reason for hiding this comment

Uh oh!

justinmk Mar 26, 2017

Choose a reason for hiding this comment

Uh oh!

justinmk commented Mar 26, 2017

Uh oh!

tweekmonster Mar 28, 2017

Choose a reason for hiding this comment

Uh oh!

tweekmonster Mar 28, 2017

Choose a reason for hiding this comment

Uh oh!

justinmk commented Mar 30, 2017

Uh oh!

justinmk commented Apr 3, 2017

Uh oh!

adriaanzon commented Apr 6, 2017

Uh oh!

zhou13 commented Apr 6, 2017

Uh oh!

adriaanzon commented Apr 7, 2017

Uh oh!

zhou13 commented Apr 7, 2017

Uh oh!

justinmk commented Apr 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adriaanzon commented Apr 8, 2017

Uh oh!

tweekmonster commented Apr 8, 2017

Uh oh!

justinmk commented Apr 8, 2017

Uh oh!

tweekmonster commented Apr 8, 2017

Uh oh!

xeyownt commented May 28, 2017

Uh oh!

Uh oh!

zhou13 commented Mar 26, 2017 •

edited

Loading

justinmk commented Apr 7, 2017 •

edited

Loading