Skip to content

E315: ml_get: invalid lnum when writing to buffer in no windows using python API #4153

@puremourning

Description

@puremourning

Basic problem

  • It's possible to trigger the above E315 using the Python API when writing to a buffer which is not currently in a window.
  • This happens when the cursor of the current window is outside what would be the bounds of the buffer written.
  • This appears to be because of an invalid assumption in SetBufferLineList in if_py_both.h

Steps to reproduce

Open the following file using vim -Nu NONE test.vim:

pyx << EOF

buffers = {}

def SetUpHiddenBuffer( buf, name ):
  buf.options[ 'buftype' ] = 'nofile'
  buf.options[ 'swapfile' ] = False
  buf.options[ 'buflisted' ] = False
  buf.options[ 'bufhidden' ] = 'hide'
  buf.name = name

def SetUpTheTest( ):
  vim.command( 'new' )
  buffers[ 'hidden' ] = vim.current.buffer
  SetUpHiddenBuffer( buffers[ 'hidden' ], 'hidden' )

  vim.command( 'enew' )
  buffers[ 'visible' ] = vim.current.buffer
  SetUpHiddenBuffer( buffers[ 'visible' ], 'visible' )

  for i in range( 1, 1000 ):
    buffers[ 'visible' ].append( str( i ) )

  vim.command( 'normal G' )

def ClearDownTheTest():
  if 'hidden' in buffers:
    vim.command( 'bwipe! hidden' )
    del buffers[ 'hidden' ]

  if 'visible' in buffers:
    vim.command( 'bwipe! visible' )
    del buffers[ 'hidden' ]

def WriteToBufferNotInWindow():
  buffers[ 'hidden' ][:] = None
  buffers[ 'hidden' ][:] = [
    'test',
  ]

SetUpTheTest()

for i in range( 1, 1000 ):
  WriteToBufferNotInWindow()

ClearDownTheTest()
EOF
  • Source the file with :so %

Explanation

Here's my understanding of the problem, based on stepping through with vimspector:

  • SetBufferLineList in if_py_both.h attempts to switch to a window containing the buffer being written: switch_to_win_for_buf(buf, &save_curwin, &save_curtab, &save_curbuf);
  • This method either switches to the window containing buf if one exists, or if not changes the current buffer curbuf to be buf in the current window
  • switch_to_win_for_buf does not change the current window cursor curwin->w_cursor
  • After applying the changes to the buffer SetBufferLineList tries to correct the cursor if it made it invalid. However, it does this
    ** regardless of whether we switched window or not
    ** before reverting the switching done by swith_to_win_for_buf
	if (buf == curbuf)
	    py_fix_cursor((linenr_T)lo, (linenr_T)hi, (linenr_T)extra);

	/* END of region without "return". */
	restore_win_for_buf(save_curwin, save_curtab, &save_curbuf);

This leads to the cursor being inspected having no relation to the buffer being edited, and hence the py_fix_cursor function hitting the invalid lnum trying to get the line in the edited buffer based on a cursor from some arbitrary buffer.

So, the test case:

  • Creates a buffer and 'hides' it (i.e. not visible in any window)
  • Writes 1000 lines to another buffer (in the current window) and puts the cursor at the 1000th line
  • Writes 1 line to the (empty) 'hidden' buffer.

This triggers the problem reliably because the current window cursor's lnum is 1000, but the buffer being written only has 1 line.

Possible fix

It's not clear to me why we are checking if ( buf == curbuf ) before fixing the snippet, but a possible fix might be if ( buf == curbuf && ! save_curbuf.br_buf ) - i.e. only fix the cursor if we didn't switch buffers in the current window.

Alternatively, doing the cursor fix after calling restore_win_for_buf might work.

I'm not submitting a PR to fix this (yet.. I can) as I'm not sure of the knock-on effects of either of the above changes.

Additional info

This isn't really all that relevant, but I thought I would share: It's taken me ages (months) to work out a minimal, repeatable test for this. I was frequently hitting it when vimspector was writing log data to a hidden buffer. Vimspector uses a channel to talk to the debug adapters and writes output events to buffers that may or may not be visible, so this error is frequently hit making it frustrating to use. Though, I did use vimspector to debug vim to find this problem.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions