I do not like the idea of placing modules into scripts/, name of the directory implies that it contains scripts. So far such things were only in test/ and src/nvim/, but first is for tests and second only contains various lists (like list of commands) ATM, so if you think that e.g. src/nvim/api/ is not fine feel free to create some new directory.

agreed, somewhere in src/ would be good

Well it is a library used by scripts/ and unlike dispatch_deprecated.lua (which actually defines stuff exposed in the nvim binary, and thus belong in src/) it is not really api source. As you mention test/ contains not only tests but also libraries for tests, is this not analogous?

so maybe scripts/helpers.lua ?

@bfredl test/ contains libraries for tests and tests, but both can’t be run on their own (tests are only runnable by busted which is a whole bunch of other lua stuff). Also libraries for tests are named “smth.lua”, tests are named “smth_spec.lua”: easily distinguishable. And all are lua, non-lua things have separate directories.

Neither distinguishable naming nor grouping by “scripts are lua and library for scripts is lua” applies here. Last, but not least: I would not expect directory named scripts/ contain any non-scripts, just like bin/ containing something which is not runnable from the command-line, not even any directory with some stuff; ls -lA {,/usr{,/local}}/bin/*(ND-^*) shows that Gentoo maintainers agree with me here: found only .keep and a bunch of broken symlinks.

@bfredl I do not like the idea of invoking anything with top-level Makefile, especially since now even make clint is going to pull in dependencies. And if I am debugging e.g. declarations generator I would not want to run it on all files, so gen_xx is immediately out even if available from build/Makefile, defined in src/nvim/CMakeLists.txt and only proxied from top-level Makefile like most other its targets are. Additionally where generated files are placed is much of a tab-completion waste then extra bin: Vim :e is nearly not as good at completion as zsh, manually specifying the target file to a reachable location is better.

I'm talking about sensible defaults; how these commands normally are invoked, it doesn't prohibit anyone in an edge case to construct​ the cmdline themselves (possibly by modifying one from verbose make gen_xx, if from top-level or build/ is always up to the individual developer and completely besides the point)

Maybe that's better, but wouldn't paths.lua then be need to be found by some mechanism? At which point the mechanism can be used to find the paths directly...

This is more about how many data would be fine to pass through that mechanism. E.g. better make user specify one path to build/ then both path to build/ and pass to source.

I guess I'm too used to languages where including code from the same directory as the code is a thing you just do...

Do not remember using such languages in Neovim, or them being common in my experience:

• Python is directed by sys.path for which you need things like pip install --editable . (or $PYTHONPATH). There is from .foo import smth, but this is only going to work in imported modules which requires them being in sys.path.
• Shell requires hacks with $0, $_ or $BASH_SOURCE.
• lua needs to adjust package.path and package.cpath via e.g. $LUA_INIT (and it is really the worst: most verbose, needs escaping, inconsistent with others which have $*PATH).
• C needs supplying -I or -isystem argument to compiler. Did not really do modules with Perl, but AFAIK it has . in @INC, but not something to run from the codes directory and not current. @INC may be adjusted with -I argument.
• CMake includes only from CMAKE_MODULE_PATH, though it is the closest: you may alternatively use include(cmake/Foo.cmake) and it would be relative to where CMakeLists.txt is located (including when you use include() from cmake/Foo.cmake, so it is not “including code from the same directory as the code is”).

I'm talking about sensible defaults; how these commands normally are invoked, it doesn't prohibit anyone in an edge case to construct​ the cmdline themselves (possibly by modifying one from verbose make gen_xx, if from top-level or build/ is always up to the individual developer and completely besides the point)

I have no idea why I would need to invoke generators separately from the build if not for debugging them. But for debugging this is not really needed because CMake puts generated files first, so one make and I see bugs in generators and need to run them manually. And also CMake is nice enough to know (because generators are specified in DEPENDS) when generators need to be rerun, so this covers “generator is in any case invoked only once” case where there is no difference in amount of waiting done in manually vs via cmake cases.

And top-level or build/ is not besides the point, getting actual commands out of the generated Makefile is harder then from top-level which is written by developers. And they tend to be overly verbose: after make VERBOSE=1 output between highlighted “Generating” and highlighted “Scanning dependencies of target nvim” is unhighlighted, wrapped by the terminal and is literally this:

cd /home/zyx/a.a/Proj/c/neovim/build/src/nvim && /usr/bin/clang /home/zyx/a.a/Proj/c/neovim/src/nvim/fileio.c -E -o /home/zyx/a.a/Proj/c/neovim/build/src/nvim/auto/fileio.i -D_GNU_SOURCE -DMIN_LOG_LEVEL=0 -DDO_NOT_DEFINE_EMPTY_ATTRIBUTES -DEXITFREE -I/home/zyx/a.a/Proj/c/neovim/build/config -I/home/zyx/a.a/Proj/c/neovim/src -I/usr/include -I/home/zyx/a.a/Proj/c/neovim/build/src/nvim/auto -I/home/zyx/a.a/Proj/c/neovim/build/include -g -O0 -g -DEXITFREE -Wconversion -O0 -g -DEXITFREE -Wconversion
cd /home/zyx/a.a/Proj/c/neovim/build/src/nvim && /home/zyx/.local-luajit/bin/luajit /home/zyx/a.a/Proj/c/neovim/scripts/gendeclarations.lua /home/zyx/a.a/Proj/c/neovim/src/nvim/fileio.c /home/zyx/a.a/Proj/c/neovim/build/src/nvim/auto/fileio.c.generated.h /home/zyx/a.a/Proj/c/neovim/build/include/fileio.h.generated.h /home/zyx/a.a/Proj/c/neovim/build/src/nvim/auto/fileio.i
cd /home/zyx/a.a/Proj/c/neovim/build && /home/zyx/.local-cmake-v2.8.7/bin/cmake -E cmake_depends "Unix Makefiles" /home/zyx/a.a/Proj/c/neovim /home/zyx/a.a/Proj/c/neovim/src/nvim /home/zyx/a.a/Proj/c/neovim/build /home/zyx/a.a/Proj/c/neovim/build/src/nvim /home/zyx/a.a/Proj/c/neovim/build/src/nvim/CMakeFiles/nvim.dir/DependInfo.cmake --color=
Dependee "/home/zyx/a.a/Proj/c/neovim/src/nvim/fileio.c" is newer than depender "src/nvim/CMakeFiles/nvim.dir/fileio.c.o".
Clearing dependencies in "/home/zyx/a.a/Proj/c/neovim/build/src/nvim/CMakeFiles/nvim.dir/depend.make".

Try to find generator invocation here. It is, of course, possible, but if that was written in top-level (not proxied) it would miss cd and use relative paths which cuts down text a lot. Though you would have troubles telling CMake invoke it, so such entry point would be for debugging purposes only.

I just meant when debugging, the starting point would be to invoke exactly the same way as the build files does.

And they tend to be overly verbose: after make VERBOSE=1 output

Hmm, I litteraly get 6 short lines before entering build/ (of course more if cmake needed rerun), zero lines after. And grepping for gendispactch gives:

[102/260] cd /home/bjorn/dev/neovim/build/src/nvim && /home/bjorn/dev/neovim/.deps/usr/bin/luajit /home/bjorn/dev/neovim/scripts/gendispatch.lua /home/bjorn/dev/neovim/src/nvim /home/bjorn/dev/neovim/src/nvim/api/buffer.h /home/bjorn/dev/neovim/src/nvim/api/tabpage.h /home/bjorn/dev/neovim/src/nvim/api/ui.h /home/bjorn/dev/neovim/src/nvim/api/vim.h /home/bjorn/dev/neovim/src/nvim/api/window.h /home/bjorn/dev/neovim/build/include/api/buffer.h.generated.h /home/bjorn/dev/neovim/build/include/api/tabpage.h.generated.h /home/bjorn/dev/neovim/build/include/api/ui.h.generated.h /home/bjorn/dev/neovim/build/include/api/vim.h.generated.h /home/bjorn/dev/neovim/build/include/api/window.h.generated.h /home/bjorn/dev/neovim/build/src/nvim/auto/api/private/dispatch_wrappers.generated.h /home/bjorn/dev/neovim/build/src/nvim/auto/api/private/funcs_metadata.generated.h /home/bjorn/dev/neovim/build/api_metadata.mpack

it seems readable enough, I don't see how it should be more or less verbose. But yeah individual make gen_xx would probably be overblow, make gen or make generators is probably enough. Generators depend on each other anyway, and rerunning just the needed generators will be alot faster than also rerunnning gcc (which could be a lot due to header dependencies). I find it rerunning cmake when needed practical, but again, no one is forced to use top-level short-cuts just because they exist.

This is still in scripts.