VimL (Vimscript) to Lua translator #243
Conversation
|
Ref #170. |
|
@tarruda Since I am taking code from eval.c and other vim files I do not like to use bison. Also note that lexer can only be used with expressions (there are no other cross-command entities) and expressions parser is already written and working. |
| static void print_node(int indent, ExpressionNode *node) | ||
| { | ||
| char *name = NULL; | ||
| switch (node->type) { |
There was a problem hiding this comment.
I'm unclear on how much of this is WIP and how much isn't, but I'm uncomfortable with how liberally this switch relies on fall-through. It's, like, 40 cases.
There was a problem hiding this comment.
Everything after 1185 is temporary I believe
There was a problem hiding this comment.
And it is prefixed with //FIXME!!! and commit message says that after FIXME everything is to be removed. Don't bother, it was the quickest way to get what I need: some code for manual parser testing.
02.03.14, 12:23, "commonquail" notifications@github.com":
In src/expr.c:> + puts((char *) error.message);
- puts((char *) error.position);
- free_node(result);
- return NULL;
- }
- return result;
+}
+//FIXME!!!
+#ifdef COMPILE_TEST_VERSION
+#include <stdio.h>
+static void print_node(int indent, ExpressionNode *node)
+{
- char *name = NULL;
- switch (node->type) {
I'm unclear on how much of this is WIP and how much isn't, but I'm uncomfortable with how liberally this switch relies on fall-through. It's, like, 40 cases.
—
Reply to this email directly or view it on GitHub.
|
In parse2() and parse3() the node types are switched. |
| ExpressionNode *parse0_err(char_u *arg, ExpressionParserError *error) | ||
| { | ||
| ExpressionNode *result = NULL; | ||
| char_u *p = arg; |
There was a problem hiding this comment.
Why initialize p to arg here, when it's initialized again later before it is used.
| } | ||
|
|
||
| /* | ||
| * Return 5 if "p" starts with "<SID>" or "<SNR>" (ignoring case). |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
Fantastic work, I didn't actually understand all of it on a quick readthrough, but I can see it's impressive! |
|
Ref #387 |
|
It's amazing to see such quick progress and our donations being put to good use. I am so excited for this you all have no idea. 😄 |
As far as I know, the donations are only used to pay for @tarruda 's work on neovim (that's all the features in his branches + management), so the (indeed amazing as well) work @ZyX-I has been doing is entirely volunteer, unless people have been talking in private channels for sharing donations. So I think an applause and some props is in order. Great work @ZyX-I ! Let's keep this issue on topic now though ;) |
I think that telling someone how awesome they are for their contributions is always on-topic. Thanks @ZyX-I for all your hard work!! 😄 |
|
They cannot be backported. Their functionality will be worth reimplementing, but I have not even started actually creating a translator yet*. If extended-funcref branch and following lambda support branch were merged then there was something to backport. I am also not working on eval.c (and never was, to be precise). It did not contain much parser code, most of it was processed in the first weekend.
11.04.14, 15:17, "Marco Hinz" notifications@github.com":
Sent from Yandex.Mail for mobile: http://m.ya.ru/ymail |
So you'd make a
As I surmise, |
|
@aktau |
|
11.04.14, 17:12, "Nicolas Hillegeer" notifications@github.com":
I would actually like to create uncrustify equivalent: currently I can test whether parser is correct only if I do a huge amount of :%s to make styling be the same. Making test code emit valid VimL was a convenient decision: now I have a simple way to verify whether parsing works correctly.
From this point of view they can. I just thought you were asking whether it makes sense to backport them into this branch. I would say that this depends on the decision whether this branch should be the part of the first release or not. If it should it does not make sense. If it should not it makes sense. Note that as I can work on this branch only at weekends or during summer (or when I am officially ill so do not have to work like two or three weeks ago) it is unlikely to be finished until summer.
Sent from Yandex.Mail for mobile: http://m.ya.ru/ymail |
@ZyX-I understood. I'm not even sure what the release policy is (I believe it might be a bit early, we still have a lot of churn going on). As for when to incorporate luaviml: as soon as possible, and it doesn't need to be perfect. The catch is that we'll put it behind a runtime "experimental" switch, so the adventurous can turn it on and report bugs. With that in mind, I'm sure we can patch the current |
|
Putting it in a nutshell: there should be no problem just backporting all missing Vim patches and if parts of them get removed in the first release, so be it. Perhaps that's even better then forgetting to include certain important Vim patches that won't be removed over the course of luaviml development or, well, any other ideas/refactorings. |
|
Rebased and squashed everything into one commit. It appears that I should not have spent a few hours trying to rebase properly because git has thrown away all the work done during merges it could throw. Editing existing rebased commits so that they make any sense will require another few hours, so I just squashed everything. |
|
@ZyX-I, first of: great work! A squash might've been the right idea with the incredible amount of commits you had. Second: aren't vim variables always |
|
@aktau Vim numbers were already decided to be lua numbers. I remember the discussion, though not sure whether I expressed explicitly the result. Problem is not numbers, problem is buffer/window/etc identifiers: they are |
|
@ZyX-I I don't know if Lua does any bit wrangling, but double and uint64_t fit in the same space and it's not like they're used for printing and/or calculating, so perhaps just passing it will be fine. |
|
@aktau You mean using union hack/ |
|
As far as I know, a union "hack" is perfectly legal C though (which is why I used it for the high resolution timer / profiling code that interacts with VimL). The undefined behaviour is when you just plain cast (through void or whatever). You see the union type punning in a lot of codebases as well (Windows (QPC), Linux, ...). I find it rather clean, as long as you only perform operations on one of the "aliases". |
Reasons: - One does not have to do `s[len] = NUL` to work with these functions if they do not need to replace the whole string: thus `s` may be const. - One does not have to save/restore p_cpo to work with them.
It is like findoption(), but works with non-NUL-terminated strings.
Code that expected NUL-terminated strings allowed them and this behaviour is actually used.
Progress so far:
- All of the commands are covered by parsers.
- Almost no commands are supported by translator (mainly supporting block
(:while/:if/:for/:try)).
- No commands use Vim API.
- No developer documentation.
- Existing implementation was not replaced.
- Top-level parser was created.
- Top-level translator was created, most of commands not covered by translator
are expected to work automatically once underlying implementation is added.
- Lua bindings have incomplete types support, and complete operator and
assignment support.
- Most of used functions that are to be documented are documented.
- Lua bindings do not support integers that do not fit into float: luajit
appears to have integers, but not visible C function for creating them, lua
5.3 integers support was not tested.
Some notes:
- GCC may emit declarations like the following:
static size_t
# 42 "this/file.c"
function(args)
for code like this:
static FDEF(function)
. Declaration generator script does not expect `# …` inside declarations. This
commit makes it treat such things as comments.
- Integer division is rather hacky: math.floor(15 / 7) will return 2 (same as
integer division), but math.floor(-15 / 7) will return -3, while integer
division returns -2.
Thus I use math.floor() on absolute value and then apply sign.
- According to :h expr-/ division by zero must have always the same results for
Numbers:
> When dividing a Number by zero the result depends on the value:
> 0 / 0 = -0x80000000 (like NaN for Float)
> >0 / 0 = 0x7fffffff (like positive infinity)
> <0 / 0 = -0x7fffffff (like negative infinity)
> (before Vim 7.2 it was always 0x7fffffff)
- According to :h expr-%
> When the righthand side of '%' is zero, the result is 0.
- Modulo with negative numbers behaves different in Vim:
n1 | n2 | n1 % n2 (Vim) | n1 % n2 (lua) | Difference
-- | -- | ------------- | ------------- | ----------
1 | 2 | 1 | 1 | absent
-1 | 2 | -1 | 1 | -+
1 | -2 | 1 | -1 | +-
-1 | -2 | -1 | -1 | absent
I do not see this behavior mentioned in help though.
- Function properties recorded in ephemeron table (so I do not have to care
about GC’ing them):
1. Function name.
2. Starting position (line number and column) of the first child of :function.
3. Position of the character just before `:endfunction` command.
4. Function arguments (stringified version).
5. `dict`, `abort` and `range` attributes.
6. `state.code` and `state.fname`.
Reasoning:
- 1 till 6 are for `:function Func` implementation. It looks like this:
function {1}({4}){" abort" if 5.abort}{" range" if 5.range}{" dict" if 5.dict}
{" Last set from "..6.fname if verbose and 6.fname:sub(1, 1) ~= '<'}
1 {6.code[2.line]:sub(2.column, -1)}
… {6.code[2.line + 1] till 6.code[3.line - 1]}
N {6.code[3.line]:sub(1, 3.column)}
endfunction
. More understandable example:
function d.Abc(a, b, ...) range
Last set from ~/foo.vim
1 echo a:a
2 echo a:b
3 return a:000
endfunction
- 1. is for use in error messages (except for E118/E119 where exactly the same
value is embedded by the translator without requiring to use value from
ephemeron table).
- 5.dict is for raising E725 (calling dict function without a dictionary).
- Printer allows overriding written newlines because to proceed with first stage
of neovim#1536 it is needed to supply old parser correct `fgetline` function. But it
may appear that dumped newline is actually part of the command line (e.g. when
parsing `:execute "normal iabc\ndef"`). To distinguish newlines between
commands with in-command ones ability to override in-command newline with some
string that contains NUL is needed (if string does *not* contain NUL it is
still possible for it to appear inside `normal` command).
Replaces a number of arguments with two structures: CommandParserState and CommandParserResult. Code for saving highlighting regions was temporary removed.
Fix for other style errors in src/nvim/viml was integrated in previous commits. For this file it is not easy to do due to the refactoring.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
StatusChecklist shows that the VimL-to-Lua layer is implemented in this PR. But it was not merged because of performance concerns.
Locked to keep the summary visible. You can always chat or open a ticket if you have new information/topics to discuss. |
This is the current work on VimL to lua translator.
TODO:
:*docommand family (argdo,bufdo, etc):editcommand family:s,:g, …)set_one_cmd_context:*docommand family (argdo,bufdo, etc):editcommand family:s,:g, …):ifblock:whileblock:forblock:tryblockbreakandcontinue:letassignment:letslice assignment:functiondefinitions:letmodifying assignment (+=,-=,.=):unlet:delfunction:commanddefinitions:*do,:e +cmd, …):ifblock:whileblock:forblock:tryblockbreakandcontinue:letassignment:letslice assignment:functiondefinitions:letmodifying assignment (+=,-=,.=):unlet:delfunction:commanddefinitions:*do,:e +cmd, …)+-*/%)[key]dictionary, list, string and number subscripting[idx1:idx2]list, string and number slicing(args)function callstring(),call(),type(),copy(),deepcopy()function():functiondefinitionsabortmodifier+-*/%)[key]dictionary, list, string and number subscripting[idx1:idx2]list, string and number slicing(args)function callstring(),copy(),deepcopy()function():functiondefinitionsabortmodifier