To: vim_dev@googlegroups.com Subject: Patch 8.2.2204 Fcc: outbox From: Bram Moolenaar Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ------------ Patch 8.2.2204 Problem: Vim9: using -> both for method and lambda is confusing. Solution: Use => for lambda in :def function. Files: runtime/doc/vim9.txt, src/vim9compile.c, src/userfunc.c, src/testdir/test_vim9_expr.vim *** ../vim-8.2.2203/runtime/doc/vim9.txt 2020-12-04 19:11:53.877306973 +0100 --- runtime/doc/vim9.txt 2020-12-24 14:51:56.195539878 +0100 *************** *** 6,12 **** THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE ! Vim9 script commands and expressions. *vim9* Most expression help is in |eval.txt|. This file is about the new syntax and features in Vim9 script. --- 6,12 ---- THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE ! Vim9 script commands and expressions. *Vim9* Most expression help is in |eval.txt|. This file is about the new syntax and features in Vim9 script. *************** *** 123,130 **** Many errors are already found when compiling, before the function is executed. The syntax is strict, to enforce code that is easy to read and understand. ! Compilation is done when: ! - the function is first called - when the `:defcompile` command is encountered in the script where the function was defined - `:disassemble` is used for the function. --- 123,130 ---- Many errors are already found when compiling, before the function is executed. The syntax is strict, to enforce code that is easy to read and understand. ! Compilation is done when either of these is encountered: ! - the first time the function is called - when the `:defcompile` command is encountered in the script where the function was defined - `:disassemble` is used for the function. *************** *** 132,139 **** reference `:def` has no options like `:function` does: "range", "abort", "dict" or ! "closure". A `:def` function always aborts on an error, does not get a range ! passed and cannot be a "dict" function. The argument types and return type need to be specified. The "any" type can be used, type checking will then be done at runtime, like with legacy --- 132,140 ---- reference `:def` has no options like `:function` does: "range", "abort", "dict" or ! "closure". A `:def` function always aborts on an error (unless `:silent!` was ! used for the command or inside a `:try` block), does not get a range passed ! cannot be a "dict" function, and can always be a closure. The argument types and return type need to be specified. The "any" type can be used, type checking will then be done at runtime, like with legacy *************** *** 163,182 **** When using `:function` or `:def` to specify a nested function inside a `:def` function, this nested function is local to the code block it is defined in. ! In a `:def` function it is not possible to define a script-local function. it is possible to define a global function by using the "g:" prefix. When referring to a function and no "s:" or "g:" prefix is used, Vim will search for the function: ! - in the function scope - in the script scope, possibly imported - in the list of global functions However, it is recommended to always use "g:" to refer to a global function for clarity. In all cases the function must be defined before used. That is when it is ! called, when `:defcompile` causes the it to be compiled, or when code that ! calls it is being compiled (to figure out the return type). The result is that functions and variables without a namespace can usually be found in the script, either defined there or imported. Global functions and --- 164,183 ---- When using `:function` or `:def` to specify a nested function inside a `:def` function, this nested function is local to the code block it is defined in. ! In a `:def` function it is not possible to define a script-local function. It is possible to define a global function by using the "g:" prefix. When referring to a function and no "s:" or "g:" prefix is used, Vim will search for the function: ! - in the function scope, in block scopes - in the script scope, possibly imported - in the list of global functions However, it is recommended to always use "g:" to refer to a global function for clarity. In all cases the function must be defined before used. That is when it is ! called, when `:defcompile` causes it to be compiled, or when code that calls ! it is being compiled (to figure out the return type). The result is that functions and variables without a namespace can usually be found in the script, either defined there or imported. Global functions and *************** *** 186,191 **** --- 187,197 ---- Vim9 script script-local functions are defined once when the script is sourced and cannot be deleted or replaced. + When compiling a function and a function call is encountered for a function + that is not (yet) defined, the |FuncUndefined| autocommand is not triggered. + You can use an autoload function if needed, or call a legacy function and have + |FuncUndefined| triggered there. + Variable declarations with :var, :final and :const ~ *vim9-declaration* *:var* *************** *** 334,339 **** --- 340,379 ---- number of arguments and any return type. The function can be defined later. + Lamba using => instead of -> ~ + + In legacy script there can be confusion between using "->" for a method call + and for a lambda. Also, when a "{" is found the parser needs to figure out if + it is the start of a lambda or a dictionary, which is now more complicated + because of the use of argument types. + + To avoid these problems Vim9 script uses a different syntax for a lambda, + which is similar to Javascript: > + var Lambda = (arg) => expression + + No line break is allowed in the arguments of a lambda up to and includeing the + "=>". This is OK: > + filter(list, (k, v) => + v > 0) + This does not work: > + filter(list, (k, v) + => v > 0) + This also does not work: + filter(list, (k, + v) => v > 0) + + Additionally, a lambda can contain statements in {}: > + var Lambda = (arg) => { + g:was_called = 'yes' + return expression + } + NOT IMPLEMENTED YET + + Note that the "{" must be followed by white space, otherwise it is assumed to + be the start of a dictionary: > + var Lambda = (arg) => {key: 42} + + Automatic line continuation ~ In many cases it is obvious that an expression continues on the next line. In *************** *** 388,393 **** --- 428,436 ---- var result = start :+ print + Note that the colon is not required for the |+cmd| argument: > + edit +6 fname + It is also possible to split a function header over multiple lines, in between arguments: > def MyFunc( *************** *** 395,400 **** --- 438,453 ---- separator = '-' ): string + Since a continuation line cannot be easily recognized the parsing of commands + has been made stricter. E.g., because of the error in the first line, the + second line is seen as a separate command: > + popup_create(some invalid expression, { + exit_cb: Func}) + Now "exit_cb: Func})" is actually a valid command: save any changes to the + file "_cb: Func})" and exit. To avoid this kind of mistake in Vim9 script + there must be white space between most command names and the argument. + + Notes: - "enddef" cannot be used at the start of a continuation line, it ends the current function. *************** *** 414,427 **** < This does not work: > echo [1, 2] [3, 4] - - No line break is allowed in the arguments of a lambda, between the "{" and - "->". This is OK: > - filter(list, {k, v -> - v > 0}) - < This does not work: > - filter(list, {k, - v -> v > 0}) - No curly braces expansion ~ --- 467,472 ---- *************** *** 622,627 **** --- 667,679 ---- if has('feature') | use-feature | endif enddef + Other differences ~ + + Patterns are used like 'magic' is set, unless explicitly overruled. + The 'edcompatible' option value is not used. + The 'gdefault' option value is not used. + + ============================================================================== 3. New style functions *fast-functions* *************** *** 791,802 **** This can be a problem when the "any" type is undesired and the actual type is expected to always be the same. For example, when declaring a list: > var l: list = [1, g:two] ! This will give an error, because "g:two" has type "any". To avoid this, use a ! type cast: > var l: list = [1, g:two] ! < *type-casting* ! The compiled code will then check that "g:two" is a number at runtime and give ! an error if it isn't. This is called type casting. The syntax of a type cast is: "<" {type} ">". There cannot be white space after the "<" or before the ">" (to avoid them being confused with --- 843,856 ---- This can be a problem when the "any" type is undesired and the actual type is expected to always be the same. For example, when declaring a list: > var l: list = [1, g:two] ! At compile time Vim doesn't know the type of "g:two" and the expression type ! becomes list. An instruction is generated to check the list type before ! doing the assignment, which is a bit inefficient. ! *type-casting* ! To avoid this, use a type cast: > var l: list = [1, g:two] ! The compiled code will then only check that "g:two" is a number and give an ! error if it isn't. This is called type casting. The syntax of a type cast is: "<" {type} ">". There cannot be white space after the "<" or before the ">" (to avoid them being confused with *************** *** 926,932 **** location of the script file itself. This is useful to split up a large plugin into several files. - An absolute path, starting with "/" on Unix or "D:/" on MS-Windows. This ! will be rarely used. - A path not being relative or absolute. This will be found in the "import" subdirectories of 'runtimepath' entries. The name will usually be longer and unique, to avoid loading the wrong file. --- 980,986 ---- location of the script file itself. This is useful to split up a large plugin into several files. - An absolute path, starting with "/" on Unix or "D:/" on MS-Windows. This ! will rarely be used. - A path not being relative or absolute. This will be found in the "import" subdirectories of 'runtimepath' entries. The name will usually be longer and unique, to avoid loading the wrong file. *************** *** 1128,1134 **** `:var`. This is used in many languages. The semantics might be slightly different, but it's easily recognized as a declaration. ! Using `:const` for constants is common, but the semantics vary. Some languages only make the variable immutable, others also make the value immutable. Since "final" is well known from Java for only making the variable immutable we decided to use that. And then `:const` can be used for making --- 1182,1188 ---- `:var`. This is used in many languages. The semantics might be slightly different, but it's easily recognized as a declaration. ! Using `:const` for constants is common, but the semantics varies. Some languages only make the variable immutable, others also make the value immutable. Since "final" is well known from Java for only making the variable immutable we decided to use that. And then `:const` can be used for making *************** *** 1165,1171 **** def Func(arg1 number, arg2 string) bool The first is more familiar for anyone used to C or Java. The second one ! doesn't really has an advantage over the first, so let's discard the second. Since we use type inference the type can be left out when it can be inferred from the value. This means that after `var` we don't know if a type or a name --- 1219,1225 ---- def Func(arg1 number, arg2 string) bool The first is more familiar for anyone used to C or Java. The second one ! doesn't really have an advantage over the first, so let's discard the second. Since we use type inference the type can be left out when it can be inferred from the value. This means that after `var` we don't know if a type or a name *************** *** 1180,1198 **** Expressions ~ ! Expression evaluation was already close to what JavaScript and other languages ! are doing. Some details are unexpected and can be fixed. For example how the ! || and && operators work. Legacy Vim script: > ! var value = 44 ! ... ! var result = value || 0 # result == 1 ! ! Vim9 script works like JavaScript/TypeScript, keep the value: > ! var value = 44 ! ... ! var result = value || 0 # result == 44 ! ! TODO: the semantics of || and && need to be reconsidered. Import and Export ~ --- 1234,1268 ---- Expressions ~ ! Expression evaluation was already close to what other languages are doing. ! Some details are unexpected and can be improved. For example a boolean ! condition would accept a string, convert it to a number and check if the ! number is non-zero. This is unexpected and often leads to mistakes, since ! text not starting with a number would be converted to zero, which is ! considered false. Thus using a string for a condition would often not give an ! error and be considered false. That is confusing. ! ! In Vim9 type checking is stricter to avoid mistakes. Where a condition is ! used, e.g. with the `:if` command and the `||` operator, only boolean-like ! values are accepted: ! true: `true`, `v:true`, `1`, `0 < 9` ! false: `false`, `v:false`, `0`, `0 > 9` ! Note that the number zero is false and the number one is true. This is more ! permissive than most other languages. It was done because many builtin ! functions return these values. ! ! If you have any type of value and want to use it as a boolean, use the `!!` ! operator: ! true: !`!'text'`, `!![99]`, `!!{'x': 1}`, `!!99` ! false: `!!''`, `!![]`, `!!{}` ! ! From a language like JavaScript we have this handy construct: > ! GetName() || 'unknown' ! However, this conflicts with only allowing a boolean for a condition. ! Therefore the "??" operator was added: > ! GetName() ?? 'unknown' ! Here you can explicitly express your intention to use the value as-is and not ! result in a boolean. This is called the |falsy-operator|. Import and Export ~ *** ../vim-8.2.2203/src/vim9compile.c 2020-12-23 20:27:26.737538542 +0100 --- src/vim9compile.c 2020-12-24 15:05:23.608615687 +0100 *************** *** 2967,2978 **** return FAIL; } ufunc = rettv.vval.v_partial->pt_func; ++ufunc->uf_refcount; clear_tv(&rettv); ! // The function will have one line: "return {expr}". ! // Compile it into instructions. compile_def_function(ufunc, TRUE, cctx); clear_evalarg(&evalarg, NULL); --- 2967,2978 ---- return FAIL; } + // "rettv" will now be a partial referencing the function. ufunc = rettv.vval.v_partial->pt_func; ++ufunc->uf_refcount; clear_tv(&rettv); ! // Compile the function into instructions. compile_def_function(ufunc, TRUE, cctx); clear_evalarg(&evalarg, NULL); *************** *** 3565,3570 **** --- 3565,3579 ---- if (**arg == '{') { // lambda call: list->{lambda} + // TODO: remove this + if (compile_lambda_call(arg, cctx) == FAIL) + return FAIL; + } + else if (**arg == '(') + { + // Funcref call: list->(Refs[2])() + // or lambda: list->((arg) => expr)() + // TODO: make this work if (compile_lambda_call(arg, cctx) == FAIL) return FAIL; } *************** *** 3928,3933 **** --- 3937,3944 ---- && VIM_ISWHITE(after[-2])) || after == start + 1) && IS_WHITE_OR_NUL(after[1])) + // TODO: if we go with the "(arg) => expr" syntax + // remove this ret = compile_lambda(arg, cctx); else ret = compile_dict(arg, cctx, ppconst); *************** *** 3959,3986 **** break; /* * nested expression: (expression). */ ! case '(': *arg = skipwhite(*arg + 1); ! // recursive! ! if (ppconst->pp_used <= PPSIZE - 10) ! { ! ret = compile_expr1(arg, cctx, ppconst); ! } ! else ! { ! // Not enough space in ppconst, flush constants. ! if (generate_ppconst(cctx, ppconst) == FAIL) ! return FAIL; ! ret = compile_expr0(arg, cctx); ! } ! *arg = skipwhite(*arg); ! if (**arg == ')') ! ++*arg; ! else if (ret == OK) ! { ! emsg(_(e_missing_close)); ! ret = FAIL; } break; --- 3970,4024 ---- break; /* * nested expression: (expression). + * lambda: (arg, arg) => expr + * funcref: (arg, arg) => { statement } */ ! case '(': { ! char_u *start = skipwhite(*arg + 1); ! char_u *after = start; ! garray_T ga_arg; ! // Find out if "=>" comes after the (). ! ret = get_function_args(&after, ')', NULL, ! &ga_arg, TRUE, NULL, NULL, ! TRUE, NULL, NULL); ! if (ret == OK && VIM_ISWHITE( ! *after == ':' ? after[1] : *after)) ! { ! if (*after == ':') ! // Skip over type in "(arg): type". ! after = skip_type(skipwhite(after + 1), TRUE); ! ! after = skipwhite(after); ! if (after[0] == '=' && after[1] == '>' ! && IS_WHITE_OR_NUL(after[2])) ! { ! ret = compile_lambda(arg, cctx); ! break; ! } ! } ! ! // (expression): recursive! ! *arg = skipwhite(*arg + 1); ! if (ppconst->pp_used <= PPSIZE - 10) ! { ! ret = compile_expr1(arg, cctx, ppconst); ! } ! else ! { ! // Not enough space in ppconst, flush constants. ! if (generate_ppconst(cctx, ppconst) == FAIL) ! return FAIL; ! ret = compile_expr0(arg, cctx); ! } ! *arg = skipwhite(*arg); ! if (**arg == ')') ! ++*arg; ! else if (ret == OK) ! { ! emsg(_(e_missing_close)); ! ret = FAIL; ! } } break; *** ../vim-8.2.2203/src/userfunc.c 2020-12-22 17:35:50.043978116 +0100 --- src/userfunc.c 2020-12-24 14:43:47.061251954 +0100 *************** *** 154,159 **** --- 154,160 ---- /* * Get function arguments. + * "argp" is advanced just after "endchar". */ int get_function_args( *************** *** 458,464 **** --- 459,489 ---- #endif /* + * Skip over "->" or "=>" after the arguments of a lambda. + * Return NULL if no valid arrow found. + */ + static char_u * + skip_arrow(char_u *start, int equal_arrow) + { + char_u *s = start; + + if (equal_arrow) + { + if (*s == ':') + s = skip_type(skipwhite(s + 1), TRUE); + s = skipwhite(s); + if (*s != '=') + return NULL; + ++s; + } + if (*s != '>') + return NULL; + return skipwhite(s + 1); + } + + /* * Parse a lambda expression and get a Funcref from "*arg". + * "arg" points to the { in "{arg -> expr}" or the ( in "(arg) => expr" * When "types_optional" is TRUE optionally take argument types. * Return OK or FAIL. Returns NOTDONE for dict or {expr}. */ *************** *** 484,499 **** int *old_eval_lavars = eval_lavars_used; int eval_lavars = FALSE; char_u *tofree = NULL; ga_init(&newargs); ga_init(&newlines); ! // First, check if this is a lambda expression. "->" must exist. s = skipwhite(*arg + 1); ! ret = get_function_args(&s, '-', NULL, types_optional ? &argtypes : NULL, types_optional, NULL, NULL, TRUE, NULL, NULL); ! if (ret == FAIL || *s != '>') return NOTDONE; // Parse the arguments again. --- 509,528 ---- int *old_eval_lavars = eval_lavars_used; int eval_lavars = FALSE; char_u *tofree = NULL; + int equal_arrow = **arg == '('; + + if (equal_arrow && !in_vim9script()) + return NOTDONE; ga_init(&newargs); ga_init(&newlines); ! // First, check if this is a lambda expression. "->" or "=>" must exist. s = skipwhite(*arg + 1); ! ret = get_function_args(&s, equal_arrow ? ')' : '-', NULL, types_optional ? &argtypes : NULL, types_optional, NULL, NULL, TRUE, NULL, NULL); ! if (ret == FAIL || skip_arrow(s, equal_arrow) == NULL) return NOTDONE; // Parse the arguments again. *************** *** 502,519 **** else pnewargs = NULL; *arg = skipwhite(*arg + 1); ! ret = get_function_args(arg, '-', pnewargs, types_optional ? &argtypes : NULL, types_optional, &varargs, NULL, FALSE, NULL, NULL); ! if (ret == FAIL || **arg != '>') ! goto errret; // Set up a flag for checking local variables and arguments. if (evaluate) eval_lavars_used = &eval_lavars; // Get the start and the end of the expression. - *arg = skipwhite_and_linebreak(*arg + 1, evalarg); start = *arg; ret = skip_expr_concatenate(arg, &start, &end, evalarg); if (ret == FAIL) --- 531,558 ---- else pnewargs = NULL; *arg = skipwhite(*arg + 1); ! ret = get_function_args(arg, equal_arrow ? ')' : '-', pnewargs, types_optional ? &argtypes : NULL, types_optional, &varargs, NULL, FALSE, NULL, NULL); ! if (ret == FAIL || (*arg = skip_arrow(*arg, equal_arrow)) == NULL) ! return NOTDONE; // Set up a flag for checking local variables and arguments. if (evaluate) eval_lavars_used = &eval_lavars; + *arg = skipwhite_and_linebreak(*arg, evalarg); + + // Only recognize "{" as the start of a function body when followed by + // white space, "{key: val}" is a dict. + if (equal_arrow && **arg == '{' && IS_WHITE_OR_NUL((*arg)[1])) + { + // TODO: process the function body upto the "}". + emsg("Lambda function body not supported yet"); + goto errret; + } + // Get the start and the end of the expression. start = *arg; ret = skip_expr_concatenate(arg, &start, &end, evalarg); if (ret == FAIL) *************** *** 525,537 **** evalarg->eval_tofree = NULL; } ! *arg = skipwhite_and_linebreak(*arg, evalarg); ! if (**arg != '}') { ! semsg(_("E451: Expected }: %s"), *arg); ! goto errret; } - ++*arg; if (evaluate) { --- 564,579 ---- evalarg->eval_tofree = NULL; } ! if (!equal_arrow) { ! *arg = skipwhite_and_linebreak(*arg, evalarg); ! if (**arg != '}') ! { ! semsg(_("E451: Expected }: %s"), *arg); ! goto errret; ! } ! ++*arg; } if (evaluate) { *** ../vim-8.2.2203/src/testdir/test_vim9_expr.vim 2020-12-23 20:27:26.737538542 +0100 --- src/testdir/test_vim9_expr.vim 2020-12-24 15:10:58.575419218 +0100 *************** *** 1887,1892 **** --- 1887,1986 ---- CheckDefFailure(['var Fx = {a -> [0', ' 1]}'], 'E696:', 2) enddef + def NewLambdaWithComments(): func + return (x) => + # some comment + x == 1 + # some comment + || + x == 2 + enddef + + def NewLambdaUsingArg(x: number): func + return () => + # some comment + x == 1 + # some comment + || + x == 2 + enddef + + def Test_expr7_new_lambda() + var lines =<< trim END + var La = () => 'result' + assert_equal('result', La()) + assert_equal([1, 3, 5], [1, 2, 3]->map((key, val) => key + val)) + + # line continuation inside lambda with "cond ? expr : expr" works + var ll = range(3) + map(ll, (k, v) => v % 2 ? { + ['111']: 111 } : {} + ) + assert_equal([{}, {111: 111}, {}], ll) + + ll = range(3) + map(ll, (k, v) => v == 8 || v + == 9 + || v % 2 ? 111 : 222 + ) + assert_equal([222, 111, 222], ll) + + ll = range(3) + map(ll, (k, v) => v != 8 && v + != 9 + && v % 2 == 0 ? 111 : 222 + ) + assert_equal([111, 222, 111], ll) + + var dl = [{key: 0}, {key: 22}]->filter(( _, v) => v['key'] ) + assert_equal([{key: 22}], dl) + + dl = [{key: 12}, {['foo']: 34}] + assert_equal([{key: 12}], filter(dl, + (_, v) => has_key(v, 'key') ? v['key'] == 12 : 0)) + + assert_equal(false, NewLambdaWithComments()(0)) + assert_equal(true, NewLambdaWithComments()(1)) + assert_equal(true, NewLambdaWithComments()(2)) + assert_equal(false, NewLambdaWithComments()(3)) + + assert_equal(false, NewLambdaUsingArg(0)()) + assert_equal(true, NewLambdaUsingArg(1)()) + + var res = map([1, 2, 3], (i: number, v: number) => i + v) + assert_equal([1, 3, 5], res) + + # Lambda returning a dict + var Lmb = () => {key: 42} + assert_equal({key: 42}, Lmb()) + END + CheckDefSuccess(lines) + + CheckDefFailure(["var Ref = (a)=>a + 1"], 'E1001:') + CheckDefFailure(["var Ref = (a)=> a + 1"], 'E1001:') + CheckDefFailure(["var Ref = (a) =>a + 1"], 'E1001:') + + CheckDefFailure(["filter([1, 2], (k,v) => 1)"], 'E1069:', 1) + # error is in first line of the lambda + CheckDefFailure(["var L = (a) -> a + b"], 'E1001:', 1) + + # TODO: lambda after -> doesn't work yet + # assert_equal('xxxyyy', 'xxx'->((a, b) => a .. b)('yyy')) + + # CheckDefExecFailure(["var s = 'asdf'->{a -> a}('x')"], + # 'E1106: One argument too many') + # CheckDefExecFailure(["var s = 'asdf'->{a -> a}('x', 'y')"], + # 'E1106: 2 arguments too many') + # CheckDefFailure(["echo 'asdf'->{a -> a}(x)"], 'E1001:', 1) + + CheckDefSuccess(['var Fx = (a) => {k1: 0,', ' k2: 1}']) + CheckDefFailure(['var Fx = (a) => {k1: 0', ' k2: 1}'], 'E722:', 2) + CheckDefFailure(['var Fx = (a) => {k1: 0,', ' k2 1}'], 'E720:', 2) + + CheckDefSuccess(['var Fx = (a) => [0,', ' 1]']) + CheckDefFailure(['var Fx = (a) => [0', ' 1]'], 'E696:', 2) + enddef + def Test_expr7_lambda_vim9script() var lines =<< trim END vim9script *** ../vim-8.2.2203/src/version.c 2020-12-24 13:33:42.904570857 +0100 --- src/version.c 2020-12-24 13:53:35.324126895 +0100 *************** *** 752,753 **** --- 752,755 ---- { /* Add new patch number below this line */ + /**/ + 2204, /**/ -- hundred-and-one symptoms of being an internet addict: 26. You check your mail. It says "no new messages." So you check it again. /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///