bug: macros produce incorrect parse trees

[nickav]

Did you check existing issues?

• I have read all the tree-sitter docs if it relates to using the parser
• I have searched the existing issues of tree-sitter-c

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

No response

Describe the bug

The following code snippet produces a parse tree that seems to suggest the if is inside the preproc_if:

#if 0
if (false) {
#endif
}

Steps To Reproduce/Bad Parse Tree

Produces the following tree (copied from Playground):

translation_unit [0, 0] - [4, 0]
  preproc_if [0, 0] - [3, 1]
    condition: number_literal [0, 4] - [0, 5]
    if_statement [1, 0] - [3, 1]
      condition: parenthesized_expression [1, 3] - [1, 10]
        false [1, 4] - [1, 9]
      consequence: compound_statement [1, 11] - [3, 1]
        preproc_call [2, 0] - [3, 0]
          directive: preproc_directive [2, 0] - [2, 6]
    MISSING "#endif" [3, 1] - [3, 1]

Expected Behavior/Parse Tree

However, I would expect the macro to take the highest precedence. Meaning the parse tree I would expect would include everything inside the #if 0 / #endif and then have a single dangling } brace as an error at the end.

Example parse tree:

translation_unit [0, 0] - [4, 0]
  ERROR [0, 0] - [3, 0]
    number_literal [0, 4] - [0, 5]
    parenthesized_expression [1, 3] - [1, 10]
      false [1, 4] - [1, 9]
    preproc_call [2, 0] - [3, 0]
      directive: preproc_directive [2, 0] - [2, 6]
ERROR [3, 0] - [3, 1]

Repro

[carl-tud]

@nickav I'm facing the same issue, except with linkage specifications:

#ifdef __cplusplus
extern "C" {
#endif

struct A {};
void f(void);

#ifdef __cplusplus
}
#endif

From my very limited understanding of the grammar:

I believe the problem is the compound statement (the opening curly brace) in combination with the preproc_call rule.

tree-sitter-c/grammar.js

Lines 154 to 159 in 7fa1be1

	preproc_call: $ => seq(
	field('directive', $.preproc_directive),
	field('argument', optional($.preproc_arg)),
	token.immediate(/\r?\n/),
	),

tree-sitter-c/grammar.js

Line 166 in 7fa1be1

preproc_directive: _ => /#[ \t]*[a-zA-Z0-9]\w*/,

IMO, tree sitter has no chance of 'identifying an error' (extra #endif) and hence cannot apply any precedence rules, because the 'extra' #endif is recognized as a preproc_call.

If tree sitter would support lookaheads, you could exclude the #endif from preproc_directive. As that does not work, I managed to get it working (symbols not nested in linkage specification anymore) by disallowing preprocessor directives starting with 'e'. Yeah, this is hacky, I know.

preproc_directive: _ => /#[ \t]*[^e][a-zA-Z0-9]\w*/,

This is probably absolutely not the way to do it, but it works for the moment. Does anyone know how you'd fix this properly? I don't know if this really is a bug, though.