Skip to content

Conversation

leandrolanzieri
Copy link
Contributor

@leandrolanzieri leandrolanzieri commented Nov 1, 2019

Contribution description

This introduces the IS_ACTIVE macro to kernel_defines.h. It is based on IS_BUILTIN defined in Linux. The main idea behind it is to allow to check macro definitions directly in C expressions (i.e. not with preprocessor directives), even when the macro is not defined (e.g. MODULE_<module> macros, which will be defined to 1 when the module is used, but won't be defined when it is not).

The main advantage is that we can get rid of lots of #ifdef s in the code and use if statements instead, so the compiler can see this code and check for errors. If not used, the compiler will remove those blocks.

EDIT

Problem statement

  • #ifdef logic is really hard to follow most of times and hides the code from the tooling (e.g. cppcheck, coccinelle, IDEs) which prevents from checking syntax, types, symbol references, etc.

  • This is adopted by different communities (GNU, Linux).

  • There is a tendency towards using C conditionals where possible.


Example

With the current approach:

/* this will be compiled with no errors */
#ifdef MODULE_NOT_USED
a+++;
do_something();
#endif

Using this macro:

/* this fails to compile */
if (IS_ACTIVE(MODULE_NOT_USED)) {
   a+++;
   do_something();
}

Testing procedure

  1. Try something like this patch to see the behaviour of the macro:
diff --git a/examples/default/main.c b/examples/default/main.c
index 9d6542d0f..ecdcbd1c0 100644
--- a/examples/default/main.c
+++ b/examples/default/main.c
@@ -25,6 +25,7 @@
 #include <stdio.h>
 #include <string.h>
 
+#include "kernel_defines.h"
 #include "thread.h"
 #include "shell.h"
 #include "shell_commands.h"
@@ -36,12 +37,27 @@
 
 int main(void)
 {
+    int a = 0;
 #ifdef MODULE_NETIF
     gnrc_netreg_entry_t dump = GNRC_NETREG_ENTRY_INIT_PID(GNRC_NETREG_DEMUX_CTX_ALL,
                                                           gnrc_pktdump_pid);
     gnrc_netreg_register(GNRC_NETTYPE_UNDEF, &dump);
 #endif
 
+/* this error is not caught */
+#ifdef MODULE_THAT_DOES_NOT_EXIST
+    a+++;
+#endif
+
+    if (IS_ACTIVE(MODULE_NETIF)) {
+        puts("Netif is active");
+    }
+
+    if (IS_ACTIVE(MODULE_THAT_DOES_NOT_EXIST)) {
+        puts("Should not get in here");
+        a+++;
+    }
+
     (void) puts("Welcome to RIOT!");
 
     char line_buf[SHELL_DEFAULT_BUFSIZE];
  1. It's kind of a tricky macro, so please check that documentation is clear enough.

Issues/PRs references

None.

@leandrolanzieri leandrolanzieri added Area: core Area: RIOT kernel. Handle PRs marked with this with care! Type: new feature The issue requests / The PR implemements a new feature for RIOT labels Nov 1, 2019
@kaspar030
Copy link
Contributor

Well,

[kaspar@ng riot.tmp/examples/hello-world (pr12626)]$ cat main.c 
#include "kernel_defines.h"

int main(void)
{
    if (IS_ACTIVE(MODULE_foobar)) {
        do_something();
    }

    return 0;
}
[kaspar@ng riot.tmp/examples/hello-world (pr12626)]$

still fails to compile:

[kaspar@ng riot.tmp/examples/hello-world (pr12626)]$ make
Building application "hello-world" for "native" with MCU "native".

main.c: In function ‘main’:
main.c:6:9: error: implicit declaration of function ‘do_something’ [-Werror=implicit-function-declaration]
    6 |         do_something();
      |         ^~~~~~~~~~~~
cc1: all warnings being treated as errors
make[1]: *** [/home/kaspar/src/riot.tmp/Makefile.base:89: /home/kaspar/src/riot.tmp/examples/hello-world/bin/native/application_hello-world/main.o] Error 1
make: *** [/home/kaspar/src/riot.tmp/examples/hello-world/../../Makefile.include:522: /home/kaspar/src/riot.tmp/examples/hello-world/bin/native/application_hello-world.a] Error 2
[kaspar@ng riot.tmp/examples/hello-world (pr12626)]$

@leandrolanzieri
Copy link
Contributor Author

leandrolanzieri commented Nov 1, 2019

still fails to compile:

And it's correct, the function does not exist. If that was an #ifdef block then you would not get that check.

@leandrolanzieri
Copy link
Contributor Author

leandrolanzieri commented Nov 1, 2019

Just to clarify, I'm not proposing to remove all our #ifdefs, just saying that we can reduce them using this (which is actually suggested in Linux's coding convention to which we adhere).

@maribu
Copy link
Member

maribu commented Nov 1, 2019

Just to clarify, I'm not proposing to remove all our #ifdefs, just saying that we can reduce them using this (which is actually suggested in Linux's coding convention to which we adhere).

This makes sense to me. The approach will simply not work in any case (e.g. additional cases in a switch statement.). But where applicable, the resulting code looks more readable to me.

Also: If the generated boolean is a compile time constant, any compiler called with -O<foo> where <foo> != 0 should yield no increase in ROM size. So there should be no obvious disadvantage of adding this, with the exception that coders now have to be aware of two tools for basically the same use case.

To me, the improvement in readability is worth the effort of getting to know a second tool; especially as its use is rather trivial.

@kaspar030
Copy link
Contributor

still fails to compile:

And it's correct, the function does not exist. If that was an #ifdef block then you would not get that check.

I know. Just wanted to point out that it can only applied in some cases.

Also: If the generated boolean is a compile time constant, any compiler called with -O<foo> where <foo> != 0 should yield no increase in ROM size. So there should be no obvious disadvantage of adding this, with the exception that coders now have to be aware of two tools for basically the same use case.

Yes, we're relying on that already in the DEBUG macro.

To me, the improvement in readability is worth the effort of getting to know a second tool; especially as its use is rather trivial.

I'd like to add a convention to only use this if it can actually be used in all cases throughout a file. Otherwise we'll have some occurrences using IF_ACTIVE() (if it works and a reviewer realized), others using plain ifdef, in one file. I'd rather give up some checks for consistency in those cases.

What's wrong with IF_BUILTIN as name?

Copy link
Member

@maribu maribu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should work, but only when the macros are defined to 1 and not to e.g. (1) or 1U. It would not hurt to document this so that noone changes the way the build system defines those modules without adapting this header.

Update: There was a not missing in a crucial place of the sentence before.

@leandrolanzieri
Copy link
Contributor Author

What's wrong with IF_BUILTIN as name?

Personally I would like to keep IS as the first part, otherwise I just proposed IS_ACTIVE as a broader meaning (not just modules builtin or not), but I'm also not 100% convinced by it.

@leandrolanzieri
Copy link
Contributor Author

@maribu

It would not hurt to document this so that noone changes the way the build system defines those modules without adapting this header.

Here or on modules.inc.mk? both?

@maribu
Copy link
Member

maribu commented Nov 1, 2019

Here or on modules.inc.mk? both?

I personally have no strong feelings about this, but I would do it like this: Document it here and add a short reference to that doc in modules.inc.mk, so that people modifying something there will find that doc.

Copy link
Member

@smlng smlng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

documentation wise this looks good to me, quite clear and elaborate 😄

Copy link
Contributor

@kaspar030 kaspar030 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to get more opinions on this.

I'm usually advertising RIOT as "mostly macro-free". And it is.
ARRAY_SIZE was simple, usable everywhere and also automatically enforcable. This is neither.
My gut feeling is that the slight benefits in syntax-checking code (and subjective readability) for some cases is not worth the downsides of using the macro.

(Blocking to get more time for discussion.)

@kaspar030
Copy link
Contributor

the downsides of using the macro

Just to name some:

  1. as discussed here, this can only be used if the enclosed code compiles without the module being active. So this would lead to multiple (inconsistent) ways of excluding code based on module selection.

  2. Currently, to find optional dependees of a module, ````grep "ifdef MODULE_FOO"``` is sufficient. I do have scripts that depend on that. Sure they can be updated...

  3. This comes with all the downsides of a macro. Like, in my case, ctags doesn't recognize it, so my nice "take me to definition" button tells me "IS_ACTIVE(): not found".

I'd like to suggest to use, where possible, something explicit like:

#ifndef MODULE_FOO
#define MODULE_FOO 0
#endif

/* ... */

void foo() {
  if (MODULE_FOO) { ... }

@jia200x
Copy link
Member

jia200x commented Nov 4, 2019

I personally like the idea of get rid of #ifdef wherever it's possible. I really think it's important to have syntax checking.

I don't remember where, but I remember seeing some cases where the static checks didn't detect a dereferenced NULL pointer because they were surrounded by #ifdef. So I really think in this case the downsides of a macro are not too bad.

I think it would be a huge overhead to define MODULE_FOO=0 every time that check is needed and I don't think we would benefit that much, considering we could just use the proposed macro.

@leandrolanzieri
Copy link
Contributor Author

I'd like to get more opinions on this.

Of course!

I'd like to add a convention to only use this if it can actually be used in all cases throughout a file. Otherwise we'll have some occurrences using IF_ACTIVE() (if it works and a reviewer realized), others using plain ifdef, in one file. I'd rather give up some checks for consistency in those cases.

I don't see this as inconsistency, they are meant for different things. We should encourage the usage of C conditionals where possible, so we can get the benefits of checking and readability, and allow the author to use ifdef only in cases where symbols won't be defined. If we describe and document this properly there should be no confusions.

My gut feeling is that the slight benefits in syntax-checking code (and subjective readability) for some cases is not worth the downsides of using the macro.

If used as intended (not just limited to "only use if can actually be used in all cases throughout a file") I think the benefits are not slight.

  1. as discussed here, this can only be used if the enclosed code compiles without the module being active. So this would lead to multiple (inconsistent) ways of excluding code based on module selection.

Again, I don't think having multiple ways is inconsistent, as we can clearly define in which cases to use each of the alternatives.

I'd like to suggest to use, where possible, something explicit like:

I can understand this seems more explicit than a macro, but I think it will create a lot of overhead to define this every time we want to check for a module or submodule, compared to having a single macro to do it.

@kaspar030
Copy link
Contributor

I can understand this seems more explicit than a macro, but I think it will create a lot of overhead to define this every time we want to check for a module or submodule, compared to having a single macro to do it.

How many submodule dependencies do we expect per file / subsystem header (that would get checked either by IS_ACTIVE() or using ifdefs?
If there are one or two, do you think that would be too much overhead still?
Where did the idea initially pop up? Certainly not while doing gnrc code, as there usually the necessary headers are also ifdef'ed.

I don't see this as inconsistency, they are meant for different things. We should encourage the usage of C conditionals where possible, so we can get the benefits of checking and readability, and allow the author to use ifdef only in cases where symbols won't be defined.

I agree with that. The syntax checks helped a lot for cleaning up broken DEBUG() statements.

Still, macros are bad. They should be avoided, if possible. In this case, it is possible. The overhead of a couple of clear #ifndef/#define/#endif lines is traded with the cognitive overhead of a non-standard macro.

@kaspar030
Copy link
Contributor

Again, I don't think having multiple ways is inconsistent, as we can clearly define in which cases to use each of the alternatives.

Yes, but the code base will not reflect what we define.

@leandrolanzieri
Copy link
Contributor Author

Still, macros are bad. They should be avoided, if possible. In this case, it is possible.

I understand many of the concerns with using macros in general, but I think that they are also an useful tool, and when used with caution they can simplify the code a lot. Which are your concerns regarding introducing this particular macro?

@kaspar030
Copy link
Contributor

Which are your concerns regarding introducing this particular macro?

What it does is simple: Turn IS_ACTIVE(foo) into 0 or 1 depending on whether foo is defined to 1 or not or undefined.

How it works, I don't understand (yet). Its definition is spread over 5 lines, doing, in my eyes, preprocessor magic. If it breaks, I cannot debug or fix it. I cannot forsee if it can or will be misused (what happens on if (IS_ACTIVE(MODULE_FOO && MODULE_BAR))?) I cannot say if it requires anything special from the preprocessor or if works with ANSI C. From the name (without documentation), it is not clear what the macro does. ctags doesn't find the definition, so for me, manual grepping is needed to find the documentation. It breaks some of my scripts. Given there is a (maybe more verbose but dead simple) alternative, I'm in the `let's not be clever' camp on this one.

@kaspar030
Copy link
Contributor

I cannot forsee if it can or will be misused

Something that comes to mind:

if (IS_ACTIVE(MODULE_SIXLOWPAN_IPHC))...

... will happily compile even if the module is actually named MODULE_GNRC_SIXLOWPAN_IPHC. Because by definition, IS_ACTIVE() doesn't error out on undefined parameters.

Same with the current #ifdef solution. But going with #ifndef/define/#endif at the top of the file, all consecutive #ifdef can be replaced with #if (or C if when possible), erroring out on misspelled or not specified module defines.

@maribu
Copy link
Member

maribu commented Nov 4, 2019

I cannot say if it requires anything special from the preprocessor or if works with ANSI C.

It will work with any preprocessor that is fully C99 compliant.

I cannot forsee if it can or will be misused (what happens on if (IS_ACTIVE(MODULE_FOO && MODULE_BAR))?)

For anything that the preprocessor does not replace by the literal 1, it will return 0.

Given there is a (maybe more verbose but dead simple) alternative, I'm in the `let's not be clever' camp on this one.

If the build system would be aware of all possible module (and pseudo-module) names, it could simple define MODULE_<foo> for every possible value and define it to 1 if used, and 0 otherwise. That would certainly be less magical and the use of both the C-Style if (MODULE_<foo>) {...} and the preprocessor style #if MODULE_<foo> ... #endif would be consistent. (Also: The build system could point out typos in module names rather than a linker error.)

For "real" modules traversing the filesystem could yield the list (but would come with a performance penalty?). For pseudomodules we could parse the PSEUDOMODULES variable, if we stop using wildcards like PSEUDOMODULES += fmt_%. So having a list of all possible modules (including those provided by the application developer out of tree) will certainly not come for free.

@kaspar030
Copy link
Contributor

If the build system would be aware of all possible module (and pseudo-module) names, it could simple define MODULE_<foo> for every possible value and define it to 1 if used, and 0 otherwise. That would certainly be less magical and the use of both the C-Style if (MODULE_<foo>) {...} and the preprocessor style #if MODULE_<foo> ... #endif would be consistent. (Also: The build system could point out typos in module names rather than a linker error.)

Yup. I'm experimenting with that. One big thing is getting out the information which module uses which other modules "optionally", as in, it does not depend on it but is changed by it. I did that with scripts grepping for #ifdef MODULE_foo. Those are the ones that break by using this PR. That's just my personal out-of-tree use of RIOT, but there may be others.

@maribu
Copy link
Member

maribu commented Nov 5, 2019

That's just my personal out-of-tree use of RIOT, but there may be others.

Maybe we could bring a solution to that upstream. In that case we can actually use if (MODULE_<foo>) as proper C expression in a consistent way with the preprocessor versions. And I think no one will complain that we broke their scripts if we at the same time provide an upstream tool doing the same.

I think @leandrolanzieri, @smlng, and @jia200x would also be fine with this, right? This would be basically the same idea as presented here, only implemented a bit different.

@cgundogan
Copy link
Member

I'd like to get more opinions on this.

I'm usually advertising RIOT as "mostly macro-free". And it is.
ARRAY_SIZE was simple, usable everywhere and also automatically enforcable. This is neither.
My gut feeling is that the slight benefits in syntax-checking code (and subjective readability) for >some cases is not worth the downsides of using the macro.

(Blocking to get more time for discussion.)

My two cents: extensive usage of conditional compilation using preprocessor statements is bad for several reasons:

  • code becomes unreadable and unmaintainable with higher code complexity
  • preprocessor conditionals lead to poor tool support: code analyzers have to be configured and run with different compile options. Hello state explosion for each added macro configuration.
  • I don't know much about ctags issues. I moved a long time ago to using proper language servers that provide me contextual code completion and syntax highlighting. For this case, it actually hurts to have a lot of preprocessor conditionals, because of the previous bullet point: code analyzers do not see ifdefed code ..

IMO, the IS_ACTIVE macro is an elegant way to reduce a lot of preprocessor usage in our code base .. we make use of #if or #ifdef throughout the code base without any ruling (coding conventions?), which introduces many inconsistencies ..

#ifndef MODULE_FOO
#define MODULE_FOO 0
#endif

I don't see how such a long list can be maintained. Modules are added and removed and the list has to stay in sync. Without any sophisticated, scripted auto generation of such a list, I would be against hardcoding. Especially since IS_ACTIVE perfectly handles this tri-state behaviour of (defined 0, defined 1, undefined).

  1. Currently, to find optional dependees of a module, ````grep "ifdef MODULE_FOO"``` is sufficient. I do have scripts that depend on that. Sure they can be updated...

Out of curiosity, what is this script used for? Grepping for #ifdef MODULE_FOO seems to be a bit error-prone .. are you traversing all possible included .h files? In any case, grepping for IS_ACTIVE(..) is also a viable solution?

@leandrolanzieri
Copy link
Contributor Author

@maribu what do you mean by this?

  • and the syntax is more consistent

@maribu
Copy link
Member

maribu commented Nov 8, 2019

@maribu what do you mean by this?

  • and the syntax is more consistent
if (MODULE_FOO) {
    ...
}

#if MODULE_FOO

#endif

vs

if (IS_ACTIVE(MODULE_FOO)) {
    ...
}

#if MODULE_FOO
    ...
#endif

@leandrolanzieri
Copy link
Contributor Author

@maribu what do you mean by this?

  • and the syntax is more consistent
if (MODULE_FOO) {
    ...
}

#if MODULE_FOO

#endif

vs

if (IS_ACTIVE(MODULE_FOO)) {
    ...
}

#if MODULE_FOO
    ...
#endif

I see, actually if we include the list:

#ifndef MODULE_FOO
#define MODULE_FOO 0
#endif

if (MODULE_FOO) {
    ...
}

#if MODULE_FOO

#endif

vs.

if (IS_ACTIVE(MODULE_FOO)) {
    ...
}

#if MODULE_FOO
    ...
#endif

@jia200x
Copy link
Member

jia200x commented Nov 8, 2019

if the problem is about consistency, the example could also be written like:

if (IS_ACTIVE(MODULE_FOO)) {
    ...
}

#if IS_ACTIVE(MODULE_FOO)
    ...
#endif

vs

#ifndef MODULE_FOO
#define MODULE_FOO 0
#endif

if (MODULE_FOO) {
    ...
}

#if MODULE_FOO

#endif

@waehlisch
Copy link
Member

@maribu thanks for the summary.

looking from the outside, I don't see syntax inconsistency in (2). i also don't see that (1) is simpler; in fact, i would argue the contrary (see @cgundogan comments)

Therefore, 👍 for this PR (#12626) from my side.

@cgundogan
Copy link
Member

I'd like to get more opinions on this.
...
(Blocking to get more time for discussion.)

@kaspar030 how should we proceed with this PR now? The 10-day discussion seems to converge towards an ACK. The only thing missing is now to resolve the block ..

@kaspar030 kaspar030 dismissed their stale review November 11, 2019 09:40

I got my discussuon, thanks everyone!

@cgundogan
Copy link
Member

👍 @leandrolanzieri could you squash the commits?

IS_ACTIVE allows to evaluate macro definitions in non-preprocessor
expressions. It takes a macro that may be defined to 1 or not defined at
all and expands to 1 or 0 respectively.
@leandrolanzieri leandrolanzieri force-pushed the pr/core/kernel_defines_is_active branch from 9eada9f to 62b6e5e Compare November 11, 2019 11:54
@leandrolanzieri
Copy link
Contributor Author

@cgundogan squashed

@maribu maribu added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Reviewed: 1-fundamentals The fundamentals of the PR were reviewed according to the maintainer guidelines Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines Reviewed: 4-code-style The adherence to coding conventions by the PR were reviewed according to the maintainer guidelines Reviewed: 5-documentation The documentation details of the PR were reviewed according to the maintainer guidelines labels Nov 11, 2019
@maribu
Copy link
Member

maribu commented Nov 11, 2019

The approach is well tested in the Linux community. I guess this can count as tested.

@maribu maribu added the Reviewed: 3-testing The PR was tested according to the maintainer guidelines label Nov 11, 2019
Copy link
Member

@maribu maribu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK. The macro works as promised and will allow reducing the number of code chunks disabled by #ifdef and friends. The majority of maintainers favored this approach of the alternatives that were discussed.

@leandrolanzieri leandrolanzieri added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Nov 11, 2019
@jia200x
Copy link
Member

jia200x commented Nov 11, 2019

thanks all for reviewing! Let's go with this one

@jia200x jia200x merged commit a78bdaa into RIOT-OS:master Nov 11, 2019
@leandrolanzieri leandrolanzieri deleted the pr/core/kernel_defines_is_active branch November 11, 2019 15:18
@leandrolanzieri
Copy link
Contributor Author

Thank you all for the thorough reviews and discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: core Area: RIOT kernel. Handle PRs marked with this with care! CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Reviewed: 1-fundamentals The fundamentals of the PR were reviewed according to the maintainer guidelines Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines Reviewed: 3-testing The PR was tested according to the maintainer guidelines Reviewed: 4-code-style The adherence to coding conventions by the PR were reviewed according to the maintainer guidelines Reviewed: 5-documentation The documentation details of the PR were reviewed according to the maintainer guidelines Type: new feature The issue requests / The PR implemements a new feature for RIOT
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants