Skip to content

Conversation

fanquake
Copy link
Member

@fanquake fanquake commented Apr 12, 2022

We recently added a clang-tidy job to the CI, which generates a compilation database. We can leverage that now existing database to begin running include-what-you-use over the codebase.

This PR demonstrates using a mapping_file to indicate fixups / includes that may differ from IWYU suggestions. In this case, I've added some fixups for glibc includes that I've upstreamed changes for:

# Fixups / upstreamed changes
[
  { include: [ "<bits/termios-c_lflag.h>", private, "<termios.h>", public ] },
  { include: [ "<bits/termios-struct.h>", private, "<termios.h>", public ] },
  { include: [ "<bits/termios-tcflow.h>", private, "<termios.h>", public ] },
]

The include "fixing" commits of this PR:

  • Adds missing includes.
  • Swaps C headers for their C++ counterparts.
  • Removes the pointless / unmaintainable //for abc, xyz comments. When using IWYU, if anyone wants to see / generate those comments, to see why something is included, it is trivial to do so (IWYU outputs them by default). i.e:
// The full include-list for compat/stdin.cpp:
#include <compat/stdin.h>
#include <poll.h>                  // for poll, pollfd, POLLIN
#include <termios.h>               // for tcgetattr, tcsetattr
#include <unistd.h>                // for isatty, STDIN_FILENO

TODO:

  • Qt mapping_file. There is one in the IWYU repo, but it's for Qt 5.11. Needs testing.
  • Boost mapping_file. There is one in the IWYU repo, but it's for Boost 1.75. Needs testing.

I'm not suggesting we turn this on the for entire codebase, or immediately go-nuts refactoring all includes. However I think our dependency includes are now slim enough, and our CI infrastructure in place such that we can start doing this in some capacity, and just automate away include fixups / refactorings etc.

@jonatack
Copy link
Member

Concept ACK

@jonatack
Copy link
Member

It might be good to add some developer notes docs on this if it moves forward.

@hebasto
Copy link
Member

hebasto commented Apr 12, 2022

Concept ACK.

@ryanofsky
Copy link
Contributor

Concept ACK and thanks for working on this! I've used IWYU on other projects and it was nice once you got it set up. I think ideally it would be easy to run IWYU locally, but getting it working on CI is useful, and should make it easier to run locally as well.

@laanwj
Copy link
Member

laanwj commented Apr 13, 2022

Concept ACK

@@ -3,18 +3,18 @@
// file COPYING or http://www.opensource.org/licenses/mit-license.php.

#if defined(HAVE_CONFIG_H)
#include <config/bitcoin-config.h>
#include <config/bitcoin-config.h> // IWYU pragma: keep
Copy link
Member

@maflcko maflcko Apr 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed? (I mean the include)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed? (I mean the include)

In this case, the include isn't actually needed. Although in other cases we'll likely use the same pragma on this same include to suppress IWYU output, given we aren't building for all configurations (I think that's ok). I'm removing it now.

Add missing includes.

Swap C headers for their C++ counterparts.

Remove pointless / unmaintainable include comments. This is even more the case
when we are actually using IWYU, as if anyone wants to see the comments they can
just get IWYU to generate them.
@fanquake fanquake changed the title [POC] tidy: add include-what-you-use tidy: add include-what-you-use Apr 20, 2022
@fanquake fanquake force-pushed the include_what_you_use branch from 2f7817d to 9b0a13a Compare April 20, 2022 13:15
@jamesob
Copy link
Contributor

jamesob commented Apr 20, 2022

Concept ACK. Awesome!

@maflcko
Copy link
Member

maflcko commented Apr 20, 2022

review ACK 9b0a13a

@DrahtBot
Copy link
Contributor

DrahtBot commented Apr 20, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

Copy link
Member

@jonatack jonatack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 9b0a13a reviewed changes and run CI output in https://cirrus-ci.com/task/4750910332076032

@maflcko
Copy link
Member

maflcko commented Apr 28, 2022

Going to merge this so that ppl can start experimenting with this and also use it in more places. Hopefully this will help us reduce compile time by nuking useless includes and reduce review burden by automating the review discussions on which headers to include/remove.

If there are any issues, it should be trivial to disable in CI by editing ci/test/06_script_b.sh.

@maflcko maflcko merged commit 9446de1 into bitcoin:master Apr 28, 2022
@fanquake fanquake deleted the include_what_you_use branch April 28, 2022 08:14
@@ -37,6 +37,8 @@ fi
if [ "${RUN_TIDY}" = "true" ]; then
export P_CI_DIR="${BASE_BUILD_DIR}/bitcoin-$HOST/src/"
CI_EXEC run-clang-tidy "${MAKEJOBS}"
export P_CI_DIR="${BASE_BUILD_DIR}/bitcoin-$HOST/"
CI_EXEC "python3 ${BASE_SCRATCH_DIR}/iwyu/include-what-you-use/iwyu_tool.py src/compat src/init -p . ${MAKEJOBS} -- -Xiwyu --cxx17ns -Xiwyu --mapping_file=${BASE_BUILD_DIR}/bitcoin-$HOST/contrib/devtools/iwyu/bitcoin.core.imp"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is setting the return code to non-zero if it fails?

Based on the passing test in https://cirrus-ci.com/task/4869683827441664?logs=ci#L6712 it does not?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #26763.

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Apr 29, 2022
@bitcoin bitcoin locked and limited conversation to collaborators Dec 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants