Skip to content

Conversation

luke-jr
Copy link
Member

@luke-jr luke-jr commented Mar 25, 2020

Possibly necessary to get threadsafe stdlib behaviours

#17740 removed this, under the (accidental) false pretence that AC_CHECK_LIB didn't actually cause linkage (it does).

It appears this hasn't been necessary since mingw 2.0 released in 2011 turned it into a dummy library, but CVE-2012-1910 occurred in 2012. Furthermore, we don't have a minimum required version of mingw, and it's possible future releases might use it again (GCC -mthreads still links to it).

Seems safest to just add it back in...

@luke-jr
Copy link
Member Author

luke-jr commented Mar 25, 2020

[03:10:58] <fanquake> I just checked that binaries produced with and without -lmingwthrd are basically identical. I see 5 bytes difference, and that's the git commit and what looks like 2 timestamps.

So at least gitian isn't affected

Copy link
Member

@fanquake fanquake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was some discussion in IRC.

In 22b6398a8acf17e6687375c414fae832888de53a (Jun 2009) the contents of mingwthrd_mt.c which becomes libmingwthrd.a lib was changed to:

/* As _CRT_MT is getting defined in libgcc when using shared version, or it is getting defined by startup code itself,
   this library is a dummy version for supporting the link library for gcc's option -mthreads.  As we support TLS-cleanup
   even without specifying this library, this library is deprecated and just kept for compatibility.  */
int _CRT_MT_OLD = 1;

I've re-tested building with and without -lmingwthrd and I'm not seeing any significance difference in the binaries produced.

configure.ac Outdated
@@ -487,6 +487,7 @@ case $host in
use_pkgconfig=no

TARGET_OS=windows
AC_CHECK_LIB([mingwthrd],[main],, AC_MSG_ERROR(libmingwthrd missing))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're going to add this back, I think there should at least be a comment stating why it's here.

@fanquake
Copy link
Member

Furthermore, we don't have a minimum required version of mingw,

From what I can gather, if you're using the oldest version of GCC we are meant to support (4.8), then the oldest version of mingw you would be able to use is v3.

@luke-jr luke-jr force-pushed the 2020mingwthrd branch 2 times, most recently from 2e1e4c9 to 7fe4967 Compare March 25, 2020 13:48
@laanwj
Copy link
Member

laanwj commented Mar 25, 2020

From what I can gather, if you're using the oldest version of GCC we are meant to support (4.8), then the oldest version of mingw you would be able to use is v3.

If this isn't a problem with any of the compilers that are realistically used to build bitcoin core, I tend towards NACK. We shouldn't keep old compatibility cruft around forever.

@luke-jr
Copy link
Member Author

luke-jr commented Mar 25, 2020

If this isn't a problem with any of the compilers that are realistically used to build bitcoin core, I tend towards NACK. We shouldn't keep old compatibility cruft around forever.

It's still linked by current MinGW. Even if it does nothing, it is incorrect to omit it.

At the very least, if removing it, we should have some kind of check for a minimum MinGW version, which is probably more complex than just doing the right thing here...

(Alternatively, if someone wants to put in the time to see if -mthreads works correctly with static binaries now, that's probably the best route to go)

@laanwj
Copy link
Member

laanwj commented Mar 25, 2020

At the very least, if removing it, we should have some kind of check for a minimum MinGW version,

I'm not opposed to a adding a minimum-gcc check to enforce the minimum that is in dependencies.md.

OTOH, are any of the compilers that support C++11 and C++11 threads (so can even compile our code at all) affected?

(Alternatively, if someone wants to put in the time to see if -mthreads works correctly with static binaries now, that's probably the best route to go)

Yes, if that's the general suggestion, it would be good to add that, preferable to manually linking against a threading library.

@luke-jr
Copy link
Member Author

luke-jr commented Mar 25, 2020

AFAIK this is related to the mingw runtime libraries, NOT the GCC version.

@laanwj
Copy link
Member

laanwj commented Mar 25, 2020

Sure, but the mingw toolchain distributions tend to be monolithic, packaging a certain gcc/c++ version with a certain version of the library.

@luke-jr
Copy link
Member Author

luke-jr commented Mar 25, 2020

Dunno about that, Gentoo has them all separate...

@DrahtBot
Copy link
Contributor

DrahtBot commented Mar 26, 2020

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #24391 (build: stop overriding user autoconf flags by fanquake)
  • #23969 (build: remove use of TARGET_OS and BUILD_OS by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@DrahtBot
Copy link
Contributor

Gitian builds

File commit 7f9dedb
(master)
commit 000a0df
(master and this pull)
bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz 8aa34845a916014d... c0a46fb27ea76b7f...
bitcoin-0.19.99-aarch64-linux-gnu.tar.gz 0fb3dd16db30ea00... 613f83c1342f0d19...
bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz 797f185a8e62f6fe... 1d3190e65ea16180...
bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz 871094b510da0ac7... ab4f1bab610ba24d...
bitcoin-0.19.99-osx-unsigned.dmg a159b1b99b607907... 9b54eff3cb6d6001...
bitcoin-0.19.99-osx64.tar.gz 1fb6370553afcb22... 88633976c6c2dfc8...
bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz 0b86916151aea5a3... 06108beddd783936...
bitcoin-0.19.99-riscv64-linux-gnu.tar.gz 4efb63771485711b... 027e22e080781d28...
bitcoin-0.19.99-win64-debug.zip dd825e6168d39c90... 7ec0e8f8fa330f00...
bitcoin-0.19.99-win64-setup-unsigned.exe d1f790ad8c257da0... fed87385acd90900...
bitcoin-0.19.99-win64.zip 49dc4119ae2a01c4... b2ce87180ac786cf...
bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz 2a144f502390b6e3... 478cfce249cfb427...
bitcoin-0.19.99-x86_64-linux-gnu.tar.gz af6a5d2d555569a4... 6b08a2fd153b3674...
bitcoin-0.19.99.tar.gz 900859565b6ee1fe... e96f6df02e6f8922...
bitcoin-core-linux-0.20-res.yml 70c720b679a19c93... de7a093350696c6d...
bitcoin-core-osx-0.20-res.yml 5766a2f7c8d8547a... 542e315422e30f96...
bitcoin-core-win-0.20-res.yml a95a890f0d0cb882... 557b9a071e4b2194...
linux-build.log 8510c43606df6b17... ee2e5ef607dba92b...
osx-build.log b2fce58471126628... 1dc8c06799733fdc...
win-build.log 630e3d23ee0657f4... 8de7a5edccc66acd...
bitcoin-core-linux-0.20-res.yml.diff 2d079917b3329b76...
bitcoin-core-osx-0.20-res.yml.diff eee1b6f28f0946e5...
bitcoin-core-win-0.20-res.yml.diff 854fd320b2f1ba63...
linux-build.log.diff 1c27c216a088102f...
osx-build.log.diff fb1270e4d4066e99...
win-build.log.diff 9fb2960f5e3712e9...

@laanwj
Copy link
Member

laanwj commented Mar 28, 2020

Dunno about that, Gentoo has them all separate...

Even with that, the range of libc versions that can be used with a certain gcc/c++ version tends to be restricted. There's also not really a reason to use old mingw libraries. E.g. for glibc we target an old version because of compatibility reasons, but newer mingw do support old windows versions. And with that, using old windows versions for Bitcoin Core is a bad idea anyway as they're all unmaintained.

@luke-jr
Copy link
Member Author

luke-jr commented Apr 22, 2020

There's also not really a reason to use old mingw libraries.

New ones do have outstanding issues with their build system FWIW.

@maflcko
Copy link
Member

maflcko commented Oct 22, 2021

Needs rebase if still relevant.

@luke-jr
Copy link
Member Author

luke-jr commented Dec 18, 2021

Rebased

@hebasto
Copy link
Member

hebasto commented Jan 3, 2022

https://gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/x86-Options.html:

-mthreads

Support thread-safe exception handling on MinGW. Programs that rely on thread-safe exception handling must compile and link all code with the -mthreads option. When compiling, -mthreads defines -D_MT; when linking, it links in a special thread helper library -lmingwthrd which cleans up per-thread exception-handling data.

All supported compiler version has the -mthreads option.

This PR aims to introduce a precaution in possible cases when the -mthreads option is broken or changed, right?

If that case how one can assure that the libmingwthrd library is linked in a correct order?


Furthermore, we don't have a minimum required version of mingw,

From what I can gather, if you're using the oldest version of GCC we are meant to support (4.8), then the oldest version of mingw you would be able to use is v3.

Could we use https://doc.qt.io/qt-5.15/windows.html to define a minimum required version of mingw?

@luke-jr
Copy link
Member Author

luke-jr commented Jan 3, 2022

My understanding is that for some reason (I forget), we don't/can't use -mthreads, so this is a somewhat hacky way to ensure we get the expected linking for threading support.

If we can use -mthreads with all supported versions now, that would probably be a superior fix. But until someone has time to properly evaluate that, this seems to be good enough for now?

@DrahtBot
Copy link
Contributor

DrahtBot commented Apr 5, 2022

🐙 This pull request conflicts with the target branch and needs rebase.

Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".

Copy link
Member

@fanquake fanquake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't convince myself that re-linking to mingwthrd is needed. I realise that adding this was the solution to an issue ~10 years ago, however I my current understanding is that this workaround was likely only needed for mingw, and that using mingw-w64, and only supporting 64 bit windows, has obsoleted any need for it.

GCC and mingw-w64 don't make trying to understand this easy. Both seem to be full of hacks / workarounds for each other, or just incorrect documentation. i.e mingw-w64 has the following related commentary:

/* mingw.org's version macros: these make gcc to define
   MINGW32_SUPPORTS_MT_EH and to use the _CRT_MT global
   and the __mingwthr_key_dtor() function from the MinGW
   CRT in its private gthr-win32.h header. */

However GCC hasn't cared about __MINGW32_MAJOR_VERSION or __MINGW32_MINOR_VERSION since 2009, where it started defining MINGW32_SUPPORTS_MT_EH unconditionally for _WIN32. See this commit.

@@ -667,6 +673,7 @@ case $host in
AC_MSG_ERROR([windres not found])
fi

dnl As with above, _MT is required and normally defined by -mthreads
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As with above, _MT is required

Can you explain what it's required for? I see that is is defined by GCC when using -mthreads. I also see that it's defined in mingw-w64.

However I can't seem to figure out what uses it. Nothing in our code, or dependency code seems to check whether it's defined, and/or modify it's behaviour. A Guix build run with/without it makes no difference.

I think if we are going add a comment about something being required, ideally we can mention why.

@@ -638,6 +638,12 @@ AC_ARG_WITH([daemon],
case $host in
*mingw*)
TARGET_OS=windows

dnl Some versions of MinGW may require libmingwthrd linked for a thread-safe stdlib implementation
dnl Normally, this is automatically linked by GCC's -mthreads flag, but for some reason (lost in the dust of time), that doesn't work when static linking
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but for some reason (lost in the dust of time)

I really don't think adding comments like this are valuable. It's more wording that doesn't explain anything to the reader, now, or in the future. This also seems like something that could just be tested, to (re-)determine why it doesn't work, rather than adding a comment to say we don't know why. "Some versions" is also ambiguous, and I'm fairly certain it can only be mingw-w64 1.x.

@fanquake
Copy link
Member

What is the status of this? Are you planning on rebasing and responding to review comments?

@laanwj
Copy link
Member

laanwj commented May 12, 2022

I can't convince myself that re-linking to mingwthrd is needed.

Also we explicitly require the POSIX threads implementation of mingw-w64. I'm quite sure mingwthrd is for win32-style threading in the old MinGW.

See also: msys2/MINGW-packages#9850

Closing.

@laanwj laanwj closed this May 12, 2022
@bitcoin bitcoin locked and limited conversation to collaborators May 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants