Skip to content

Conversation

justinmk
Copy link
Member

Inherited signal mask may block SIGCHLD, which causes libuv to hang at
epoll_wait.

Closes #5230

@justinmk justinmk changed the title signal_init: Always unblock SIGCHLD. [RFC] signal_init: Always unblock SIGCHLD. Aug 24, 2016
@marvim marvim added the RFC label Aug 24, 2016
@nhooyr
Copy link
Contributor

nhooyr commented Aug 24, 2016

Sorry, I'm confused, why does vim not need to do this?

edit: I understand now, the key difference is epoll.

@justinmk
Copy link
Member Author

macos gcc failure is weird:

/Users/travis/build/neovim/neovim/src/nvim/os/signal.c: In function 'signal_init':
/Users/travis/build/neovim/neovim/src/nvim/os/signal.c:51:3: error: conversion to 'sigset_t' from 'int' may change the sign of the result [-Werror=sign-conversion]
   sigaddset(&mask, SIGCHLD);
   ^

@justinmk justinmk force-pushed the unblock-sigchld branch 2 times, most recently from 62874f7 to a71dea1 Compare August 24, 2016 19:59
@@ -45,6 +46,14 @@ void signal_init(void)
signal_watcher_init(&main_loop, &spwr, NULL);
signal_watcher_start(&spwr, on_signal, SIGPWR);
#endif

#ifdef SIGCHLD
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that it matters too much in this, but to indicate that this should be more like a "clean slate", it might be better to move this code to the top of the function.

@aktau
Copy link
Contributor

aktau commented Aug 24, 2016

LGTM

@justinmk
Copy link
Member Author

The macos gcc error seems insane.

@jszakmeister
Copy link
Contributor

I think I see where the OS X error is coming from:

from signal.h:

int  sigaddset(sigset_t *, int);
#define     sigaddset(set, signo)   (*(set) |= __sigbits(signo), 0)

from sys/_types/_sigset.h:

typedef __darwin_sigset_t            sigset_t;

and from sys/_types.h:

typedef __uint32_t   __darwin_sigset_t;      /* [???] signal set */

So it seems, OS X said sigset_t is unsigned, but then ORs in an int, mixing the types. So GCC seems correct here... sadly.

@aktau
Copy link
Contributor

aktau commented Aug 26, 2016

Something weird is definitely going on. OSX clang (the default compiler) with -Weverything doesn't warn for:

#include <stdio.h>
#include <signal.h>

int main() {
  sigset_t s;
  sigemptyset(&s);
  sigaddset(&s, SIGCHLD);
  return 0;
}

But gcc (6) does.

-std=c11 -Wall -Wextra -Wsign-conversion sig.c -o sig.o |& sed 's/^/gcc-6: /' ; clang -std=c11 -Weverything -Wsign-conversion sig.c -o sig.o |& sed 's/^/clang: /'
gcc-6: sig.c: In function 'main':
gcc-6: sig.c:7:3: warning: conversion to 'sigset_t {aka unsigned int}' from 'int' may change the sign of the result [-Wsign-conversion]
gcc-6:    sigaddset(&s, SIGCHLD);
gcc-6:    ^~~~~~~~~

Perhaps it's using different include paths. I think I've seen something similar to this before, from way back when, when I was working on the automatic C header to Lua FFI cleaner.

@aktau
Copy link
Contributor

aktau commented Aug 26, 2016

Something is suppressing those warnings. Running the preprocessor with both gcc and clang generates near identical files, barring some whitespace, standard defines and macros.

Still, the code looked like it should warn, so I did:

$ clang -Weverything sig.c -o sig
<no warning>
$ clang -Weverything $(mv =(clang -E sig.c) tmp.c ; echo tmp.c) -o sig.clang
sig.c:6:13: warning: implicit conversion changes signedness: 'int' to 'unsigned int' [-Wsign-conversion]
  (*(&s) |= __sigbits(20), 0);
         ~~ ^~~~~~~~~~~~~
1 warning generated.

Hah! I got you, clang, you conniving compiler...

@justinmk
Copy link
Member Author

So the system header is broken. Is the #pragma approach in this PR the appropriate way to address that?

@aktau
Copy link
Contributor

aktau commented Aug 26, 2016

So the system header is broken. Is the #pragma approach in this PR the appropriate way to address that?

I kinda want to know how Apple is avoiding those warnings. But I can't find it...

Inherited signal mask may block SIGCHLD, which causes libuv to hang at
epoll_wait.

Closes neovim#5230

Helped-by: Nicolas Hillegeer <nicolas@hillegeer.com>
Helped-by: John Szakmeister <john@szakmeister.net>

Note: the #pragma gymnastics are a workaround for broken system headers on
macOS.

  signal.h:
      int  sigaddset(sigset_t *, int);
      #define     sigaddset(set, signo)   (*(set) |= __sigbits(signo), 0)
  sys/_types/_sigset.h:
      typedef __darwin_sigset_t            sigset_t;
  sys/_types.h:
      typedef __uint32_t   __darwin_sigset_t;      /* [???] signal set */

sigset_t is defined as unsigned int, but the sigaddset() ORs it with an int,
mixing the types.  So GCC generates a sign-conversion warning:

  sig.c:9:13: warning: implicit conversion changes signedness: 'int' to 'unsigned int' [-Wsign-conversion]
    (*(&s) |= __sigbits((sigset_t) 20), 0);
           ~~ ^~~~~~~~~~~~~~~~~~~~~~~~
  1 warning generated.

System headers are normally ignored when the compiler generates warnings:
https://gcc.gnu.org/onlinedocs/cpp/System-Headers.html

  > GCC gives code found in system headers special treatment. All warnings,
  > other than those generated by ‘#warning’ (see Diagnostics), are suppressed
  > while GCC is processing a system header. Macros defined in a system header
  > are immune to a few warnings wherever they are expanded. This immunity is
  > granted on an ad-hoc basis, when we find that a warning generates lots of
  > false positives because of code in macros defined in system headers.

Instead of the #pragma workaround, we could cast the sigset_t pointer:

    # if defined(__APPLE__)
      sigaddset((int *)&mask, SIGCHLD);
    # else
      sigaddset(&mask, SIGCHLD);
    # endif

but that could break if the headers are later fixed.
@aktau
Copy link
Contributor

aktau commented Aug 27, 2016

I kinda want to know how Apple is avoiding those warnings. But I can't find it...
Hide all checks

So, I really can't find any pragmas. But I encountered this in the GCC docs:

-Wsystem-headers
Issue warnings for code in system headers. These are normally unhelpful in finding bugs in your own code, therefore suppressed. If you are responsible for the system library, you may want to see them.
https://gcc.gnu.org/onlinedocs/gcc/Preprocessor-Options.html

And

The header files declaring interfaces to the operating system and runtime libraries often cannot be written in strictly conforming C. Therefore, GCC gives code found in system headers special treatment. All warnings, other than those generated by ‘#warning’ (see Diagnostics), are suppressed while GCC is processing a system header. Macros defined in a system header are immune to a few warnings wherever they are expanded. This immunity is granted on an ad-hoc basis, when we find that a warning generates lots of false positives because of code in macros defined in system headers.
https://gcc.gnu.org/onlinedocs/cpp/System-Headers.html

Which implies that system headers are usually automatically warning-free. But the case is different for macro's. Macro's in system headers are not exempt from all warnings. Perhaps -Wsign-conversion is system-macro-exempt for Clang but not GCC? I'm unsure where to find an exhaustive list of these things...

That would explain why this happens:

$ clang-3.8 -std=c11 -Weverything -Wsign-conversion sig.c -o sig.o
$ clang-3.8 -std=c11 -Weverything -Wsign-conversion $(clang-3.8 -E sig.c -o tmp.c && echo tmp.c) -o sig.o
sig.c:6:13: warning: implicit conversion changes signedness: 'int' to 'unsigned int' [-Wsign-conversion]
  (*(&s) |= __sigbits(20), 0);
         ~~ ^~~~~~~~~~~~~
1 warning generated.

...in the second case, the code is inlined, and no longer in a system header.

@aktau
Copy link
Contributor

aktau commented Aug 27, 2016

...and another reference from the Clang devs from 2012. Still no word on where to find exactly which warnings/macro's are considered: http://clang-developers.42468.n3.nabble.com/Warnings-from-macro-expansions-vs-system-headers-td4028109.html

@aktau
Copy link
Contributor

aktau commented Aug 27, 2016

Man, that pragma solution is unbelievably ugly, but it's starting to look like the only way. libuv doesn't have the same warning levels as us, AFAIK. Perhaps we should file a bug to the GCC devs to make an exception for this one. Or... a rdar:// bug for the Apple folks, but something tells me they'll be less than responsive.

@jszakmeister
Copy link
Contributor

Given that we know how this is generated under OS X, we could maybe just cast the argument under OS X:

sigaddset(&s, (sigset_t) SIGCHLD);

It's not great either, but less ugly than the pragma stuff and a decent comment would help future readers know why we're doing.

Just a thought.

@aktau I've filed several development related bugs in the past for similar, and they're very unresponsive to them. :-(

@justinmk
Copy link
Member Author

@jszakmeister see the first smurf commit. That did not seem to work.

@aktau
Copy link
Contributor

aktau commented Aug 27, 2016

@jszakmeister as @justinmk mentioned, you cannot get around it with a cast. The problem is in the implementation of sigaddset itself. Clang tells us:

sig.c:9:13: warning: implicit conversion changes signedness: 'int' to 'unsigned int' [-Wsign-conversion]
  (*(&s) |= __sigbits((sigset_t) 20), 0);
         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.

Thre return value of __sigbits is int, not sigset_t.

NOTE: SIGCHLD = 20.

@jszakmeister
Copy link
Contributor

I tried it with gcc-5 before making the suggestion, but it looks like I just missed the warning. Sorry for the noise. I also didn't see the __sigbits() in there to help screw up that trick. Oh well.

Thre return value of __sigbits is int, not sigset_t.

Right... I wasn't trying to say it was, I just missed __sigbits being in there. It is, however, trying to OR an int and an unsigned int, and it was that which I was hoping to avoid. :-)

@jszakmeister
Copy link
Contributor

You could go the other way... since it's the mask that it's being OR'd into, you could cast it instead. That seems to get rid of the warning:

sigaddset((int *) &mask, SIGCHLD);

@justinmk
Copy link
Member Author

Ok, we still need to detect when this should be done. Because for systems that don't have such headers we could end up with the reverse problem....

@aktau
Copy link
Contributor

aktau commented Aug 29, 2016

You could go the other way... since it's the mask that it's being OR'd into, you could cast it instead. That seems to get rid of the warning:

Very clever!

Ok, we still need to detect when this should be done. Because for systems that don't have such headers we could end up with the reverse problem....

Exactly.

@justinmk
Copy link
Member Author

sigaddset((int *) &mask, SIGCHLD);

I would guess that will break if Apple fixes their headers. So I think disabling the warning for that single line is the least costly approach.

@justinmk
Copy link
Member Author

justinmk commented Aug 29, 2016

Updated, including notes from this discussion in the commit message. Will merge after build.

@justinmk justinmk merged commit 10a54ad into neovim:master Aug 29, 2016
@justinmk justinmk deleted the unblock-sigchld branch August 29, 2016 17:39
@justinmk justinmk removed the RFC label Aug 29, 2016
@aktau
Copy link
Contributor

aktau commented Aug 30, 2016

I would guess that will break if Apple fixes their headers. So I think disabling the warning for that single line is the least costly approach.

Indeed that would be the idea. At that point we'll know they fixed their headers and we can adjust the #ifdef to only apply to old Apple platforms, up until the point we decide not to support macOS with broken headers anymore.

@@ -29,6 +32,9 @@ static bool rejecting_deadly;

void signal_init(void)
{
// Ensure that SIGCHLD is unblocked, else libuv (epoll_wait) may hang.
Copy link
Contributor

@aktau aktau Aug 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could probably make a direct reference to the issue that caused it here.

(unsure what project policy is w.r.t that in this day and age.)

@aktau
Copy link
Contributor

aktau commented Aug 30, 2016

Said another way. if we go with the current way (pragmas), we might never know when Apple fixes this, and we might get stuck with technical debt forever. I believe this is how Vim reached its current state as well. Eternal manual vigilance vs automatic machine vigilance and a bit of fixer-upping. I know which one I'd chose.

aktau added a commit to aktau/neovim that referenced this pull request Sep 1, 2016
As discussed on neovim#5243.

The goal is to cleanup the code (temporarily ignoring warnings is really
ugly) _and_ to find out when Apple fixes their headers. This code will start
to warn whenever they do so. Thanks go to John Szakmeister for the idea.

Helped-by: John Szakmeister <john@szakmeister.net>
aktau added a commit to aktau/neovim that referenced this pull request Sep 3, 2016
As discussed on neovim#5243 and neovim#5283.

Helped-by: John Szakmeister <john@szakmeister.net>
Helped-by: Justin M. Keyes <justinkz@gmail.com>
justinmk pushed a commit that referenced this pull request Sep 4, 2016
As discussed on #5243 and #5283.

Helped-by: John Szakmeister <john@szakmeister.net>
Helped-by: Justin M. Keyes <justinkz@gmail.com>
justinmk added a commit to justinmk/neovim that referenced this pull request Oct 27, 2016
FEATURES:
0b5a7e4 neovim#4432 API: external UIs can render custom popupmenu
c6ac4f8 neovim#4934 API: call any API method from vimscript
31df051 neovim#4568 API: nvim_call_atomic(): multiple calls in a single request
b268ba3 neovim#5424 API: nvim_win_get_number(), nvim_tabpage_get_number()
e7e2844 has("nvim-1.2.3") checks for a specific Nvim version
522b885 neovim#5295, neovim#5493 `:CheckHealth` checks tmux, terminfo, performance
719dae2 neovim#5384 events: allow event processing in getchar()
f25797f neovim#5386 API: metadata: Nvim version & API level
22dfe69 neovim#5389 API: metadata: "since", "deprecated_since"
605e743 Added QuickFixLine highlight group

CHANGES:
4af6ec7 neovim#5253 perf: Disable clipboard in do_cmdline()
6e9f329 neovim#5299 perf: Skip foldUpdate() in insert-mode.
9d4fcec neovim#5426 perf: Do not auto-update folds for some foldmethods.
eeec0ca neovim#5419 tui: Default to normal-mode cursor shape.

FIXES:
e838452 neovim#5436 tui: Fix "weird characters" / "bleeding termcodes"
10a54ad neovim#5243 signal_init: Always unblock SIGCHLD.
bccb49b neovim#5316 eval.c: Fix memory leak for detached pty job
626065d neovim#5227 tchdir: New tab should inherit CWD.
cd321b7 neovim#5292 getcwd(): Return empty string if CWD is invalid.
6127eae shada: Fix non-writeable ShaDa directory handling
ca65514 neovim#2789 system(): Respect shellxescape, shellxquote
2daf54e neovim#4874 Restore vim-like tab dragging
0c536b5 neovim#5319 syntax.c: Support bg/fg special color-names.
3c53371 neovim#4972 from justinmk/schedule-ui_refresh
68bcb32 neovim#4789 tui.c: Do not wait for tui loop on teardown.
c8b6ec2 neovim#5409 v:count broken in command-line window
6bc3bce neovim#5461 fix emoji display
51937e1 neovim#5470 fix :terminal with :argadd, :argu
79d77da neovim#5481 external UIs: opening multiple files from command-line
657ba62 neovim#5501 rplugin: resolve paths in manifest file
6a6f188 neovim#5502 system('foo &', 'bar'): Show error, don't crash.
1ff162c neovim#5515 os_nodetype: open fd with O_NONBLOCK
2a6c5bb neovim#5450 modeline: Handle version number overflow.
0ade1bb neovim#5225 CI tests now run against Windows!
justinmk added a commit to justinmk/neovim that referenced this pull request Oct 28, 2016
FEATURES:
0b5a7e4 neovim#4432 API: external UIs can render custom popupmenu
c6ac4f8 neovim#4934 API: call any API method from vimscript
31df051 neovim#4568 API: nvim_call_atomic(): multiple calls in a single request
b268ba3 neovim#5424 API: nvim_win_get_number(), nvim_tabpage_get_number()
e7e2844 has("nvim-1.2.3") checks for a specific Nvim version
522b885 neovim#5295, neovim#5493 `:CheckHealth` checks tmux, terminfo, performance
719dae2 neovim#5384 events: allow event processing in getchar()
f25797f neovim#5386 API: metadata: Nvim version & API level
22dfe69 neovim#5389 API: metadata: "since", "deprecated_since"
605e743 Added QuickFixLine highlight group

CHANGES:
4af6ec7 neovim#5253 perf: Disable clipboard in do_cmdline()
6e9f329 neovim#5299 perf: Skip foldUpdate() in insert-mode.
9d4fcec neovim#5426 perf: Do not auto-update folds for some foldmethods.
eeec0ca neovim#5419 tui: Default to normal-mode cursor shape.

FIXES:
e838452 neovim#5436 tui: Fix "weird characters" / "bleeding termcodes"
10a54ad neovim#5243 signal_init: Always unblock SIGCHLD.
bccb49b neovim#5316 eval.c: Fix memory leak for detached pty job
626065d neovim#5227 tchdir: New tab should inherit CWD.
cd321b7 neovim#5292 getcwd(): Return empty string if CWD is invalid.
6127eae shada: Fix non-writeable ShaDa directory handling
ca65514 neovim#2789 system(): Respect shellxescape, shellxquote
2daf54e neovim#4874 Restore vim-like tab dragging
0c536b5 neovim#5319 syntax.c: Support bg/fg special color-names.
3c53371 neovim#4972 from justinmk/schedule-ui_refresh
68bcb32 neovim#4789 tui.c: Do not wait for tui loop on teardown.
c8b6ec2 neovim#5409 v:count broken in command-line window
6bc3bce neovim#5461 fix emoji display
51937e1 neovim#5470 fix :terminal with :argadd, :argu
79d77da neovim#5481 external UIs: opening multiple files from command-line
657ba62 neovim#5501 rplugin: resolve paths in manifest file
6a6f188 neovim#5502 system('foo &', 'bar'): Show error, don't crash.
1ff162c neovim#5515 os_nodetype: open fd with O_NONBLOCK
2a6c5bb neovim#5450 modeline: Handle version number overflow.
0ade1bb neovim#5225 CI tests now run against Windows!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants