There seems to be something wrong on macOS... Maybe the spec is blocking, somehow.

EDIT: I disabled the spec on darwin, and apparently it passes.

EDIT: yes, one fiber fails not_nil for some reason 😢

  can read/write fifo file without blocking

Unhandled exception in spawn: Nil assertion failed (NilAssertionError)
  from src/nil.cr:113:7 in 'not_nil!'
  from src/nil.cr:109:3 in 'not_nil!'
  from spec/std/file_spec.cr:51:3 in '->'
  from src/wait_group.cr:68:13 in '->'
  from src/fiber.cr:170:11 in 'run'
  from src/fiber.cr:105:3 in '->'

The error is easy to fix: wait for the thread to be terminated so either the writer is opened or it will re-raise an exception.

Problem is, with less than 4 reads/writes then the spec terminates, but starting with 4+ read/write loops, then it eventually blocks 😭

Here's the evloop tracing, with execution contexts enabled:

evloop.write 2257866931450 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3600:writer fd=9
evloop.write 2257866943212 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3600:writer fd=9
evloop.wait_writable 2257866948573 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3600:writer fd=9
evloop.kevent 2257866953322 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3600:writer op=add fd=9 index=38654705664
evloop.read 2257866967971 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3540:reader fd=10
evloop.read 2257866980312 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3540:reader fd=10
evloop.wait_readable 2257866986096 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3540:reader fd=10
evloop.kevent 2257866992343 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3540:reader op=add fd=10 index=42949672960
evloop.run 2257867003458 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3540:reader blocking=0
evloop.event 2257867025557 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3540:reader fd=9 index=38654705664 filter=-2 flags=33 fflags=0
evloop.event 2257867043832 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3540:reader fd=10 index=42949672960 filter=-2 flags=33 fflags=0
evloop.done_writable 2257867050735 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3600:writer fd=9
evloop.write 2257867057484 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3600:writer fd=9
evloop.wait_writable 2257867062706 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3600:writer fd=9
evloop.run 2257867067933 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3600:writer blocking=0
evloop.run 2257867646682 thread=0x7ff859141a00:DEFAULT fiber=0x1103c3cc0:DEFAULT:loop blocking=1

It might an issue with the kqueue event loop: there's one event for fd=9 (writer) and fd=10 (reader) but only the writer seems to be resumed.

EDIT: it doesn't reproduce on FreeBSD.

EDIT: there's definitely something wrong on macOS:

• writer fiber: waits for writable (fd=9)
• writer fiber: runs the evloop
• writer fiber: gets a kevent for fd=9
• ... nothing is enqueued 🤨

EDIT: the kevent.filter is... the opposite of what it should be: it reports EVFILT_READ for the writer (fd=9), and EVFILT_WRITE for the reader (fd=10), which leads to resume nothing because there's no waiter for the reported readiness

EDIT: If I reverse the reader and the writer spawns, then the fibers start talking, then it waits for write, we get an kevent that fd=9 and fd=10 are ready for write, which resumes the writer that ends up waiting again... then we never get the read readiness kevent for fd=10 and it hangs forever.

AFAICT: Non-blocking fifo read/write is broken on macOS for some reason. Maybe it gets confused by the two EVFILT_READ + EVFILT_WRITE kevent registration... but that doesn't happen for sockets, so that's very weird.

Maybe it gets confused because the fifo file is opened by the same thread in the same process, which is maybe an odd case?

I tried always resuming a reader along with a writer, but to no avail: it does resume both fibers but both read and write will fail with EAGAIN 😭

EDIT: I tried libevent: the fifo read/write also hangs forever on macOS, aka it's macOS that's broken 🤷

Files now default to blocking on macOS because polling of non-blocking fifo files appears to be incorrect (at the OS level). This reinforces the argument that we should never assume any specific configuration for the system fd 😓

I investigated some more, using two threads as well as two processes (one reader, one writer) and sending/reading enough data to overflow the kernel buffers: the FIFO communication always gets stuck whenever I set O_NONBLOCK on macOS, but always succeeds when I keep the fd to blocking.

I.e. non-blocking FIFO doesn't work on macOS.