Compiled-in page size makes jemalloc binaries unportable #467
Description
I see that configure uses sysconf(_SC_PAGESIZE) at compile time to determine the page size. However, this can vary depending on kernel configuration, so a jemalloc compiled on one machine may have issues running on some other machine. For instance, Debian aarch64 uses 4k pages, but Fedora uses 64k, so a Debian jemalloc effectively can't run correctly on Fedora.
The concrete issue I encountered was in trying to bootstrap Rust aarch64 on Fedora. Because Rust is self-hosting, I have to use upstream binaries to get started. I was getting strange crashes, and when I tried strace I saw this:
2 madvise(0x3ff7a398000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7a3cd000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7a465000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7970b000, 65536, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7a6d6000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7a7aa000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7a7e0000, 8192, MADV_DONTNEED) = 0
2 madvise(0x3ff7a927000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7aab0000, 8192, MADV_DONTNEED) = 0
2 madvise(0x3ff79659000, 86016, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7aacf000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff778ae000, 28672, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7ad07000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7aeb7000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff7b0c2000, 8192, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 madvise(0x3ff779ba000, 200704, MADV_DONTNEED) = -1 EINVAL (Invalid argument)
2 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x8} ---
So it seems clearly expecting to have 4k pages. Most of the madvise calls were not on the real 64k page boundaries, so they get EINVAL. A few of them happen to align, so they succeed -- but the requested length was less than 64k and the kernel rounds up! Thus it will dump more memory than intended, and I suspect the SEGV addr=0x8 is a deref of a nulled-pointer from the cleared page.
I have managed to get around this with: export MALLOC_CONF=lg_dirty_mult:-1. But I think it would be better if jemalloc read the page size at runtime, and FWIW Firefox's forked "mozjemalloc" does exactly that.