compilation with stack fails because argument list to hsc2hs is too long #49206
Description
Issue description
I am seeing an error when using stack to compile Haskell code on NixOS. It appears that the argument list passed to hsc2hs is extremely long, and hsc2hs calls gcc with that long argument list. gcc fails to deal with an extremely long argument list.
This seems to be another symptom of #41340.
Steps to reproduce
I've created a reproduction with the haskell-gi package. You can try it out with the following steps:
$ git clone git@github.com:cdepillabout/haskell-gi.git
$ git checkout hsc2hs-error
$ stack build --verbose --cabal-verbose --ghc-options '-v3'
...
2018-10-27 12:14:01.626632: [info] Preprocessing library for haskell-gi-0.21.5..
2018-10-27 12:14:01.626660: [info] creating .stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/Data/GI/CodeGen
2018-10-27 12:14:01.766178: [info] /nix/store/f31slvv0xydlgf6jhvkyz5m6xchsa54q-ghc-8.4.3/bin/hsc2hs '--cc=/nix/store/10yq7kwlvbc6h658izmrlsspry1g9f3c-gcc-wrapper-7.3.0/bin/cc' '--ld=/nix/store/10yq7kwlvbc6h658izmrlsspry1g9f3c-gcc-wrapper-7.3.0/bin/cc' '--cflag=-fno-stack-protector' '--lflag=-fno-stack-protector' '--cflag=-D__GLASGOW_HASKELL__=804' '--cflag=-Dlinux_BUILD_OS=1' '--cflag=-Dx86_64_BUILD_ARCH=1' '--cflag=-Dlinux_HOST_OS=1' '--cflag=-Dx86_64_HOST_ARCH=1' '--cflag=-I/nix/store/mdfyly2rf93k0xfx9h6dagl4ycclkswg-gobject-introspection-1.56.0-dev/include/gobject-introspection-1.0' '--cflag=-I/nix/store/gbmk2qgalg06yz78fvhlyy87rzpqgjcy-glib-2.56.0-dev/include/glib-2.0' '--cflag=-I/nix/store/hknrb3kdacm9iqwky9bsj0ap45ngbc9z-glib-2.56.0/lib/glib-2.0/include' '--cflag=-I/nix/store/mdfyly2rf93k0xfx9h6dagl4ycclkswg-gobject-introspection-1.56.0-dev/include/gobject-introspection-1.0' '--cflag=-I/nix/store/gbmk2qgalg06yz78fvhlyy87rzpqgjcy-glib-2.56.0-dev/include/glib-2.0' '--cflag=-I/nix/store/hknrb3kdacm9iqwky9bsj0ap45ngbc9z-glib-2.56.0/lib/glib-2.0/include' '--cflag=-I.stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/autogen' '--cflag=-I.stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/global-autogen' '--cflag=-include' '--cflag=.stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/autogen/cabal_macros.h' '--lflag=-L/nix/store/f31slvv0xydlgf6jhvkyz5m6xchsa54q-ghc-8.4.3/lib' '--lflag=-L/nix/store/jg88la9hmpxj34gss9fsvkz9d983mb7x-glibc-locales-2.27/lib' '--lflag=-L/nix/store/sbmbixrjalyq1w679ypjfg90pq81b6ag-gobject-introspection-1.56.0/lib' '--lflag=-L/nix/store/hknrb3kdacm9iqwky9bsj0ap45ngbc9z-glib-2.56.0/lib' '--lflag=-Wl,-R,/nix/store/f31slvv0xydlgf6jhvkyz5m6xchsa54q-ghc-8.4.3/lib' '--lflag=-Wl,-R,/nix/store/jg88la9hmpxj34gss9fsvkz9d983mb7x-glibc-locales-2.27/lib' '--lflag=-Wl,-R,/nix/store/sbmbixrjalyq1w679ypjfg90pq81b6ag-gobject-introspection-1.56.0/lib' '--lflag=-Wl,-R,/nix/store/hknrb3kdacm9iqwky9bsj0ap45ngbc9z-glib-2.56.0/lib' '--lflag=-lgirepository-1.0' '--lflag=-lgobject-2.0' '--lflag=-lglib-2.0' '--cflag=-I/nix/store/0081p4ygwd0i54rzwqdzhbz13fv9vzwq-cups-2.2.6-dev/include' '--cflag=-I/nix/store/0713bi6xyfbh9fkal7h9y2nd8ricfplc-fribidi-1.0.5/include' '--cflag=-I/nix/store/1nha78cyiip6sb288sb7a9jfr7pxwjjf-gtk+3-3.22.30-dev/include' '--cflag=-I/nix/store/1vqby5ck2qzf7374iq30irhwk53z0v2l-libpng-apng-1.6.34-dev/include' '--cflag=-I/nix/store/27x4ncqbj01qas11mq875ym9y10l2w8d-freetype-2.9-dev/include' '--cflag=-I/nix/store/2fiabpwraw8d7qcb6i4aij7f0bpfl37n-libXrandr-1.5.1-dev/include' '--cflag=-I/nix/store/3ribnzxnl0vr6cxzqj8882dc58ip3cyi-renderproto-0.11.1/include' '--cflag=-I/nix/store/3zdsx2a7dama2bhj2dnhrn8c888dg50w-wayland-1.16.0/include' '--cflag=-I/nix/store/473mnkkmm8k6pjy5drzsfkxqyxsigwv2-vte-0.52.2/include' '--cflag=-I/nix/store/4mjzmf7kdhxazy3lvg8b45vj5q6gw1zc-expat-2.2.6-dev/include' ... (very long line with many, many duplicated arguments) ...
2018-10-27 12:14:02.743304: [warn] gcc: error trying to exec '/nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/libexec/gcc/x86_64-unknown-linux-gnu/7.3.0/cc1': execv: Argument list too long
2018-10-27 12:14:02.743809: [warn] compiling .stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/Data/GI/CodeGen/GType_hsc_make.c failed (exit code 1)
2018-10-27 12:14:02.986312: [warn] command was: /nix/store/10yq7kwlvbc6h658izmrlsspry1g9f3c-gcc-wrapper-7.3.0/bin/cc -c .stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/Data/GI/CodeGen/GType_hsc_make.c -o .stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/Data/GI/CodeGen/GType_hsc_make.o -fno-stack-protector -fno-stack-protector -D__GLASGOW_HASKELL__=804 -Dlinux_BUILD_OS=1 -Dx86_64_BUILD_ARCH=1 -Dlinux_HOST_OS=1 -Dx86_64_HOST_ARCH=1 -I/nix/store/mdfyly2rf93k0xfx9h6dagl4ycclkswg-gobject-introspection-1.56.0-dev/include/gobject-introspection-1.0 -I/nix/store/gbmk2qgalg06yz78fvhlyy87rzpqgjcy-glib-2.56.0-dev/include/glib-2.0 -I/nix/store/hknrb3kdacm9iqwky9bsj0ap45ngbc9z-glib-2.56.0/lib/glib-2.0/include -I/nix/store/mdfyly2rf93k0xfx9h6dagl4ycclkswg-gobject-introspection-1.56.0-dev/include/gobject-introspection-1.0 -I/nix/store/gbmk2qgalg06yz78fvhlyy87rzpqgjcy-glib-2.56.0-dev/include/glib-2.0 -I/nix/store/hknrb3kdacm9iqwky9bsj0ap45ngbc9z-glib-2.56.0/lib/glib-2.0/include -I.stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/autogen -I.stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/global-autogen -include .stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/build/autogen/cabal_macros.h -I/nix/store/0081p4ygwd0i54rzwqdzhbz13fv9vzwq-cups-2.2.6-dev/include -I/nix/store/0713bi6xyfbh9fkal7h9y2nd8ricfplc-fribidi-1.0.5/include -I/nix/store/1nha78cyiip6sb288sb7a9jfr7pxwjjf-gtk+3-3.22.30-dev/include -I/nix/store/1vqby5ck2qzf7374iq30irhwk53z0v2l-libpng-apng-1.6.34-dev/include -I/nix/store/27x4ncqbj01qas11mq875ym9y10l2w8d-freetype-2.9-dev/include -I/nix/store/2fiabpwraw8d7qcb6i4aij7f0bpfl37n-libXrandr-1.5.1-dev/include -I/nix/store/3ribnzxnl0vr6cxzqj8882dc58ip3cyi-renderproto-0.11.1/include -I/nix/store/3zdsx2a7dama2bhj2dnhrn8c888dg50w-wayland-1.16.0/include -I/nix/store/473mnkkmm8k6pjy5drzsfkxqyxsigwv2-vte-0.52.2/include -I/nix/store/4mjzmf7kdhxazy3lvg8b45vj5q6gw1zc-expat-2.2.6-dev/include -I/nix/store/5wa2iz55m5wxihqgyavn3aqlx9xgbkfq-libXft-2.3.2-dev/include -I/nix/store/667iiwrza3vijjan3kdjkim8lbqn0kgw-libXcomposite-0.4.4-dev/include -I/nix/store/7hhl013303zyvw3srgq3bm5674c9i5ka-pango-1.42.4-dev/include -I/nix/store/8hvap2i7mvmk0fbc187rz3gqsxr5pily-nettle-3.4-dev/include -I/nix/store/8mxjfpzh86w64rvx9yp9dz5y9xwx34z2-inputproto-2.3.2/include -I/nix/store/90ba2awivci6a2wzycgd65ib31irmky2-bzip2-1.0.6.0.1-dev/include -I/nix/store/a05xi83cwl4ar1107wnpfsadhzv77561-gsettings-desktop-schemas-3.28.0/include -I/nix/store/admjzx2dh8f3qfx5dlaqgpli0dm8nyb5-libGL-1.0.0-dev/include -I/nix/store/aihspv6bj7c9nypynrvk1jhbkz449mdg-pcre2-10.31-dev/include -I/nix/store/akg4r91sskr24vqyj5vjd1ypjvnf0d6p-kbproto-1.0.7/include -I/nix/store/asj2jl9wmzb8855lwkz35bhaw6nj849i-harfbuzz-1.8.8-dev/include -I/nix/store/avjpsh02a7m53dz98njdb8clwi451izn-libICE-1.0.9-dev/include -I/nix/store/b0xrl3mx0gfvb2pkqlzb9dcnhyjp0saa-xproto-7.0.31/include -I/nix/store/c12vcx8wjlymy0hlnndh5xrrm0andlaz-libX11-1.6.6-dev/include -I/nix/store/ciyl95gffhazl6b578ig2yls5ppck4z3-xcb-util-0.4.0-dev/include -I/nix/store/craky6lkv02qd5gh4ckdlrmbv4jdsvfi-graphite2-1.3.6/include ... (very long line with many, many duplicated arguments) ...
2018-10-27 12:14:02.986672: [debug] Start: getPackageFiles /home/illabout/git/haskell-gi/haskell-gi.cabal
2018-10-27 12:14:03.010702: [debug] Finished in 24ms: getPackageFiles /home/illabout/git/haskell-gi/haskell-gi.cabal
2018-10-27 12:14:03.010912: [debug] Encoding /home/illabout/git/haskell-gi/.stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/stack-build-caches/lib
2018-10-27 12:14:03.011104: [debug] Finished writing /home/illabout/git/haskell-gi/.stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1/stack-build-caches/lib
-- While building package haskell-gi-0.21.5 using:
/home/illabout/.stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_2.2.0.1_ghc-8.4.3 --verbose --builddir=.stack-work/dist/x86_64-linux-nix/Cabal-2.2.0.1 build lib:haskell-gi --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
I thought this might be fixed with haskell/cabal#5356, so I tried backporting it to Cabal-2.2 in ghc-8.4.3, but this did not appear to fix the problem. You can try out my fix like the following, although it recompiles GHC, so it takes a long time:
$ git clone git@github.com:cdepillabout/haskell-gi.git
$ git checkout hsc2hs-error-try-fix-cabal
$ stack build --verbose --cabal-verbose --ghc-options '-v3'
...
(fails with the exact same error as above)
Just like #41340, I guess this is ultimately a Nix problem, but it would be nice if Cabal could be fixed to not send duplicate arguments to hsc2hs. It would also be nice if hsc2hs would be fixed similarly.
Should I open up an issue on the Cabal or hsc2hs repos?
Pinging @nh2, @ElvishJerricco, @Ericson2314, @domenkozar, and @dmjio since they seemed to be active on issue #41340. Also pinging @23Skidoo, @IvanMalison, and @angerman since they were active on haskell/cabal#5356. Finally, pinging @garetxe since he is the author of the haskell-gi library, which I am using to cause the error above (although I don't think haskell-gi is particularly doing anything bad here).
Technical details
- system:
"x86_64-linux" - host os:
Linux 4.14.51, NixOS, 18.03.132768.94d80eb7247 (Impala) - multi-user?:
yes - sandbox:
no - version:
nix-env (Nix) 2.0.4 - channels(root):
"nixos-18.03" - channels(illabout):
"unstable-18.09pre145679.dae9cf6106d, nixos-18.03" - nixpkgs:
/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs