We continue trying to figure out in which ways we could optimize our Dune build setup at Ahrefs.

One of the things that quickly stands out is that when enabling --trace-file, we can see that the "Source tree scan" step of folder . takes more than 38s.

One of the instances of the Source tree scan events shows the folder backend/api/src, which takes ~37s. But after that event, there's only very short lived events which don't add much information.

Is there any way in which we could modify Dune to get more granular information about where the time is being spent? For example, if would be interesting to know if this time is spent parsing rules (of which dune files), executing commands that generate sexp files, etc. If those things is something that can happen during the Source tree scan stage, of course.

Please find below the trace file (obtained with dune build @fe.all @aa --trace-file fe.all-aa.txt) and a screenshot loading the trace in Brave (Chrome-like browser):

dune-build-fe.all-aa-trace.txt

Specifications

• Version of dune: 3.15.0
• Version of ocaml: 4.14.0
• Operating system (distribution and version): Debian 12, kernel 6.5.0