Skip to content

Conversation

AdallomRoy
Copy link
Contributor

@AdallomRoy AdallomRoy commented Jun 20, 2025

Describe your PR and link to any relevant issues.

This is a performance improvement set to bulk load packages wherever possible - it has major performance implications - in a large monorepo it improved generation time from 1:30m to 24s

I have:

  • Added tests covering the bug / feature (see testing)
  • Updated any relevant documentation (see docs)

@AdallomRoy AdallomRoy force-pushed the master branch 2 times, most recently from f7b7952 to b59a78f Compare June 20, 2025 13:48
@coveralls
Copy link

coveralls commented Jun 20, 2025

Coverage Status

coverage: 73.069% (+0.05%) from 73.016%
when pulling 40450e5 on AdallomRoy:master
into a31de27 on 99designs:master.

@AdallomRoy
Copy link
Contributor Author

@StevenACoffman would really appreciate your attention here as this improves generation performance by 2-3x in large monorepos... Thank you!

Copy link
Collaborator

@StevenACoffman StevenACoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! I really appreciate you contributing this. One of the most valuable things is that you actually benchmarked it in a real large monorepo!

It's very hard for me to guess where to spend time on improving performance when it might turn out to have been a bad trade-off in practice (extra complexity might not be worth a few ms).

I'm happy to merge it as is, but I had a few possible suggestions if you were interested in them.

This is an unpaid volunteer community led project, so we really appreciate your contributions, and would love to hear feedback on any specific pain points or areas we could improve on.

@AdallomRoy
Copy link
Contributor Author

Hi @StevenACoffman ! Thanks for your review, I will incorporate your fixes.
I found another minor improvement (which I will push here)
Just to give you some data, I added some logging, and these are my main results (after the second improvement):
Loading 16 package names done: 2819
Loading 14 packages done: 7323
Loading 23 packages done: 5141
Loading 23 packages done: 5313
Loading 9 package names done: 3158
Binomial took 30.057885167s

Out of 30s of generation, roughly 23s are package loading - so this is very significant.
I have some improvements to the Go SDK, hopefully those will be acceptable at some point.

@StevenACoffman StevenACoffman merged commit be74a5b into 99designs:master Jun 24, 2025
18 checks passed
@StevenACoffman
Copy link
Collaborator

Thanks very much! If you want to have a quick chat about the broader changes you are thinking about, I'm always happy to do that too. I'm in UTC-4 so I can either get up early or stay up late. Feel free to send me something at gears@umich.edu or https://www.linkedin.com/in/steve-coffman-79322175/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants