Skip to content

ArPow: Handle official build output that is too large to publish to Azure DevOps feeds #2034

@dagood

Description

@dagood

After getting dotnet/runtime to build with ArPow infra, the intermediate nupkg was 1.092 GB. This is significantly more than the <500 MB that I found a while ago when I took an official dotnet/runtime build and put all its outputs in a zip.

The big difference is symbol stripping. I got a 422 MB intermediate nupkg by enabling symbol stripping (normally disabled for source-build, so the distro can strip the symbols into debuginfo packages later on). Removing .symbols.nupkg (sympkgs) that ArPow doesn't need helps, but not a whole lot. Data:

  Remove sympkg Keep sympkg
Strip symbols (native) 182 MB 422 MB
Keep symbols 741 MB 1,092 MB

The AzDO feed limit is 500 MB. https://docs.microsoft.com/en-us/azure/devops/artifacts/reference/limits?view=azure-devops. But when I tried publishing the 422 MB file to a test feed, I wasn't able to get it to go through--I got a 503 error instead. 😕 I don't know if I'm hitting a bug, or if the true upper limit is somewhere lower than 500 MB.

The planning doc says we should just split it up. I don't see any problem with this yet. The ArPow infra in Arcade will probably need to be adjusted to allow it.

Stripping symbols and removing sympkgs to get down to 182 MB is an option. As long as downstream repos don't change their behavior when they get stripped vs. unstripped binaries, it don't affect anything. This is a regression risk, however.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-infraSource-build infrastructure and reporting

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions