Skip to content

Reduce the internal formula API to contain the bare minimum data #20051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 6, 2025

Conversation

Rylan12
Copy link
Member

@Rylan12 Rylan12 commented Jun 5, 2025

The next step in #19204

This PR modifies the internal, sharded formula API to only include the bare minimum amount of data needed to download the bottle manifest file. The data needed, as far as I can tell, is simply the name, pkg_version, rebuild and the bottle checksum.


The format I landed on is as follows:

{
  "name": ["pkg_version", "rebuild", "checksum"],
  "a2ps": ["4.15.6", 0, "7effcbad404640b62723336fc0d1024f04f3cf98e4e40d595243512422897f4f"]
  ...
}

For the major supported platforms, this results in compressed (gzip) file sizes listed below. For reference, the formula.json file currently in use is 4.6 MB compressed (and 29 MB uncompressed).

Platform Compressed Size Uncompressed Size Size Compared to formula.json
arm64_sequoia 359.7 kB 674 kB 7.8%
arm64_sonoma 365.0 kB 683 kB 7.9%
arm64_ventura 365.0 kB 683 kB 7.9%
sequoia 103.1 kB 276 kB 2.2%
sonoma 365.4 kB 683 kB 7.9%
ventura 365.9 kB 684 kB 8.0%
x86_64_linux 365.8 kB 684 kB 8.0%

Compressed File Size for Storing Formula Metadata

Technically, this could be improved slightly by not including empty checksums or non-zero rebuild values. However, the best improvement I was able to get only decreased the compressed file size by an average of 1 kB. In my opinion, this slight improvement is not worth the added complexity.

All of the data and a chart that's easier to read are available here.


To do item for the future: figure out what happens if there is no bottle for a given platform. In some cases, I believe older bottles are sometimes still usable (I know this sometimes happens with new macOS releases). So, should we fall-back to the previous OS version if a given platform bottle doesn't exist?

@MikeMcQuaid
Copy link
Member

So, should we fall-back to the previous OS version if a given platform bottle doesn't exist?

Yes 👍🏻

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me, nice work!

@Rylan12
Copy link
Member Author

Rylan12 commented Jun 5, 2025

Okay, now the sha256 for a given system is populated as follows:

  1. Use the :all bottle if it exists
  2. Use the bottle whose tag matches the system if it exists
  3. If on macOS, use the newest bottle that exists and satisfies the following:
    • Has the same arch as the system
    • Is not newer than the system (i.e. arm64_sonoma can't use the arm64_sequoia bottle, but can use the arm64_ventura bottle)
  4. nil

@Rylan12 Rylan12 added this pull request to the merge queue Jun 6, 2025
Merged via the queue into master with commit 1f37a11 Jun 6, 2025
35 checks passed
@Rylan12 Rylan12 deleted the internal-api-bare-minimum branch June 6, 2025 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants