Checklist

• I've read and followed the Contribution Guidelines
• README.md is updated with the changes (if needed)

Version

Requires a MINOR version update

Context

The list of env vars passed to a build can grow really large in some cases. So large that it hits the exec syscall size limit and either the Bitrise CLI process can't be launched, or one of the subprocesses that the CLI launches (envman, step processes, etc.)

This happens most often when one of the Bitrise-generated env vars is huge, such as $BITRISE_GIT_CHANGED_FILES or $BITRISE_GIT_COMMIT_MESSAGE.

Changes

Like many other tools, we should support reading env vars from an env file on disk. This requires a change in the agent launching the CLI process, so this remains an optional feature and the fallback mechanism is the same as before.

The main idea behind this change is that once all env vars are saved to a file on disk, the actual env vars in the runtime environment can be cleared so that their size stays below that limit. The CLI just needs to reconstruct the original env vars from this envfile.

There are two areas where env vars are exposed to the user:

run_if templates

This PR patches the getenv, enveq and envcontain template functions so that they return the original env var values. There should be no breaking change here.

Env vars exposed to steps and scripts

It's not possible to expose arbitrary large env vars to the step subprocess because of the syscall limits mentioned above.

Therefore, custom steps are suggested to read the original env value from the env file. We present this warning:

This is going to be a breaking change for some custom scripts and tools. We minimize the impact by:

• Only erasing the offending env vars (not present in this PR, it's in the agent codebase launching the CLI). 99% of the env vars stay below the limit and they are exposed to the CLI and steps just like before.
• When an env var needs to be erased, the env var is still exposed, but with an empty string value. Tooling that relies only on the presence of an env var should not break.

Also, considering that the builds with env vars above the limit simply do not start today, this is going to be an improvement.

Decisions

We considered truncating env vars (to stay within the system limits), but that would lead to data corruption and hard-to-debug errors for users.

We also considered removing the offending env var altogether (instead of erasing its value), but that seems like a bigger breaking change than keeping the env var with an empty string value.