[pilot] ability to upload ipas to Testflight from Linux #22014
Conversation
- add extra values to XMLTemplate.xml.erb and the IpaUploadPackageBuilder generator to support uploads from Linux - read straight from plist_data in IpaFileAnalyser - adapt tests - update readme
- make sur only 1 of auth args are used in the ShellScriptTransporterExecutor upload command - prevent ItunesTransporter asking for user input when jwt or api key is present - add extra test to expect the api key in the shell script command
|
Let me know what are your thoughts on these additional observations and if they should be addressed here, in a follow up PR, or not at all:
|
| @@ -686,7 +722,7 @@ def initialize(user = nil, password = nil, use_shell_script = false, provider_sh | |||
| use_shell_script ||= Helper.windows? | |||
| use_shell_script ||= Feature.enabled?('FASTLANE_ITUNES_TRANSPORTER_USE_SHELL_SCRIPT') | |||
|
|
|||
| if jwt.to_s.empty? | |||
| if jwt.to_s.empty? && api_key.nil? | |||
There was a problem hiding this comment.
can anyone think of any reason to continue asking the user for input when the api key is set?
|
@AliSoftware maybe? 👀 |
|
Any progress on this? This would make our workflow much smoother 😄 |
|
@rogerluan maybe you can have a look? 🙏🏻 |
|
Is there any update on this PR? MacOS runners are currently eating up a lot of our CI budget and it would be awesome to be able to perform at least this step using a Linux machine |
|
What's needed to get this PR merged? |
|
What's still needed to get this PR merged through and released? |
mostly for the maintainers to come back from vacation and review the PR 😕 in the meantime, if you want to give a shot to the changes I made then create the following source 'https://rubygems.org'
gem 'fastlane', git: 'https://github.com/ccaruceru/fastlane.git', branch: 'feature/update-linux-itms-transporter'do a |
|
@AliSoftware @joshdholtz @lacostej could we get some eyes on this PR? It seems like a number of people are waiting on this functionality in our use of fastlane |
|
ohhh I would love to use this too! I'll test it today |
|
it works!!! |
|
@AliSoftware @joshdholtz @lacostej bumping this again 🙏🏼 |
|
I'm going to bump this review up as well as this would really help us out if this work. I'm going to guess that this also then works for deliver as well if it works for pilot. |
| (if jwt.nil? | ||
| if api_key.nil? | ||
| if username.nil? || escaped_password.nil? | ||
| nil | ||
| else | ||
| "-u #{username.shellescape} -p #{escaped_password}" | ||
| end | ||
| else | ||
| "-apiIssuer #{api_key[:issuer_id]} -apiKey #{api_key[:key_id]}" | ||
| end | ||
| else | ||
| "-jwt #{jwt}" | ||
| end), |
There was a problem hiding this comment.
Same DRY-ing suggestion as above.
Actually, one could even consider extracting the credential_params variable I suggested in comment above into a dedicated helper method on the Transporter class (just like the prepare one you added), so we can then add dedicated tests for that helper function that would cover all the different cases (with or without JWT, with or without API key, with or without username/password, with a mix of all those different options, …).
e.g. something probably like this (not tested, might contain typos):
describe '#credential_params' do
shared_examples 'credential_params' do |jwt: nil, username: nil, password: nil, api_key: nil, expected:|
input_params = { jwt: jwt, username: username, password: password, api_key: api_key }
context "when passing #{input_params.compact.keys.join(', ')" do
it "calls the command line with #{expected}" do
cmd_line_param = transporter.credential_params(**input_params)
expect(cmd_line_params).to eq(expected)
end
end
end
# no command line param if nothing provided
it_behaves_like 'credential_params',
expected: []
# if we only provide JWT and nothing else
it_behaves_like 'credential_params',
jwt: jwt,
expected: ['-jwt', jwt]
# if we only provide username and password
it_behaves_like 'credential_params',
username: email, password: password,
expected: ['-u', email.shellescape, '-p', password.shellescape]
# if we only provide API key
it_behaves_like 'credential_params',
api_key: api_key,
expected: ['-apiIssuer', api_key[:issuer_id], '-apiKey', api_key[:key_id]]
# If we provide both the username and password AND an api_key, api_key takes precedence
it_behaves_like 'credential_params',
username: email, password: password,
api_key: api_key,
expected: ['-apiIssuer', api_key[:issuer_id], '-apiKey', api_key[:key_id]]
# … add all other combinations here
endThere was a problem hiding this comment.
not a bad idea 👌🏻
made a new method in TransporterExecutor called build_credential_params, overwrote that in all subclasses and added tests as you suggested.
|
@ccaruceru I was able to validate your work in a Docker container and uploaded an ipa to TestFlight using Debian. Any chance you could make the suggested changes? |
- rename api key param - update doc
Sorry it took so long for me to reply. I was away for quite a while and now I'm back. I'll check the PR these days and address any comments. |
| [ | ||
| '"' + Helper.transporter_path + '"', | ||
| '-m provider', | ||
| ("-u \"#{username.shellescape}\"" unless use_jwt), |
There was a problem hiding this comment.
the new build_credential_params doesn't surround the username with "" anymore.
this is the only place in ShellScriptTransporterExecutor that uses quotes and I tried tracing this back to #13174, but couldn't find a reason to keep them nor a reason why this should be different from the other commands
There was a problem hiding this comment.
Yeah I agree that I wouldn't expect the username to be both shellescaped and surrounded by quotes, not sure why it was before, probably a bug that got unnoticed. 👍
|
@AliSoftware this is ready for a second review 🔎 |
| ("-u #{username.shellescape}" if api_key.nil?), | ||
| ("-p #{password.shellescape}" if api_key.nil?), |
There was a problem hiding this comment.
I know that's just you moving the existing code from former lines 283–286 into here without changes… but doesn't that logic mean that it'll add -u nil if api_key is nil… even jwt isn't nil? 💭
Wouldn't it make more sense to add those only if username.nil? / password.nil? respectively instead? 🤔
There was a problem hiding this comment.
I see. It makes more sense to check for username and password rather than presence/absence of api_key. I tried to look at #20631 when this was introduced, but couldn't find a reason why should be like that 🤷🏻♂️
I also observed that we're always returning one combination of user/pass, jwt, or api key, and it would be more appropriate to change build_credential_params to return a string with the right credential combination instead of an array.
There was a problem hiding this comment.
@AliSoftware can you have one more look?
| [ | ||
| '"' + Helper.transporter_path + '"', | ||
| '-m provider', | ||
| ("-u \"#{username.shellescape}\"" unless use_jwt), |
There was a problem hiding this comment.
Yeah I agree that I wouldn't expect the username to be both shellescaped and surrounded by quotes, not sure why it was before, probably a bug that got unnoticed. 👍
|
@AliSoftware is this okay to be merged? |
|
btw we should update the docs https://docs.fastlane.tools/actions/pilot/#pilot |
docs should be updated with fastlane/docs#1282 |
|
anyone else facing issues with this recently? seems like the transporter is stuck uploading the IPA to TestFlight (i'm checking if it's a transporter regression.. probably the case) |
|
well downgrading the iTunesTransporter to 3.4.1 indeed fixes the problem.. |
|
The latest version of the transporter works for me. I’m using a Rocky Linux 9 docker image with Ruby, Fastlane, and the transporter installed |
I tried to check feature in Rocky Linux 9, Fastlane version 2.228.0 and iTMSTransporter version 4.1. 11:47:30: [iTMSTransporter] [2025-09-10 11:47:30 UTC] DEBUG: SMART-CLIENT: getCurrent(); setting current data center to: contentdelivery01.itunes.apple.com11:47:30: [iTMSTransporter] [2025-09-10 11:47:30 UTC] INFO: Setting transport log file: ac0cdb7b-e347-4bb0-b0c4-665a90d480dc15540698444925637104.tx.log11:47:30: [iTMSTransporter] [2025-09-10 11:47:30 UTC] INFO: Configuring the Software Uploader...11:47:30: [iTMSTransporter] [2025-09-10 11:47:30 UTC] INFO: Performing analysis...11:47:30: [iTMSTransporter] [2025-09-10 11:47:30 UTC] ERROR: Unable to perform software analysis on Linux. Export an AppStoreInfo.plist from Xcode, and use the -assetDescription option.11:47:30: [iTMSTransporter] Package Summary: 11:47:30: [iTMSTransporter] 1 package(s) were not uploaded because they had problems: 11:47:30: [iTMSTransporter] /tmp/d20250910-47-xuzuao/1500868687-3c0c56f4-93f3-4058-85e2-2dd348c06739.itmsp/bbd528aa542af044f34a659fcb5a638406ec1052eff25f206da5be3ceca44551.ipa - Error Messages: 11:47:30: [iTMSTransporter] Unable to perform software analysis on Linux. Export an AppStoreInfo.plist from Xcode, and use the -assetDescription option. 11:47:30: [iTMSTransporter] [2025-09-10 11:47:30 UTC] DBG-X: Returning 111:47:30: iTunes Transporter output above ^ Please, can you say me what's may be wrong? |
Checklist
bundle exec rspecfrom the root directory to see all new and existing tests passbundle exec rubocop -ato ensure the code style is validci/circlecibuilds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)Motivation and Context
Make Testflight upload work from Linux using Transporter and potentially reduce CI costs on systems where running on a macOS is expensive (like Github Actions). This requires having Transporter installed on the machine.
Description
Add support for uploading an ipa to App Store Connect via api key when running
fastlane pilotfrom a Linux machine. In order for this to happen, two major changes are needed:ShellScriptTransporterExecutor(and surrounding code) to support API keys in theuploadmethodmetadata.xmlfile in the.itmspfolder so that Transporter's asset validation process passes. This means the-f <.itmsp>argument works without having to use-assetFile <ipa>in combination with the-assetDescription <plist>arguments (which improves users experience on linux).In this PR:
IpaFileAnalyser(see CFPropertyList::List.initialize())Dir.mktmpdirfrom existing testsBuildManager,IpaUploadPackageBuilderand theXMLTemplate.xml.erbto add bundle identifier, short version and bundle version valuesTransporterExecutor.preparemethod and call that before everyexecutecommand to create the file on disk in different places, depending on what kind of executor instance we deal withShellScriptTransporterExecutor.uploadand make sure only one of user/pass, jwt or api key is used at any momentTransporterExecutor.build_credential_paramsmethod to create a list of credential parameters that each subclass will implementItunesTransporter.initializewhen api key is notnilitunes_transporter_spec.rband adapt tests inbuild_manager_spec.rbto use a real ipa and expect new calls and valuesTested on Linux (Debian 12) and MacOS (Sonoma 14).
Testing Steps