Update: Current status

This is now implemented and released in jj 0.26, but not yet the default. See https://jj-vcs.github.io/jj/latest/config/#git-subprocessing-behaviour. TLDR: jj config set --user git.subprocess=true.

Is your feature request related to a problem? Please describe.

Many jj git commands fail on less common setups, oftentimes because libgit2 doesn't support certain features (see additional context).

libgit2 itself uses libssh2 rather than OpenSSH, so it often doesn't behave the way that users would expect with their current git/ssh usage.

We've discussed having jj git commands literally shell out to git, rather than using the libgit2 implementation, most recently in #4937 (comment). I'm opening this issue to justify and track that proposal.

Describe the solution you'd like

Change jj git commands to execute git directly where appropriate.

This is not risk-free. For example, git-branchless does this and instead has issues like git sync --pull, git submit hang indefinitely when asking for password (arxanas/git-branchless#433) (note that it tries to run git in the background and communicate via stdin/stdout). But there are substantially fewer networking-related issues there, so it seems like a better default state of affairs for most users.

Describe alternatives you've considered

• Continue using libgit2+libssh2: It seems like there's a significant number of issues caused by lack of support in libgit2, to the point that it impedes new and existing users, and we don't have the bandwidth or expertise to address them all.
• Switch to Gitoxide: I don't think Gitoxide currently supports all of these workflows, and might not in the future. Gitoxide even shells out to git at present for certain use-cases (reading certain config?).

Additional context

Here's my enumeration of issues that might be resolved or resolveable by shelling out to Git. It would be nice if we could mostly solve them with a single mechanism!

SSH:

• git: SSH config not respected #63
• jj git clone doesn't support interactive authentication #401
• jj git over ssh fails with sha-1 key #440
• Support authenticated HTTP #469
• [FR] Add ability to connect to a single host using different keys per repo/remote #1455
• FR: better error for ssh hostname changes #1507
• `jj git push` stuck because of SSH authentication #1979
• Https authorization to github #2724
• ssh:// remote paths not supported. #2931
• jj git clone fails with ecdsa-sk hardware SSH key #2958
• jj git push is not working for me on Windows #3322
• libssh2-sys dependency won't compile on windows #3984
• jj can't set up new git credential.helper entries #4101
• Error: invalid or unknown remote ssh hostkey (when using /etc/ssh/ssh_known_hosts) #4333
• SSH login with gpg keycard backed ssh key and pinentry-curses for pin fails #4386
• failed to authenticate SSH session #4488
• Failing authentication for jj git fetch ... / jj git clone ... / etc. with a FIDO2 (resident) key #4591
• Unable to connect to Sourcehut over SSH on Windows with JJ #4802
• libssh2 error in jj git push to GitHub repository: can't authenticate on macOS  #4870
• FR: Support SSH authentication through GIT_SSH_COMMAND #4937
• SSH with 1Password Fails to Authenticate #4978
• Explain "failed to set hostkey preference" error somewhere #5027
• Getting "Error: invalid or unknown remote ssh hostkey; class=Ssh (23); code=Certificate (-17)" on MacOS #5120
• jj with https for a private, hosted github #5166
• FR: use a custom ssh-agent #5479

Clone/fetch/push/pull:

• jj doesn't handle negative refspecs correctly? #360
• add support for pushing to tracked remote #1278
• How to fetch from a local copy in a "non-colocated" repo? #1822
• Cannot run jj git push #1957
• Bug/FR: jj git fetch --branch main should fetch tags #2295
• fetch: SSL error: unknown error #3851
• git commands with an sso origin fail with Error: invalid argument: 'port'; class=Invalid (3) #4177
• jj git clone not cloning all tags #4682
• jj in shallow clone is unable to move bookmark sideways #4719
• jj git push ignores multiple push urls #4889
• Adding a git remote or cloning with SSH on a non-standard port #5147
• jj git push does not try all ssh keys #5238
• FR: Honor remote refspec definitions when fetching from git colocated workspaces #5323
• unable to to clone public GitLab repos using SSH #5467
• FR: Remote helper support #5518