Skip to content

Conversation

rposky
Copy link
Contributor

@rposky rposky commented May 16, 2024

This attempts to address issue #1993.

With this change, all stream I/O is moved to the head of get_binary_packet, with applicable timeouts applied, and processing of the payload only commences once read in full. This ensures that an under-read of the remainder of the packet from the stream does not lead to false-positive errors during subsequent processing, as with e.g. potential for decrypt errors in handling of the $tag value in GCM/ChaCha decrypt logic. The partial packet is also buffered in the repurposed binary_packet_buffer such that graceful re-entry into get_binary_packet is accommodated during timeout events, where packets previously could be lost and client corruption was a possibility with the remainder of the packet left in the buffer.

The $skip_channel_filter parameter is removed from both get_binary_packet and filter method signatures. It was previously used to enact timeout enforcement at the head of get_binary_packet, but became extraneous to that end with the added timeout enforcement of c20dd78 and its predecessor, i.e. with all paths through get_binary_packet now enforcing timeouts. Some channel packet type cases (MessageType::CHANNEL_DATA, MessageType::CHANNEL_EXTENDED_DATA, MessageType::CHANNEL_CLOSE, MessageType::CHANNEL_EOF) were removed from filter() packet processing, deemed extraneous considering that exec() no longer consumes get_binary_packet directly, as was the case when a3325d1 was authored, preferring get_channel_packet, and that get_channel_packet already loops over get_binary_packet to prompt fetch of the next packet under appropriate circumstances, i.e. such as receipt of packet on another channel. This allowed for some cleanup within get_channel_packet as well.

The exception InvalidPacketLengthException is introduced and thrown from the added get_binary_packet_size when encountering a bad packet_length value. This maintains handling of the OpenSSL 0.9.8e issue (reported in #1171 , changed with e5b4eef) from within _login_helper without importing the knowledge of the issue into low-level packet processing. Validation is added for a packet_length value of 0 or negative value.

Lastly, reset_connection and disconnect_helper are reorganized such that any use of disconnect_helper will now prompt reset of the SSH2 instance, or at least of what was previously captured by reset_connection, and including binary_packet_buffer and added keep_alive_sent.

Through the added SSH2 functional test testCryptoAlgorithms(), I was able to test the following algorithms and also provide decent coverage of the changes. See attachments coverage-getbinarypacket-crap4j.txt and coverage-getbinarypacket-cobertura.txt for excised coverage analysis of the functions get_binary_packet and get_binary_packet_size. Lack of coverage is mostly reflected in the error/breakout states, debug logging, and the zlib compression implementation, however unmodified here.

KEX:
- curve25519-sha256
- curve25519-sha256@libssh.org
- ecdh-sha2-nistp256
- ecdh-sha2-nistp384
- ecdh-sha2-nistp521
- diffie-hellman-group14-sha256
- diffie-hellman-group14-sha1
- diffie-hellman-group1-sha1

HostKey:
- rsa-sha2-256
- rsa-sha2-512
- ssh-rsa

Crypt:
- aes128-gcm@openssh.com
- aes128-ctr
- aes192-ctr
- aes256-ctr
- chacha20-poly1305@openssh.com

MAC:
- hmac-sha2-256-etm@openssh.com
- hmac-sha2-256
- hmac-sha1-96
- hmac-sha1

In order to prompt testing of the chacha20-poly1305@openssh.com encryption algorithm, I had to comment out this logic in getSupportedEncyrptionAlgorithms(). I only dug a little, but it looked to me like chacha20-poly1305@openssh.com might not be viable according to this as it doesn't provide a setupInlineCrypt() method or invoke createInlineCryptFunction(). Something to look into perhaps.

case 'chacha20-poly1305@openssh.com':
case 'arcfour128':
case 'arcfour256':
if ($engine != 'Eval') {
    continue 2;
}
break;

rposky added 4 commits May 16, 2024 14:41
…rocessing, buffering for graceful handling across timeouts.

Remove skip filter parameter from method signatures, now technically defunct as all requests through get_binary_packet incorporate the same timeout during blocking IO calls.
Introduce InvalidPacketLength exception and employ to detect OpenSSL 0.9.8e flaw at higher logical level than binary packet processing.
Removing case logic in binary packet filtering for channel message types, made extraneous by use of get_channel_packet, and possibly leading to discarded data packets.
Reset connection properties during disconnect. Rework callers of reset_connection to use disconnect_helper.
Bugfix for no encyrption algorithms negotiated with server.
@terrafrost
Copy link
Member

I only dug a little, but it looked to me like chacha20-poly1305@openssh.com might not be viable according to this as it doesn't provide a setupInlineCrypt() method or invoke createInlineCryptFunction(). Something to look into perhaps.

The Eval approach only really provides performance benefits when loop unrolling is being done or table lookups are being eliminated. ChaCha20 is already loop unrolled and by virtue of being a so-called ARX (add–rotate–XOR) cipher it doesn't do table lookups.

Further, the Eval approach adds a layer of complexity and unless the performance gains merit it adding extra layers of complexity is not something I'm too gung ho enthusiastic about doing.

Anyway, I'm code reviewing your changes but it's liable to take some time due to the nature of your changes.

Thanks!

@rposky
Copy link
Contributor Author

rposky commented May 18, 2024

Ah, so is the intent to avoid ChaCha20 when using the "Eval" engine then? Should the code be this then (change to "==" over "!=")?

case 'chacha20-poly1305@openssh.com':
case 'arcfour128':
case 'arcfour256':
if ($engine == 'Eval') {
    continue 2;
}
break;

@terrafrost
Copy link
Member

Good catch! I'll make that change myself as it's prob gonna need to be in the 1.0 branch as well.

Copy link
Member

@terrafrost terrafrost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So overall I like what you've done! Looks like you've quite streamlined stuff. I haven't tested it yet and I haven't looked at the unit tests yet so I still have work to do, but based n what I have looked at I'm liking what I'm seeing.

That said, in reviewing this I see that you're having to deal with a number of areas in phpseclib that I see now are problematic.

Like the fix I did for #1796 couldn't have worked. A lot of the discussion for that ticket took place over email. I put my proposed code change in a branch, asked the guy if he could test it, he never did and so I just merged it without it ever having been tested.

Truth be told idk if even the newest version of that code would work for that guys specific use case but the old version 110% wouldn't have worked.

Also, in reviewing your SSH2::login_helper() changes... without your changes $this->retry_connect kinda has a dual purpose. It's being used for the #1171 fix and for the #1970 fix, which, upon re-examining the code, seems suboptimal.

I've made code changes to address both of these issues but they've added merge conflicts with your code.

As for the ChaCha20 / RC4 issue... upon further consideration the best fix for that isn't to do $engine == 'Eval' but rather to do $engine != 'PHP'. ie. all engines other than the PHP engine should be excluded, as the newly added comment elaborates upon.

In the 1.0 / 2.0 branches there isn't an internal / PHP engine. I mean there are two implementations in the code but you can't select between them whereas with 3.0 you can. So in 1.0 / 2.0 I was doing $engine != Base::ENGINE_INTERNAL and I guess when I was making the updates for 3.0 I assumed that that was equiv to $engine != 'Eval' when in fact what it's equiv to is $engine != 'PHP'.

if ($this->retry_connect) {
$this->retry_connect = false;
$this->connect();
return $this->login_helper($username, $password);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you're moving the check from get_binary_packet() to login_helper(). That's fine but if using a different key size fixes the issue then why aren't you trying to log back in by calling $this->login_helper() again? Like after login_helper() is called a login attempt should have been made. More might be needed but, as is, if a bad algorithm is encountered, the connection will be re-established and no login attempt will have been made?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your concern is addressed, though correct me if I misunderstand your point. In the proposed change, reconnect() is called from here instead, which does perform both the connect() and login() functions.

}

$adjustLength = false;
$extractPacketLength = function ($payload) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just make this a static function? Then instead of calling $extractPacketLength(...); you'd call self::extractPacketLength(...) or whatever.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This anonymous function was a sore point for me as well. I was trying to solve for the repeated type annotations when using extract() into packet_length. It's such a trivial function, I don't think it's worth expanding the method space for though. I think I will just declare packet_length and annotate its type at the head of this switch, as extract will overwrite existing symbols, by default.

@terrafrost
Copy link
Member

I posted my preliminary findings. More week is needed on my part but I prob won't be able to get around to that until next weekend. I got other stuff I need to do!

@rposky
Copy link
Contributor Author

rposky commented May 20, 2024

Like the fix I did for #1796 couldn't have worked. A lot of the discussion for that ticket took place over email. I put my proposed code change in a branch, asked the guy if he could test it, he never did and so I just merged it without it ever having been tested.

I'm only touching SFTP to align to the changes made to reset_connection/disconnect_helper. There's no functional change to init_sftp_connection() and the canonicalize_paths logic. I'll leave it to you to rip that out or do otherwise with it.

Also, in reviewing your SSH2::login_helper() changes... without your changes $this->retry_connect kinda has a dual purpose. It's being used for the #1171 fix and for the #1970 fix, which, upon re-examining the code, seems suboptimal.

Your changes in this area make good sense to me. I had implemented similarly w.r.t. retry_connect, though I didn't go so far as to rename it, which I like login_credentials_finalized for. I'll merge it in.

rposky added 2 commits May 20, 2024 10:35
…e, remove anonymous function. Do not initialize packet->padding_length until payload size fully extracted and validated.
@terrafrost
Copy link
Member

The changes look good to me!

Since it doesn't look like you changed the behavior of setTimeout() (unless I'm mistaken?) would you be up for putting this in the 3.0 branch and I can merge that into master?

Thanks!

@rposky
Copy link
Contributor Author

rposky commented May 26, 2024

Excellent. Yes, I can commit changes to the 3.0 branch. Would you recommend a merge or rebase? And no, there are no intended behavior changes to setTimeout.

@terrafrost
Copy link
Member

Rebase.

Thanks!

@rposky rposky closed this May 28, 2024
terrafrost added a commit to phpseclib/phpseclib.github.io that referenced this pull request Oct 30, 2024
phpseclib/phpseclib#1999 (3.0.38; June 17, 2024)
made it so sleep commands can timeout but also made it so that
$ssh->reset() is now needed after a timed out $ssh->exec() call
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants