Skip to content

Lazy-load some submodules inside ncclient.transport only when needed #638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 3, 2025
Merged

Lazy-load some submodules inside ncclient.transport only when needed #638

merged 1 commit into from
Aug 3, 2025

Conversation

focksor
Copy link
Contributor

@focksor focksor commented Jun 24, 2025

This PR is aim to reduce some unnecessary memory usage during the import of ncclient.transport by lazy-loading some submodules only when they are needed.

In file ncclient/transport/__init__.py, the following submodules are imported by default:

from ncclient.transport.ssh import SSHSession
from ncclient.transport.tls import TLSSession
from ncclient.transport.unixSocket import UnixSocketSession  # if possible

These package cause a ton of unnecessary memory usage when ncclient.transport is imported, especially when the user does not use these transport methods.

For example, I only use UnixSocketSession in my scenario, but when I import ncclient.transport, it also imports SSHSession and TLSSession, which I never use. This is a waste of memory.

Here is the memory usage before the change:

$ python
Python 3.10.12 (main, May 27 2025, 17:12:29) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> with open('/proc/self/status') as f: print(''.join(line for line in f if line.startswith('VmRSS')))
...
VmRSS:     11136 kB

>>> from ncclient import manager
>>>
>>> with open('/proc/self/status') as f: print(''.join(line for line in f if line.startswith('VmRSS')))
...
VmRSS:     32424 kB

>>>

After the change, the memory usage is reduced significantly:

$ PYTHONPATH=. python
Python 3.10.12 (main, May 27 2025, 17:12:29) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> with open('/proc/self/status') as f: print(''.join(line for line in f if line.startswith('VmRSS')))
...
VmRSS:     10880 kB

>>> from ncclient import manager
>>>
>>> with open('/proc/self/status') as f: print(''.join(line for line in f if line.startswith('VmRSS')))
...
VmRSS:     18176 kB

>>>

Since this is my first PR to the ncclient project, I would like to ask for your review and feedback on this change. Any suggestions or improvements are welcome.

@coveralls
Copy link

Coverage Status

coverage: 82.543% (+0.04%) from 82.504%
when pulling 9e68d80 on focksor:master
into 802d6b4 on ncclient:master.

@einarnn einarnn merged commit 52b788b into ncclient:master Aug 3, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@einarnn einarnn einarnn approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@focksor @coveralls @einarnn