BB performance: fingerprinters #1368
Description
Is your feature request related to a problem? Please describe.
Fingerprinting and port scanning takes >30s of time per MACHINE. That means that in cases where BB test contains 2 machines, total time of port scanning and fingerprinting takes >60s.
Fingerprinting log from a personal machine:
2021-07-26 15:13:41,441 [13512:22672:INFO] monkey.start.175: Trying to get OS fingerprint from VictimHost('192.168.10.132') with module ElasticFinger
2021-07-26 15:13:41,444 [13512:22672:DEBUG] connectionpool._new_conn.230: Starting new HTTP connection (1): 192.168.10.132:9200
2021-07-26 15:13:43,517 [13512:22672:DEBUG] elasticfinger.get_host_fingerprint.46: Unknown connection error
2021-07-26 15:13:43,518 [13512:22672:INFO] monkey.start.175: Trying to get OS fingerprint from VictimHost('192.168.10.132') with module HTTPFinger
2021-07-26 15:13:43,523 [13512:22672:DEBUG] connectionpool._new_conn.943: Starting new HTTPS connection (1): 192.168.10.132:80
2021-07-26 15:13:44,529 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from https://192.168.10.132:80
2021-07-26 15:13:44,534 [13512:22672:DEBUG] connectionpool._new_conn.230: Starting new HTTP connection (1): 192.168.10.132:80
2021-07-26 15:13:45,538 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from http://192.168.10.132:80
2021-07-26 15:13:45,541 [13512:22672:DEBUG] connectionpool._new_conn.943: Starting new HTTPS connection (1): 192.168.10.132:8080
2021-07-26 15:13:46,544 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from https://192.168.10.132:8080
2021-07-26 15:13:46,546 [13512:22672:DEBUG] connectionpool._new_conn.230: Starting new HTTP connection (1): 192.168.10.132:8080
2021-07-26 15:13:47,549 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from http://192.168.10.132:8080
2021-07-26 15:13:47,554 [13512:22672:DEBUG] connectionpool._new_conn.943: Starting new HTTPS connection (1): 192.168.10.132:443
2021-07-26 15:13:48,556 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from https://192.168.10.132:443
2021-07-26 15:13:48,561 [13512:22672:DEBUG] connectionpool._new_conn.230: Starting new HTTP connection (1): 192.168.10.132:443
2021-07-26 15:13:49,566 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from http://192.168.10.132:443
2021-07-26 15:13:49,571 [13512:22672:DEBUG] connectionpool._new_conn.943: Starting new HTTPS connection (1): 192.168.10.132:8008
2021-07-26 15:13:50,573 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from https://192.168.10.132:8008
2021-07-26 15:13:50,576 [13512:22672:DEBUG] connectionpool._new_conn.230: Starting new HTTP connection (1): 192.168.10.132:8008
2021-07-26 15:13:51,577 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from http://192.168.10.132:8008
2021-07-26 15:13:51,580 [13512:22672:DEBUG] connectionpool._new_conn.943: Starting new HTTPS connection (1): 192.168.10.132:7001
2021-07-26 15:13:52,585 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from https://192.168.10.132:7001
2021-07-26 15:13:52,590 [13512:22672:DEBUG] connectionpool._new_conn.230: Starting new HTTP connection (1): 192.168.10.132:7001
2021-07-26 15:13:53,593 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from http://192.168.10.132:7001
2021-07-26 15:13:53,599 [13512:22672:DEBUG] connectionpool._new_conn.943: Starting new HTTPS connection (1): 192.168.10.132:9200
2021-07-26 15:13:54,601 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from https://192.168.10.132:9200
2021-07-26 15:13:54,607 [13512:22672:DEBUG] connectionpool._new_conn.230: Starting new HTTP connection (1): 192.168.10.132:9200
2021-07-26 15:13:55,609 [13512:22672:DEBUG] httpfinger.get_host_fingerprint.47: Timout while requesting headers from http://192.168.10.132:9200
2021-07-26 15:13:55,609 [13512:22672:INFO] monkey.start.175: Trying to get OS fingerprint from VictimHost('192.168.10.132') with module MSSQLFinger
2021-07-26 15:13:55,610 [13512:22672:INFO] mssql_fingerprint.get_host_fingerprint.47: Sending message to requested host: Victim Host 192.168.10.132: OS - [] Services - [tcp-22-{'display_name': 'unknown(TCP)', 'port': 22, 'banner': 'SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\
\
'} ] ICMP: False target monkey: None, b'\\x03'
2021-07-26 15:14:00,611 [13512:22672:INFO] mssql_fingerprint.get_host_fingerprint.53: Socket timeout reached, maybe browser service on host: Victim Host 192.168.10.132: OS - [] Services - [tcp-22-{'display_name': 'unknown(TCP)', 'port': 22, 'banner': 'SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\
\
'} ] ICMP: False target monkey: None doesnt exist
2021-07-26 15:14:00,611 [13512:22672:INFO] monkey.start.175: Trying to get OS fingerprint from VictimHost('192.168.10.132') with module MySQLFinger
2021-07-26 15:14:01,113 [13512:22672:DEBUG] mysqlfinger.get_host_fingerprint.73: Error getting mysql fingerprint: timed out
2021-07-26 15:14:01,113 [13512:22672:INFO] monkey.start.175: Trying to get OS fingerprint from VictimHost('192.168.10.132') with module PingScanner
2021-07-26 15:14:01,124 [13512:22672:INFO] monkey.start.175: Trying to get OS fingerprint from VictimHost('192.168.10.132') with module SMBFinger
2021-07-26 15:14:01,827 [13512:22672:DEBUG] smbfinger.get_host_fingerprint.188: Error getting smb fingerprint: timed out
2021-07-26 15:14:01,828 [13512:22672:INFO] monkey.start.175: Trying to get OS fingerprint from VictimHost('192.168.10.132') with module SSHFinger
2021-07-26 15:14:01,828 [13512:22672:DEBUG] base_telem._log_telem_sending.47: Sending scan telemetry.
2021-07-26 15:14:01,829 [13512:22672:DEBUG] base_telem._log_telem_sending.49: Telemetry contents: {"machine": {"ip_addr": "192.168.10.132", "domain_name": "", "os": {"type": "linux", "version": "Ubuntu-4ubuntu0.2"}, "services": {"tcp-22": {"display_name": "SSH", "port": 22, "banner": "SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\
\
", "name": "ssh"}}, "icmp": true, "monkey_exe": null, "default_tunnel...
Describe the solution you'd like
We should only enable fingerprinters/tpc/http ports relevant to the test in config templates.
Tasks
- Review configurations for each test and reduce the fingerprinter configuration to the minimum required set of ports/scanners/etc. (0.75d) @ilija-lazoroski