Unexpected errors in AwsInstance and AzureInstance #902
Description
I'm getting the following errors during environment collection...
... in AwsInstance:
File "/home/shreya/Desktop/Projects/monkey/monkey/infection_monkey/system_info/collectors/environment_collector.py", line 1, in <module>
from common.cloud.all_instances import get_all_cloud_instances
File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/all_instances.py", line 8, in <module>
all_cloud_instances = [AwsInstance(), AzureInstance(), GcpInstance()]
File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/aws/aws_instance.py", line 49, in __init__
AWS_LATEST_METADATA_URI_PREFIX + 'dynamic/instance-identity/document', timeout=2).read().decode())
File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/aws/aws_instance.py", line 81, in _extract_account_id
return json.loads(instance_identity_document_response)[ACCOUNT_ID_KEY]
File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
which arises when the account ID is being extracted in _extract_account_id() because
urllib.request.urlopen(
AWS_LATEST_METADATA_URI_PREFIX + 'dynamic/instance-identity/document', timeout=2).read().decode())returns:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
<meta content="no-cache" http-equiv="Pragma" />
<title>Waiting...</title>
<script type="text/javascript">
var pageName = '/';
top.location.replace(pageName);
</script>
</head>
<body> </body>
</html>
... and in AzureInstance:
File "/home/shreya/Desktop/Projects/monkey/monkey/infection_monkey/system_info/collectors/environment_collector.py", line 1, in <module>
from common.cloud.all_instances import get_all_cloud_instances
File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/all_instances.py", line 8, in <module>
all_cloud_instances = [AwsInstance(), AzureInstance(), GcpInstance()]
File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/azure/azure_instance.py", line 45, in __init__
self.try_parse_response(response)
File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/azure/azure_instance.py", line 54, in try_parse_response
response_data = response.json()
File "/usr/local/lib/python3.7/dist-packages/requests/models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/__init__.py", line 525, in loads
return _default_decoder.decode(s)
File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
which arises when the response is parsed here because
response = requests.get(AZURE_METADATA_SERVICE_URL,
headers={"Metadata": "true"},
timeout=SHORT_REQUEST_TIMEOUT)returns:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
<meta content="no-cache" http-equiv="Pragma" />
<title>Waiting...</title>
<script type="text/javascript">
var pageName = '/';
top.location.replace(pageName);
</script>
</head>
<body> </body>
</html>
I'm not sure why this suddenly came up but we should probably catch the exceptions.