Skip to content

Typical cve-search issues #187

@m-1-k-3

Description

@m-1-k-3

Current state

If you are running into cve-search issues please update your EMBA installation:

sudo service mongod stop
sudo /etc/init.d/redis-server stop
sudo /etc/init.d/docker restart         
sudo ifconfig emba_runs down                                                                                                                                                                                        
git clone https://github.com/e-m-b-a/emba.git new_emba
cd new_emba
sudo ./installer.sh -d

After testing you should be able to remove your old EMBA installation and the installed mongodb.

The following information is outdated:

Local network issues

With the latest updates (see #183) we have isolated the EMBA container from a networking perspective. This means EMBA is not able to reach the internet anymore. With this also some adjustments in the EMBA docker container and in the local mongodb configuration were needed.

The easiest way to get an updated EMBA installation are the following steps:

cd emba-directory
git pull
sudo docker pull embeddedanalyzer/emba
sudo ./installer.sh -d

If you are running into issues that the EMBA container is not able to access your mongodb installation you can check the following:

  • The command ifconfig should show a new network interface with the following ip address:
└─$ ifconfig emba_runs
emba_runs: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.36.0.1  netmask 255.255.0.0  broadcast 172.36.255.255
        ether 02:42:cb:c9:37:80  txqueuelen 0  (Ethernet)
        RX packets 15643  bytes 964808 (942.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 43695  bytes 1435782652 (1.3 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

If this is not the case you can try to restart docker at all with the following command
└─$ sudo /etc/init.d/docker restart
Another way is to delete the network interface and re-initialize it:

└─$ sudo docker network rm emba_runs
└─$ sudo docker-compose up --no-start

Additionally, you can try the following command

└─$ sudo systemctl restart NetworkManager docker
  • Your mongod.conf should have the following bindIp configuration:
└─$ grep bindIp /etc/mongod.conf
  bindIp: 172.36.0.1

Mongod is not starting up

EMBA is complaining that CVE search works not as expected. Checking if mongod is listening, does not return an open port:

└─$ sudo netstat -anpt | grep 27017 

Also a restart of mongod does not help!

Please check if your harddisk was running out of space (e.g. with df -h). If it is full, clean it up and retry it.

Mongod is not starting up - part 2

Check your mongodb log /var/log/mongodb/mongod.log for errors. One of the seen errors is the following:

{"t":{"$date":"2023-03-24T15:40:07.026+08:00"},"s":"E", "c":"STORAGE", "id":20557, "ctx":"initandlisten","msg":"DBException in initAndListen, terminating","attr":{"error":"IllegalOperation: Attempted to create a lock file on a read-only directory: /var/lib/mongodb"}}

Check this solution and adjust the permissions.

No results from CVE-search

Probably you are running into the issue that your network environment is working as expected and mongod is also listening on port 27017. If you are trying to query your CVE database you get no results:

image

Sometimes it happens that the CVE database is corrupt or not fully populated. So, please do a full reinitialization and update the following way:

└─$ source ./external/emba_venv/bin/activate
└─$ ./external/cve-search/sbin/db_updater.py -f

As from November 2023 cve-search switched to the new NIST API. From there on you can also use the following command:

└─$ cvexplore database initialize

Errors during database update

During a manual CVE database update you are running into the following errors:

image

It looks like this or this issue.

"Did not receive last-modified header in the response" Error during database update

During a manual CVE database update you are running into the following errors:

CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html', 'connection': 'close'}

Error is gone by adjusting max_workers to 1 in https://github.com/cve-search/cve-search/blob/1f0b50aa46814e2a683a9b7b01da8bcc0403154e/lib/DownloadHandler.py#L121 like this

thread_map(self.download_site, sites, desc="Downloading files", max_workers=1)

Afterwards ./external/cve-search/sbin/db_updater.py -f should work fine (takes longer though).

Seems like there is a rate limit with https://nvd.nist.gov/
Source: cve-search/cve-search#890 (comment)

thx for documenting this issue to @brainsht

Python dependency warnings

If you get the following warnings you need to update the requests package.

/usr/local/lib/python3.10/dist-packages/requests/__init__.py:102: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (5.0.0)/charset_normalizer (2.0.12) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "

Update the requests package the following way should solve this issue:

└─$ source ./external/emba_venv/bin/activate
└─$ sudo pip install -U requests                               

Testing CVE-search manually

  • Finally you can test it manually via starting the EMBA container and test the cve-search connection the following way:
└─$ sudo docker-compose run emba     
WARNING: The FIRMWARE variable is not set. Defaulting to a blank string.
WARNING: The LOG variable is not set. Defaulting to a blank string.
WARNING: The EMBA variable is not set. Defaulting to a blank string.
Creating emba_forked_emba_run ... done

┌──(root@d2a5960b73b2)-[/emba]
└─# source /external/emba_venv/bin/activate
┌──(root@d2a5960b73b2)-[/emba]
└─# /external/cve-search/bin/search.py -p busybox

This should show some CVE details regarding busybox.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cve-searchSome cve-search question/issuedockerdocker related thingsdocumentationImprovements or additions to documentationenhancementNew feature or requeststale

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions