bug: whatportis db purge failed #984
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
Seems that whatportis didn't purge correctly his db
Here is the log file with error
celery_1 | Previous database will be updated, do you want to continue? [y/N]: Downloading https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.csv...
celery_1 | Populating database...
celery_1 | Traceback (most recent call last):
celery_1 | File "/usr/local/bin/whatportis", line 8, in <module>
celery_1 | sys.exit(run())
celery_1 | File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
celery_1 | return self.main(*args, **kwargs)
celery_1 | File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1077, in main
celery_1 | with self.make_context(prog_name, args, **extra) as ctx:
celery_1 | File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 943, in make_context
celery_1 | self.parse_args(ctx, args)
celery_1 | File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1408, in parse_args
celery_1 | value, args = param.handle_parse_result(ctx, opts, args)
celery_1 | File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 2400, in handle_parse_result
celery_1 | value = self.process_value(ctx, value)
celery_1 | File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 2362, in process_value
celery_1 | value = self.callback(ctx, self, value)
celery_1 | File "/usr/local/lib/python3.10/dist-packages/whatportis/cli.py", line 83, in update_db
celery_1 | populate_db()
celery_1 | File "/usr/local/lib/python3.10/dist-packages/whatportis/cli.py", line 63, in populate_db
celery_1 | db.purge()
celery_1 | File "/usr/local/lib/python3.10/dist-packages/tinydb/database.py", line 255, in __getattr__
celery_1 | return getattr(self.table(self.default_table_name), name)
celery_1 | AttributeError: 'Table' object has no attribute 'purge'
ce
Expected Behavior
No error and whatportis db should be truncated
Steps To Reproduce
launch celery container
Environment
- reNgine: 2.0
- OS: Debian 11
- Python: 3.10
- Docker Engine: 24.0.1
- Docker Compose: 1.29.2
- Browser: FF 118Anything else?
Problem comes from TinyDb
According to this changelog, It seems the table.purge has been renamed table.truncate in v4.0.0
Whatportis must be updated to 0.8.2, which contains db.truncate instead of db.purge
https://github.com/ncrocfer/whatportis/blob/59a1718bf7c531f2a5a4e213cad0c047ce9c1c94/whatportis/cli.py#L63C18-L63C18
Anyway after further investigation, it seems whatportis last version is really too old (2 years) and if you upgrade it with the following pip command pip install --upgrade whatportis it downgrades a lot of python libs needed by celery, so It could bring a lot of others problems.
I think you should forget whatportis usage, or pip upgrade whatportis without downgrade dependency or only do a sed to change purge by truncate
I've tested with sed and it works fine
sed -i 's/purge()/truncate()/g' /usr/local/lib/python3.10/dist-packages/whatportis/cli.py
But it's not elegant, it's a quick and dirty fix for python 3.10 users only
Here the full pip upgrade log of whatportis inside the celery container
Requirement already satisfied: whatportis in /usr/local/lib/python3.10/dist-packages (0.8)
Collecting whatportis
Using cached whatportis-0.8.2-py3-none-any.whl (7.5 kB)
Collecting simplejson==3.17.2 (from whatportis)
Using cached simplejson-3.17.2.tar.gz (83 kB)
Preparing metadata (setup.py) ... done
Collecting tinydb==4.4.0 (from whatportis)
Using cached tinydb-4.4.0-py3-none-any.whl (21 kB)
Collecting requests==2.25.1 (from whatportis)
Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 1.9 MB/s eta 0:00:00
Collecting prettytable==2.1.0 (from whatportis)
Downloading prettytable-2.1.0-py3-none-any.whl (22 kB)
Collecting click==8.0.1 (from whatportis)
Downloading click-8.0.1-py3-none-any.whl (97 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.4/97.4 kB 5.0 MB/s eta 0:00:00
Requirement already satisfied: wcwidth in /usr/local/lib/python3.10/dist-packages (from prettytable==2.1.0->whatportis) (0.2.8)
Collecting chardet<5,>=3.0.2 (from requests==2.25.1->whatportis)
Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.7/178.7 kB 10.7 MB/s eta 0:00:00
Collecting idna<3,>=2.5 (from requests==2.25.1->whatportis)
Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.8/58.8 kB 28.4 MB/s eta 0:00:00
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests==2.25.1->whatportis) (1.26.9)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests==2.25.1->whatportis) (2023.7.22)
Building wheels for collected packages: simplejson
Building wheel for simplejson (setup.py) ... done
Created wheel for simplejson: filename=simplejson-3.17.2-cp310-cp310-linux_x86_64.whl size=123089 sha256=48c0dc5dbe3391db2e6b04be735b002e4ea07c5ab377d99ab3f125544cd0d22b
Stored in directory: /root/.cache/pip/wheels/4c/d2/65/e6ec94ca89c44483d8010d15e5fbead72b6411e01d26f90a31
Successfully built simplejson
Installing collected packages: tinydb, simplejson, prettytable, idna, click, chardet, requests, whatportis
Attempting uninstall: tinydb
Found existing installation: tinydb 4.8.0
Uninstalling tinydb-4.8.0:
Successfully uninstalled tinydb-4.8.0
Attempting uninstall: simplejson
Found existing installation: simplejson 3.19.2
Uninstalling simplejson-3.19.2:
Successfully uninstalled simplejson-3.19.2
Attempting uninstall: prettytable
Found existing installation: prettytable 3.9.0
Uninstalling prettytable-3.9.0:
Successfully uninstalled prettytable-3.9.0
Attempting uninstall: idna
Found existing installation: idna 3.3
Uninstalling idna-3.3:
Successfully uninstalled idna-3.3
Attempting uninstall: click
Found existing installation: click 8.1.7
Uninstalling click-8.1.7:
Successfully uninstalled click-8.1.7
Attempting uninstall: chardet
Found existing installation: chardet 5.0.0
Uninstalling chardet-5.0.0:
Successfully uninstalled chardet-5.0.0
Attempting uninstall: requests
Found existing installation: requests 2.31.0
Uninstalling requests-2.31.0:
Successfully uninstalled requests-2.31.0
Attempting uninstall: whatportis
Found existing installation: whatportis 0.8
Uninstalling whatportis-0.8:
Successfully uninstalled whatportis-0.8
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
celery 5.3.4 requires click<9.0,>=8.1.2, but you have click 8.0.1 which is incompatible.
censys 2.2.7 requires requests>=2.29.0, but you have requests 2.25.1 which is incompatible.
discord-webhook 1.3.0 requires requests<3.0.0,>=2.28.1, but you have requests 2.25.1 which is incompatible.
Successfully installed chardet-4.0.0 click-8.0.1 idna-2.10 prettytable-2.1.0 requests-2.25.1 simplejson-3.17.2 tinydb-4.4.0 whatportis-0.8.2