[FEATURE] JSON output format #70

HorlogeSkynet · 2020-05-08T13:10:57Z

python3 -m archey --json

Description

Simple JSON-formatting of entries data.
It's a first basic support, maybe it could be enhanced in the future.

EDIT 2020-05-13 : To keep things clean, DRY and concerns-separated, @ingrinder and myself started to work on a slight rework to make entries' values independent from their corresponding (regular and original) text output.
This will drastically simplify future work on entries data.

Reason and / or context

It would allow one to use Archey as an API or so 😄

How has this been tested ?

Test cases and our machines.

Types of changes :

Bug fix (non-breaking change which fixes an issue)
Typo / style fix (non-breaking change which improves readability)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist :

[IF NEEDED] I have updated the test cases (which pass) accordingly ;
My changes looks good ;
I agree that my code may be modified in the future ;
My code follows the code style of this project (PEP8).

ingrinder · 2020-05-08T21:21:40Z

Huh, it's funny the CI build passed, since I'm consistently getting a unit test error on this branch:

======================================================================
ERROR: test_json_output_format (archey.test.test_archey_output.TestOutputUtil)
Test how the `output` method handles JSON preferred formatting of entries
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.8/unittest/mock.py", line 1348, in patched
    return func(*newargs, **newkeywargs)
  File "/home/michael/git/archey4/archey/test/test_archey_output.py", line 504, in test_json_output_format
    output = Output(format_to_json=True)
  File "/home/michael/git/archey4/archey/output.py", line 59, in __init__
    if ansi_color and Configuration().get('colors_palette')['honor_ansi_color']:
KeyError: 'honor_ansi_color'

----------------------------------------------------------------------
Ran 108 tests in 0.075s

FAILED (errors=1)

...whether I use python -m unittest or python setup.py -q test. I haven't got a custom configuration file in /etc/archey4 or ~/.config/archey4...

HorlogeSkynet · 2020-05-09T07:27:36Z

Yes sorry, related to d559edc52d839654989f1b39e6b0d3159bb71497.
It's (again) related to how Configuration is mocked across test modules (and the fact that it is actually a Singleton never getting reset)...
It should be fixed on your machine now 🙇

HorlogeSkynet · 2020-05-12T07:55:41Z

'should be well cleaner now.
I'll even fix a little blemish in our Output.{append,output} flow when #71 will be merged.
Tell me what you think about that !

Bye 👋

> Rationale : This should be compatible against Python < 3.6

> Rationale: When #71 will be merged, issues due to keys duplication may occur. > This patch sets output values as a list anyway, sparing us an API breakage in the future. > Note: A `Colors.remove_colors` utility method has been added for DRY purposes.

+ Sets results output in a `data` dictionary of the parent document + Adds a new `meta` dictionary containing date of generation and Archey version > Rationale: To keep `Output` concise, API-related code has been moved to a new `API` module. > Rationale 2: Adding a timestamp would allow developers to perform computations based on collection date. > Adding a version identifier would allow **us** to make the API evolve in the future.

Adds proper JSON formatting to the following entries when using the `-j` option: * Disk * LanIp * RAM * Temperature * Terminal * Uptime * WanIp Additionally adds a new attribute `_format_to_json` to the `Entry` base class.

Rationale: Having the raw data in a consistent place makes it far easier to implement an API with our entries. * Removed `_format_to_json` attribute introduced in e709d0e. * Modified all entries to place a generic object (list, string, dict) into the `value` attribute which suits the data best. * Moved pretty-formatting logic to an overriden `output` method. * Tweaked API implementation to use individual entries rather than their outputs (we now deal with the `value` attribute in the API which should contain all of an entry's data, even if it outputs multiple lines). * Modified tests as appropriate :)

ingrinder · 2020-05-13T02:49:57Z

Sorry for the delays, I've been a little busy 😃. I made a few more modifications since I wasn't fully satisfied after I started work on e709d0e until seeing 32cdee5 inspired me to change how we treat Entry.value. Anyway, I'll try and get back to you with a more detailed response tomorrow if I get the chance 👍.

Accidentally removed in 2188def. Sidenote: We should probably add a test for `output` on entries now.

... and fixes minor code formats

... and fixes other minor code formats

HorlogeSkynet · 2020-05-13T09:50:51Z

Here we are ! Thanks for your huge work on improving the (raw) value/text output integration.
I think it would allow more modifiability in the future, currently costing us time though, sorry for that.

Changelog for my previous changes :

Test cases fixes
New output test cases for almost all entries
Some optimizations in entries flows/logic
max_temperature setting in entry's data (by using the API, setting the value unconditionally would be wise)
API meta-data and test enhancements

Once this has been merged, I'll rebase #71 & #72, leaving you #67 😉

Anyway, there are eventually many changes, so a proper review is needed 😨

HorlogeSkynet · 2020-05-17T09:05:28Z

Unfortunately a rebase against new master was a PITA, so I've simply merged it to resolve new conflicts.
Due to past PRs merging order, I've had to roll-back chunks of #71 not required anymore according to global rework/cleanup done here.

Anyway, a proper review is more than required, but future looks simpler once this patch will be merged 👍

ingrinder

Thanks for all the patches on top of the changes I made earlier! This is looking nice now 😃. I've left a few comments for stuff I'm not so sure about or could do with changing - I'll hopefully submit commits for some of them if I get the chance. 👍

archey/__main__.py

archey/output.py

archey/colors.py

archey/entries/ram.py

archey/entries/temperature.py

archey/entries/uptime.py

archey/test/test_archey_api.py

archey/test/test_archey_colors.py

archey/test/test_archey_disk.py

archey/test/test_archey_gpu.py

> `archey -j` --> Compact JSON output > `archey -jj` --> 2-tab indented output > `archey -jj --json` --> 4-tab indented output > `archey -jjjj --> ...

ingrinder

Nice, liking the custom assertions implementation 👍. This LGTM!

Edit: I'll refrain from merging anything and let you have the final go-ahead since last time I assumed we were going to merge the other way around 😄.

HorlogeSkynet added the enhancement ⬆️ Implements a new feature, fixes or improves existing ones label May 8, 2020

HorlogeSkynet added this to the v4.7.2 milestone May 8, 2020

HorlogeSkynet requested a review from ingrinder May 8, 2020 13:10

HorlogeSkynet self-assigned this May 8, 2020

Samuel FORESTIER added 6 commits May 12, 2020 13:23

[OUTPUT] Implements a -j option to make Archey format data to JSON

0373a90

[OUTPUT] [TEST] Relaxes JSON formatting test cases for Python < 3.6

14eb9cf

[OUTPUT] [TEST] Hardens JSON formatting test case

a34f2e3

> Rationale : This should be compatible against Python < 3.6

[OUTPUT] [TEST] Fixes test on distributions packaging an ANSI_COLOR

f0373fa

HorlogeSkynet force-pushed the feature/json_format branch from 57e60dd to 32cdee5 Compare May 12, 2020 11:23

ingrinder added 2 commits May 12, 2020 22:30

[ENTRIES] Add JSON formatted output to entries

e709d0e

Adds proper JSON formatting to the following entries when using the `-j` option: * Disk * LanIp * RAM * Temperature * Terminal * Uptime * WanIp Additionally adds a new attribute `_format_to_json` to the `Entry` base class.

ingrinder and others added 13 commits May 13, 2020 03:58

[TEMPERATURE] Re-add fallback if none detected.

5b9efa5

Accidentally removed in 2188def. Sidenote: We should probably add a test for `output` on entries now.

[OUTPUT] Sets format_to_json as internal private attribute

bdfaf07

[DOC/TEST] Fixes little blemishes in test cases and documentation

6253f2a

[LAN_IP] Simplifies internal logic and publishes output test cases

6ce8fe9

[UPTIME] Fixes output test cases and improves documentation

d509999

[TERMINAL] Fixes test cases

0b2d118

[DISTRO] Reworks the entry to improve consistency with our new flow

fa89c06

[DISK] Simplifies internals and output method, and fixes test cases

29120ef

[LAN_IP] Don't store unnecessary references to append args

7c0634c

[API] [TEST] Enforces some more verifications during testing

b490fbe

[TEMPERATURE] Sets max_temperature in raw data + Adds output tests

2687683

[GPU/MODEL/PACKAGES/DE/SHELL] Simplifies internal logic with new flow

0e19d76

... and fixes minor code formats

[RAM] Fixes failing test cases

114a0d7

Samuel FORESTIER added 2 commits May 13, 2020 11:29

[DE/WM/USER/WAN_IP] Simplifies internal logic with new flow

bcd8260

... and fixes other minor code formats

[API] [TEST] Fixes test cases for Python prior to v3.7

3a37544

[API] Adds a count meta-data attribute with the number of entries

da2cd78

HorlogeSkynet mentioned this pull request May 17, 2020

[GPU] Adds support for multiple GPUs (and fixes their display order) #71

Merged

8 tasks

Merge branch 'master' into feature/json_format

cd8c663

ingrinder requested changes May 17, 2020

View reviewed changes

Samuel FORESTIER added 6 commits May 18, 2020 09:01

[UPTIME] Adds seconds precision to API data (and improves testing)

95ee56d

[API] [TEST] Relaxes date test case for systems with low time accuracy

70e1240

[DISK] [TEST] Adds a test case for the danger color level

84748e0

[TEST] Implements an assertListEmpty for entries with multiple values

09b8c79

[COLORS] [TEST] Adds a case showing non-color escape codes are preserved

7e6a389

[API] Allows the user to enable pretty-printing (with only one argument)

9993808

> `archey -j` --> Compact JSON output > `archey -jj` --> 2-tab indented output > `archey -jj --json` --> 4-tab indented output > `archey -jjjj --> ...

HorlogeSkynet requested a review from ingrinder May 18, 2020 08:16

Samuel FORESTIER and others added 3 commits May 18, 2020 22:30

[TEST] Adds a quite important missing test case for assertListEmpty

0e11872

[API] [JSON] Improves help message about -j option

c8fe495

[COLORS/TESTS] Small documentation clarifications

af177fd

ingrinder approved these changes May 18, 2020

View reviewed changes

Samuel FORESTIER added 2 commits May 19, 2020 09:43

[API] [JSON] Unties Output and API back-ends + Adds indentation test

4a33def

[STYLES] Removes unnecessary , after last element in lists

05bea33

HorlogeSkynet merged commit f1759a9 into master May 19, 2020

HorlogeSkynet deleted the feature/json_format branch May 19, 2020 07:53

ingrinder mentioned this pull request May 20, 2020

[WIP] Disk entry rework #67

Merged

9 tasks

HorlogeSkynet mentioned this pull request Dec 16, 2020

[KERNEL] Implement kernel release comparison against <www.kernel.org> #94

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[FEATURE] JSON output format #70

[FEATURE] JSON output format #70

Uh oh!

HorlogeSkynet commented May 8, 2020 •

edited

Loading

Uh oh!

ingrinder commented May 8, 2020

Uh oh!

HorlogeSkynet commented May 9, 2020

Uh oh!

HorlogeSkynet commented May 12, 2020

Uh oh!

ingrinder commented May 13, 2020

Uh oh!

HorlogeSkynet commented May 13, 2020 •

edited

Loading

Uh oh!

HorlogeSkynet commented May 17, 2020 •

edited

Loading

Uh oh!

ingrinder left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ingrinder left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

[FEATURE] JSON output format #70

[FEATURE] JSON output format #70

Uh oh!

Conversation

HorlogeSkynet commented May 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Reason and / or context

How has this been tested ?

Types of changes :

Checklist :

Uh oh!

ingrinder commented May 8, 2020

Uh oh!

HorlogeSkynet commented May 9, 2020

Uh oh!

HorlogeSkynet commented May 12, 2020

Uh oh!

ingrinder commented May 13, 2020

Uh oh!

HorlogeSkynet commented May 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HorlogeSkynet commented May 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ingrinder left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ingrinder left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

HorlogeSkynet commented May 8, 2020 •

edited

Loading

HorlogeSkynet commented May 13, 2020 •

edited

Loading

HorlogeSkynet commented May 17, 2020 •

edited

Loading

ingrinder left a comment •

edited

Loading