dist/pythonlibs: provide unittest TestCase wrapper for testrunner #10431

miri64 · 2018-11-19T16:12:05Z

Contribution description

I had this idea when implementing #10382 and #10392 as I introduced a very similar structure to python's standard unittests in those and it could also reduce some code duplication between those two tests.

Regarding ongoing discussions about test frameworks

As far as I understand pytest so far it would also be beneficial for porting our current tests with this approach. I don't know how this looks like for RobotFramework.

Testing procedure

Currently no tests are ported and this is just an RFC. If the discussion steers into the direction of approval (or indecisiveness ;-)) I'll try to port one or two tests to this approach.

Issues/PRs references

#10382 and #10392 could greatly benefit from it, discussions after #10241 (comment) could be related.

miri64 · 2018-11-19T16:16:20Z

edit added quote for reference

[…] as I introduced a very similar structure to python's standard unittests in those

e.g. this whole section in #10392 could be methods of a PexpectTestCase class that also contains all the interaction stuff (like pktbuf_empty(), add_neighbor(), del_neighbor(), etc.) with the node as methods:

RIOT/tests/gnrc_rpl_srh/tests/01-run.py

Lines 166 to 310 in dc7012b

    
           def test_wrong_type(child, iface, hw_dst, ll_dst, ll_src): 
        
               p = srp1(Ether(dst=hw_dst) / IPv6(dst=ll_dst, src=ll_src) / 
        
                        IPv6ExtHdrRouting(type=255, segleft=1, addresses=["abcd::1"]), 
        
                        iface=iface, timeout=1, verbose=0) 
        
               assert(p is not None) 
        
               assert(ICMPv6ParamProblem in p) 
        
               assert(p[ICMPv6ParamProblem].code == 0) # erroneous header field encountered 
        
               assert(p[ICMPv6ParamProblem].ptr == 42) # routing header type field 
        
               pktbuf_empty(child) 
        
           def test_seg_left_gt_len_addresses(child, iface, hw_dst, ll_dst, ll_src): 
        
               # send routing header with no (0) addresses but segleft set to a value 
        
               # larger than 0 
        
               p = srp1(Ether(dst=hw_dst) / IPv6(dst=ll_dst, src=ll_src) / 
        
                        IPv6ExtHdrRouting(type=3, segleft=18, addresses=[]), 
        
                        iface=iface, timeout=1, verbose=0) 
        
               assert(p is not None) 
        
               assert(ICMPv6ParamProblem in p) 
        
               assert(p[ICMPv6ParamProblem].code == 0) # erroneous header field encountered 
        
               assert(p[ICMPv6ParamProblem].ptr == 43) # segleft field 
        
               pktbuf_empty(child) 
        
           def test_multicast_dst(child, iface, hw_dst, ll_dst, ll_src): 
        
               # sniffing for ICMPv6 parameter problem message 
        
               sniffer.start_sniff(lambda p: p.haslayer(ICMPv6ParamProblem)) 
        
               # send routing header with multicast destination 
        
               sendp(Ether(dst=hw_dst) / IPv6(dst="ff02::1", src=ll_src) / 
        
                     IPv6ExtHdrRouting(type=3, segleft=1, addresses=["abcd::1"]), 
        
                     iface=iface, verbose=0) 
        
               ps = sniffer.wait_for_sniff_results() 
        
               p = [p for p in ps if ICMPv6ParamProblem in p] 
        
               assert(len(p) > 0) 
        
               p = p[0] 
        
               assert(p[ICMPv6ParamProblem].code == 0) # erroneous header field encountered 
        
               assert(p[ICMPv6ParamProblem].ptr == 24) # IPv6 headers destination field 
        
               pktbuf_empty(child) 
        
           def test_multicast_addr(child, iface, hw_dst, ll_dst, ll_src): 
        
               # Send routing header with multicast address in its destinations 
        
               p = srp1(Ether(dst=hw_dst) / IPv6(dst=ll_dst, src=ll_src) / 
        
                        IPv6ExtHdrRouting(type=3, segleft=1, addresses=["ff02::1"]), 
        
                        iface=iface, timeout=1, verbose=0) 
        
               assert(p is not None) 
        
               assert(ICMPv6ParamProblem in p) 
        
               assert(p[ICMPv6ParamProblem].code == 0) # erroneous header field encountered 
        
               assert(p[ICMPv6ParamProblem].ptr == 48) # first address in routing header 
        
               pktbuf_empty(child) 
        
           def test_multiple_addrs_of_mine_uncomp(child, iface, hw_dst, ll_dst, ll_src): 
        
               dummy = "affe::1" 
        
               # add dummy IPv6 address 
        
               dst_iface = get_first_interface(child) 
        
               add_ipv6_address(child, dst_iface, dummy) 
        
               p = srp1(Ether(dst=hw_dst) / IPv6(dst=ll_dst, src=ll_src) / 
        
                        IPv6ExtHdrRouting(type=3, segleft=3, addresses=[ll_dst, ll_src, 
        
                                                                        dummy]), 
        
                        iface=iface, timeout=1, verbose=0) 
        
               assert(p is not None) 
        
               assert(ICMPv6ParamProblem in p) 
        
               assert(p[ICMPv6ParamProblem].code == 0) # erroneous header field encountered 
        
               assert(p[ICMPv6ParamProblem].ptr == 40+8+(2 * 16)) # dummy in routing header 
        
               pktbuf_empty(child) 
        
               del_ipv6_address(child, dst_iface, dummy) 
        
           def test_forward_uncomp(child, iface, hw_dst, ll_dst, ll_src): 
        
               dummy = "affe::1" 
        
               hl = random.randint(2, 255) 
        
               # sniffing for packets to dummy 
        
               sniffer.start_sniff(lambda p: p[Ether].src == hw_dst) 
        
               # add dummy IPv6 address 
        
               dst_iface = get_first_interface(child) 
        
               hw_src = get_host_hwaddr(iface) 
        
               add_neighbor(child, dst_iface, dummy, hw_src) 
        
               sendp(Ether(dst=hw_dst) / IPv6(dst=ll_dst, src=ll_src, hlim=hl) / 
        
                     IPv6ExtHdrRouting(type=3, segleft=1, addresses=[dummy]), 
        
                     iface=iface, verbose=0) 
        
               ps = sniffer.wait_for_sniff_results() 
        
               p = [p for p in ps if p[Ether].src == hw_dst] 
        
               assert(len(p) > 0) 
        
               p = p[0] 
        
               assert(IPv6 in p) 
        
               assert(IPv6ExtHdrRouting in p) 
        
               assert(p[IPv6].src == ll_src) 
        
               assert(p[IPv6].dst == dummy) 
        
               assert(p[IPv6].hlim == (hl - 1)) 
        
               assert(p[IPv6ExtHdrRouting].type == 3) 
        
               assert(p[IPv6ExtHdrRouting].segleft == 0) 
        
               pktbuf_empty(child) 
        
               del_neighbor(child, dst_iface, dummy) 
        
           def test_forward_uncomp_not_first_ext_hdr(child, iface, hw_dst, ll_dst, ll_src): 
        
               dummy = "affe::1" 
        
               hl = random.randint(2, 255) 
        
               # sniffing for packets to dummy 
        
               sniffer.start_sniff(lambda p: p[Ether].src == hw_dst) 
        
               # add dummy IPv6 address 
        
               dst_iface = get_first_interface(child) 
        
               hw_src = get_host_hwaddr(iface) 
        
               add_neighbor(child, dst_iface, dummy, hw_src) 
        
               sendp(Ether(dst=hw_dst) / IPv6(dst=ll_dst, src=ll_src, hlim=hl) / 
        
                     IPv6ExtHdrHopByHop() / 
        
                     IPv6ExtHdrRouting(type=3, segleft=1, addresses=[dummy]), 
        
                     iface=iface, verbose=0) 
        
               ps = sniffer.wait_for_sniff_results() 
        
               p = [p for p in ps if p[Ether].src == hw_dst] 
        
               assert(len(p) > 0) 
        
               p = p[0] 
        
               assert(IPv6 in p) 
        
               assert(IPv6ExtHdrRouting in p) 
        
               assert(p[IPv6].src == ll_src) 
        
               assert(p[IPv6].dst == dummy) 
        
               assert(p[IPv6].hlim == (hl - 1)) 
        
               assert(p[IPv6ExtHdrRouting].type == 3) 
        
               assert(p[IPv6ExtHdrRouting].segleft == 0) 
        
               pktbuf_empty(child) 
        
               del_neighbor(child, dst_iface, dummy) 
        
           def test_seq_left_0(child, iface, hw_dst, ll_dst, ll_src): 
        
               register_protnum(child) 
        
               sendp(Ether(dst=hw_dst) / IPv6(dst=ll_dst, src=ll_src) / 
        
                     IPv6ExtHdrRouting(type=3, segleft=0), iface=iface, verbose=0) 
        
               # we are the target, so the packet should be dumped 
        
               # empty snip 
        
               child.expect(r"~~ SNIP  0 - size:\s+0 byte, type: NETTYPE_UNDEF \(\d+\)") 
        
               ipv6_payload_len = 0 
        
               # parsed routing header 
        
               child.expect(r"~~ SNIP  1 - size:\s+(\d+) byte, type: NETTYPE_\w+ \(\d+\)") 
        
               ipv6_payload_len += int(child.match.group(1)) 
        
               # NH = 59 (IPV6_NONXT), len = 0x00, routing type = 3, segments left = 0 
        
               child.expect(r"00000000  3B  00  03  00  00  00  00  00") 
        
               # IPv6 header 
        
               child.expect(r"~~ SNIP  2 - size:\s+40 byte, type: NETTYPE_IPV6 \(\d+\)") 
        
               child.expect_exact(r"length: {}  next header: {}".format( 
        
                       ipv6_payload_len, EXT_HDR_NH[IPv6ExtHdrRouting] 
        
                   )) 
        
               child.expect_exact(r"destination address: {}".format(ll_dst)) 
        
               pktbuf_empty(child) 
        
               unregister(child)

(also Codacy is being stressed out about all the assert()s in there ;-)).

kb2ma · 2018-11-19T18:16:41Z

@MrKevinWeiss and @smlng are working on their comparison, so I would not want to assume the outcome. There is no question that the existing testrunner approach can be improved with the additional tools in pytest or Robot.

I agree that it is worthwhile now to try writing tests using these frameworks so we make an informed decision, and even better if it helps with PRs you're working on right now! IMO the ability to easily port and improve existing tests is a good reason to use pytest. Not sure that I would take it farther than that at the moment.

Copying @cladmi. He commented with some code for /tests, so may have some feedback.

miri64 · 2018-11-19T20:14:52Z

Just to be clear I did not create this PR with the intention to sway the discussion regarding framework in one direction or another. It was just an idea I has during implementing a set of tests of mine to reduce code duplication between tests and only later I realized, that it might make the porting to a test framework easier.

miri64 · 2018-11-19T20:18:22Z

(that's why I used the unittest module of python, because I assume most high-level test frameworks will have some way to integrate them).

cladmi · 2018-11-20T11:43:08Z

I would get it differently, the thing I did not wanted to do in my pytest PR, is to change the testrunner module as saying "if you rewrite everything then…" has low impact.

But my first step, if I allow myself to modify it, is to split the run function so that each part can be used alone. (so you do not need to re-use it directly).

In my pytest poc PR to write a fixture which is basically another way of doing the setup, teardown, I needed to duplicate everything from the run function. RIOT-OS:e0e02c0...cladmi:2291001#diff-85e19ffeb672d73150a8a8ae876c42a9R35
In python you cannot replace a function call by a yield and it is a known limitation.

With this, you can implement the TestCase with normal setupClass, teardownClass and even setup and teardown depending on your need.

Then your test implementation TestCase class would just inherit from it.
In all the time I wrote tests, I never needed to do this introspection with the issubclass, and I would not like to maintain it.

I used this as a base class a long time ago https://github.com/iot-lab/iot-lab-gateway/blob/344a3e166fff1af1c14fbf95c132ef1906251c7d/gateway_code/integration/test_integration_mock.py#L50-L89

Then running the test would only be to call the default unittest.run function or something.

I could propose something along those lines if you want.

And the second one, would be to write, not even in testrunner but in a normal riotnode package, a Node object with start, stop, expect_, other new functions.

As there is an object, you can subclass it (intelligently of course to be able to merge many class together), and then have a RIOTShellNode class.

kaspar030 · 2018-11-20T12:15:24Z

And the second one, would be to write, not even in testrunner but in a normal riotnode package, a Node object with start, stop, expect_, other new functions.

As there is an object, you can subclass it (intelligently of course to be able to merge many class together), and then have a RIOTShellNode class.

Sounds a lot like #7258. ;)

cladmi · 2018-11-20T12:43:34Z

And the second one, would be to write, not even in testrunner but in a normal riotnode package, a Node object with start, stop, expect_, other new functions.
As there is an object, you can subclass it (intelligently of course to be able to merge many class together), and then have a RIOTShellNode class.

Sounds a lot like #7258. ;)

+2,131 −22 line changes.

No it is definitely something more in a single PR. You do not need JSON to just write a python class.

miri64 · 2018-11-20T13:13:28Z

I would get it differently, the thing I did not wanted to do in my pytest PR, is to change the testrunner module as saying "if you rewrite everything then…" has low impact.

Arghs, I never said "rewrite everything". I just wanted a way to deduplicate code in my PRs. Porting the tests to this approach could however have benefits.

But my first step, if I allow myself to modify it, is to split the run function so that each part can be used alone. (so you do not need to re-use it directly).

That is a completely different direction from what I did here, but okay.

In my pytest poc PR to write a fixture which is basically another way of doing the setup, teardown, I needed to duplicate everything from the run function. RIOT-OS:e0e02c0...cladmi:2291001#diff-85e19ffeb672d73150a8a8ae876c42a9R35
In python you cannot replace a function call by a yield and it is a known limitation.

Don't really know how this is related, but okay (link leads to an empty diff btw).

With this, you can implement the TestCase with normal setupClass, teardownClass and even setup and teardown depending on your need.

I am not really convinced if either setupClass or setup do what I want with __init__, but if you say that's the way to go, then I'll try.

Then your test implementation TestCase class would just inherit from it.
In all the time I wrote tests, I never needed to do this introspection with the issubclass, and I would not like to maintain it.

I used this as a base class a long time ago https://github.com/iot-lab/iot-lab-gateway/blob/344a3e166fff1af1c14fbf95c132ef1906251c7d/gateway_code/integration/test_integration_mock.py#L50-L89

Yeah, we might be able to remove that, if the setupClass approach is chosen.

Then running the test would only be to call the default unittest.run function or something.

Most important thing for me was that it is still compatible with testsrunner's run function to not duplicate code.

I could propose something along those lines if you want.

Let me try fixing it myself first.

And the second one, would be to write, not even in testrunner but in a normal riotnode package, a Node object with start, stop, expect_, other new functions.

As there is an object, you can subclass it (intelligently of course to be able to merge many class together), and then have a RIOTShellNode class.

It would be a follow-up step to have module-specific mixins, as we have with the ReleaseSpec tests.

kaspar030 · 2018-11-20T15:23:09Z

Sounds a lot like #7258. ;)

+2,131 −22 line changes.

No it is definitely something more in a single PR. You do not need JSON to just write a python class.

I'm talking about the concept. Node class, RiotNode class, ShellNodeClass. You'll end up with JSON (or some other configuration format) once you realize that at some point, configuration of tests doesn't fit so nicely in code anymore.

miri64 · 2018-12-15T11:30:28Z

So how do we proceed with this? The decision of the framework should be unrelated to the fact if we have this or not, right?

MrKevinWeiss · 2018-12-17T08:35:36Z

@miri64 Agreed, I don't think that framework will replace this anyways, meaning that it is still needed. Any improvements on this would be valuable. It is also not unreasonable to think the framework could eventually take advantage of this as well.

miri64 · 2019-01-11T12:55:01Z

#10382 was merged so how do we proceed with this one. I can easily port the tests in #10382 and #10392
(and my future sock_dns tests I promised in #10739) to this framework.

miri64 · 2019-01-18T13:49:53Z

Ping @cladmi @MrKevinWeiss @smlng?

cladmi · 2019-01-21T16:43:08Z

I would split in two commits, the first one where setup_child and teardown_child are declared as non private functions, which could be re-used somewhere else.
And the second one introducing the unittest.TestCase.

And as testrunner was changed to a package, PexpectTestCase could be put in its own module testrunner/unittest.py.

e.g. this whole section in #10392 could be methods of a PexpectTestCase class that also contains all the interaction stuff (like pktbuf_empty(), add_neighbor(), del_neighbor(), etc.) with the node as methods:

In the long term, I am in favor or providing these methods in a non test class, so in your case would result in a testcase method self.spawn.add_neighbor(neigh) or self.node.add_neighbor it not called spawn but starting to go in this direction in parallel is still a good thing in my opinion.

Providing a base class for this is in my goals during this release.

miri64 · 2019-01-22T09:07:00Z

I would split in two commits, the first one where setup_child and teardown_child are declared as non private functions, which could be re-used somewhere else.
And the second one introducing the unittest.TestCase.

Does this mean I should squash?

miri64 · 2019-01-22T09:09:48Z

And as testrunner was changed to a package, PexpectTestCase could be put in its own module testrunner/unittest.py.

Will do.

e.g. this whole section in #10392 could be methods of a PexpectTestCase class that also contains all the interaction stuff (like pktbuf_empty(), add_neighbor(), del_neighbor(), etc.) with the node as methods:

In the long term, I am in favor or providing these methods in a non test class, so in your case would result in a testcase method self.spawn.add_neighbor(neigh) or self.node.add_neighbor it not called spawn but starting to go in this direction in parallel is still a good thing in my opinion.

Yes, maybe having some shell interaction class in-between would be a good idea!

miri64 · 2019-01-22T09:32:27Z

Rebased to current master and squashed as a consequence to also address @cladmi's comment.

dist/pythonlibs/testrunner/spawn.py

dist/pythonlibs/testrunner/unittest.py

cladmi · 2019-01-29T17:30:05Z

dist/pythonlibs/testrunner/__init__.py

-    except subprocess.CalledProcessError:
-        # make reset yields error on some boards even if successful
-        pass
+    child = setup_child(timeout, env=os.environ,


This one is not needed but was here before, so I can remove it later.

I guess the os.environ? Yeah, better save than sorry ;-)

cladmi · 2019-01-29T17:36:01Z

dist/pythonlibs/testrunner/unittest.py

+
+class PexpectTestCase(unittest.TestCase):
+    TIMEOUT = 10
+    LOGFILE = None


Not sure in unittest will be able to capture test output without LOGFILE to stdout or something, even on failures.
But I cannot really verify without using it.

Not sure either.. I usually use the output mainly for debugging, so that's why I deactivated it and didn't really think much about actual unittest integration. Let's tackle this when it actually becomes an issue.

cladmi · 2019-01-29T19:19:43Z

I tested tests with also failed outputs and the behavior is the same as before.
I could also re-use the pytest_child functions in my code.

Please squash and enable murdock, I would do a final pass then.

miri64 · 2019-01-29T19:23:58Z

Squashed

aabadie

Found some potential issues.

Also one fundamental question, how the PexpectTestCase is supposed to be used ? What is the benefit compared to the testrunner.run function ?

sys/net/gnrc/routing/rpl/gnrc_rpl_auto_init.c

dist/pythonlibs/testrunner/__init__.py

dist/pythonlibs/testrunner/spawn.py

This allows for re-usability of those functions in other contexts.

I had this idea when implementing RIOT-OS#10382 and RIOT-OS#10392 as I introduced a very similar structure to python's standard unittests in those and it could also reduce some code duplication between those two tests.

aabadie · 2019-01-29T20:21:05Z

also Codacy is being stressed out about all the assert()s in there

And it is right: assert is a keyword and should not be used as a function (already said that today ;) ). Having assert(a == b) is equivalent to assert (a == b), e.g assert a == b. It is syntactically valid but could lead to confusion.

miri64 · 2019-01-29T20:22:18Z

And it is right: assert is a keyword and should not be used as a function (already said that today ;) ). Having assert(a == b) is equivalent to assert (a == b), e.g assert a == b. It is syntactically valid but could lead to confusion.

I also used print() as a function even before I had to, just because it is confusing not to.

miri64 · 2019-01-29T20:23:23Z

Also: wrong PR I believe. I don't use assert() here ;-)

aabadie · 2019-01-29T20:24:11Z

I know it's off topic, just noticed it while reading the history of comments ;)

miri64 · 2019-01-29T20:35:13Z

Anyway. Murdock is now happy.

cladmi

ACK I reviewed what I could now. Other unrelated changes will come later.

I tested both test with both normal and error case, and I could reuse the pytest_child in my own context.

miri64 · 2019-01-30T06:29:19Z

@aabadie, your comments were addressed. What is your verdict?

comments addressed

miri64 · 2019-02-26T18:25:02Z

I give the CI another run and then I would merge. Having this ACK'd without merge is a long enough period for someone to complain, I believe ;-)

miri64 added Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation Area: tests Area: tests and testing framework Area: tools Area: Supplementary tools labels Nov 19, 2018

miri64 requested review from kb2ma, smlng and MrKevinWeiss November 19, 2018 16:12

miri64 added the Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR label Nov 19, 2018

miri64 mentioned this pull request Nov 23, 2018

RFC: Use RobotFramework to write RIOT tests #10241

Closed

miri64 mentioned this pull request Dec 15, 2018

tests/gnrc_ipv6_ext: port to scapy #10382

Merged

3 tasks

miri64 mentioned this pull request Jan 18, 2019

testutils: add helpers for running Release Specs RIOT-OS/Release-Specs#79

Closed

miri64 force-pushed the dist/enh/testcase-testrunner-wrapper branch from cb2c45b to ac50244 Compare January 22, 2019 09:32

cladmi reviewed Jan 28, 2019

View reviewed changes

dist/pythonlibs/testrunner/spawn.py Outdated Show resolved Hide resolved

cladmi reviewed Jan 28, 2019

View reviewed changes

dist/pythonlibs/testrunner/unittest.py Outdated Show resolved Hide resolved

cladmi reviewed Jan 29, 2019

View reviewed changes

miri64 force-pushed the dist/enh/testcase-testrunner-wrapper branch from 7a04ca8 to 47992d6 Compare January 29, 2019 19:23

miri64 added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jan 29, 2019

aabadie previously requested changes Jan 29, 2019

View reviewed changes

sys/net/gnrc/routing/rpl/gnrc_rpl_auto_init.c Outdated Show resolved Hide resolved

dist/pythonlibs/testrunner/__init__.py Outdated Show resolved Hide resolved

miri64 force-pushed the dist/enh/testcase-testrunner-wrapper branch from 47992d6 to acb2380 Compare January 29, 2019 19:35

cladmi reviewed Jan 29, 2019

View reviewed changes

dist/pythonlibs/testrunner/spawn.py Show resolved Hide resolved

dist/pythonlibs: expose functions to setup/teardown pexpect spawn

b2407e4

This allows for re-usability of those functions in other contexts.

miri64 force-pushed the dist/enh/testcase-testrunner-wrapper branch from acb2380 to a6594c5 Compare January 29, 2019 19:42

dist/pythonlibs: provide unittest TestCase wrapper for testrunner

4b5b5d9

I had this idea when implementing RIOT-OS#10382 and RIOT-OS#10392 as I introduced a very similar structure to python's standard unittests in those and it could also reduce some code duplication between those two tests.

miri64 force-pushed the dist/enh/testcase-testrunner-wrapper branch from a6594c5 to 4b5b5d9 Compare January 29, 2019 20:16

cladmi approved these changes Jan 29, 2019

View reviewed changes

This was referenced Jan 31, 2019

tests: add tests to reproduce #10881 #10908

Merged

tests/periph_rtt: add Python script for automatic testing + use 5s delay between Hellos #10893

Merged

cladmi mentioned this pull request Feb 5, 2019

riotnode: node abstraction package #10949

Closed

miri64 added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Feb 26, 2019

miri64 merged commit cfee6fa into RIOT-OS:master Feb 26, 2019

miri64 deleted the dist/enh/testcase-testrunner-wrapper branch February 26, 2019 18:55

danpetry added this to the Release 2019.04 milestone Mar 12, 2019

dist/pythonlibs: provide unittest TestCase wrapper for testrunner #10431

dist/pythonlibs: provide unittest TestCase wrapper for testrunner #10431

Uh oh!

Conversation

miri64 commented Nov 19, 2018

Contribution description

Regarding ongoing discussions about test frameworks

Testing procedure

Issues/PRs references

Uh oh!

miri64 commented Nov 19, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kb2ma commented Nov 19, 2018

Uh oh!

miri64 commented Nov 19, 2018

Uh oh!

miri64 commented Nov 19, 2018

Uh oh!

cladmi commented Nov 20, 2018

Uh oh!

kaspar030 commented Nov 20, 2018

Uh oh!

cladmi commented Nov 20, 2018

Uh oh!

miri64 commented Nov 20, 2018

Uh oh!

kaspar030 commented Nov 20, 2018

Uh oh!

miri64 commented Dec 15, 2018

Uh oh!

MrKevinWeiss commented Dec 17, 2018

Uh oh!

miri64 commented Jan 11, 2019

Uh oh!

miri64 commented Jan 18, 2019

Uh oh!

cladmi commented Jan 21, 2019

Uh oh!

miri64 commented Jan 22, 2019

Uh oh!

miri64 commented Jan 22, 2019

Uh oh!

miri64 commented Jan 22, 2019

Uh oh!

Uh oh!

Uh oh!

cladmi Jan 29, 2019

Choose a reason for hiding this comment

Uh oh!

miri64 Jan 29, 2019

Choose a reason for hiding this comment

Uh oh!

cladmi Jan 29, 2019

Choose a reason for hiding this comment

Uh oh!

miri64 Jan 29, 2019

Choose a reason for hiding this comment

Uh oh!

cladmi commented Jan 29, 2019

Uh oh!

miri64 commented Jan 29, 2019

Uh oh!

aabadie left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aabadie commented Jan 29, 2019

Uh oh!

miri64 commented Jan 29, 2019

Uh oh!

miri64 commented Jan 29, 2019

Uh oh!

aabadie commented Jan 29, 2019

Uh oh!

miri64 commented Jan 29, 2019

Uh oh!

miri64 commented Nov 19, 2018 •

edited

Loading

aabadie left a comment •

edited

Loading