The issue

#106 introduced a retry on HTTP 429 code meaning "retry later" as many links on Github experienced this issues.
But even if it mostly works, sometimes it's obviously not enough.

Example screenshot of a CI pipeline that needed to be retried twice before being all green on link checking. Without changing any thing between the runs of course.
The matching commit is Consensys/doc.goquorum@f3eb2ca

The possible causes

It may be because of too many links tested and Github increases the expected retry delay value then when retry is done on our side, it may be too soon as other requests made the value increase.
There's some research to do here.

Options

One idea it raises is that, as other open issues (#40 #111), having a broader computation that just links taken one by one could make this improve. Having a context where we know if we have other links waiting for retry and what the latest duration value is for a specific domain may help.
But first we have to clarify what the issue is exactly. It's not easy to test as it's not something we can reproduce locally when targeting Github links (429 only happens when ran from CI) but we can try and perhaps ask Github directly.