We should make contribution guidelines clearer. We got the feedback that "Although the current project seems welcoming to new developers, there is little concrete information on how to get involved. For example, what code style does the project follow? What are the criteria for including a new algorithm: what documentation does it need to have, test cases, etc? There is little in the way of API docs either, this admittedly is not critical as the files are themselves quite readable, but I'd suggest adding at least one-line docstrings to classes, methods, etc (and consider splitting up the scripts into more separate methods as well)"

This is great feedback. To address it, let's re-think the checklist for including a new algorithm. Such a checklist should include:

1. pre-commit utilities: sort dependencies, remove unused variables and imports, format code using black, and check word spelling Introduce pre-commit pipelines #107.
2. Empirical analysis and benchmark: we should adopt a similar guide from sb3-contrib with a bit of our spin. The implemented algorithm should come with tracked experiments that
   • match the reported performance in the paper (if applicable)
   • match the reported performance in a high-quality reference implementation (SB3, Tianshou, and others) (if applicable).
   • We should also add documentation on how exactly we want the tracked experiments to be done (i.e., what W&B project? should they capture video recording?)
3. Documentation: the proposed algorithm should also come with documentation at https://docs.cleanrl.dev/rl-algorithms/ to
   • explain crucial implementation details
   • add links to the original paper and related papers (if applicable)
   • add links to the PR related to the algorithm
   • add links to the tracked experiments and benchmark results.
4. Tests: the proposed algorithm should come with an e2e test that makes sure the algorithm does not crash.

I will try to make some examples next week.