API

Enabling StatsD reporting could be done via a CLI flag, --statsd, that would accept a host/IP. Additionally we should fallback to the environment variable RSPECQ_STATSD.

Metrics

Metrics we could report (<ns> stands for <namespace>) grouped by type:

Counters

• number of successful builds <ns>.builds.total
• number of successful builds <ns>.builds.successful
• number of successful but flaky builds <ns>.builds.successfulFlaky
• number of failed builds <ns>.builds.failed
• number of failed-fast builds <ns>.builds.failed_fast
• number of builds with a non-example error <ns>.builds.errored

Timers

• [reporter] build total run time <ns>.totalRuntime
• [worker] queue initialization run time <ns>.queueInitRuntime
• [reporter] run times of slowest jobs (top 10) <ns>.slowestJobs.<job>

Gauges

• [reporter] number of examples executed <ns>.examples
• [queue] number of flaky examples <ns>.flakeyTests
• [reporter] number of requeues <ns>.requeues
• [reporter] number of example failures <ns>.failures
• [reporter] number of non-example errors (e.g. syntax errors) <ns>.errors
• [worker?] number of worker failures <ns>.workerFailures
• [worker] total number of spec files <ns>.specFiles
• [worker] total queue size (aka. number of jobs) <ns>.queueSize
• [worker] number of spec files splitted <ns>.filesSplitted
• [worker] number of jobs generated from the splitted files <ns>.jobsFromSplit
• [worker] new (untimed) job received <ns>.untimedJobs