We've (mostly @cbruckmayer) put a significant amount of time into our performance measurement dashboards based on influx/grafana. I want to share this with the world because it's most awesome and easy to use. It's also nice if people don't go to some PaaS provider just for this...

Currently, the SUMA test-suite takes about 6 hours to complete, often fails in the first tests, which set up the environment for the rest of the tests, those tests are what we named "core features". To solve this problem we had planned to move from Jenkins Job to a Jenkins Pipeline, having stages to split the test suite into core features, initialize clients, secondary features. So,if one stage fails, the rest of the stages will not be executed.

There are attempts to solve some kernel deadlock with using lockless printk ringbuffer. The proposed implementation is pretty complex (6 stages, 6 write and 6 read barriers, two buffers, entries linked via list, ...) I have a idea how to make it easier with tracking the state and sequence number in one atomic variable. It might allow to remove the lists and many barriers. It is possible that it will just not work. Let's see.