Hackweek is here! I think this is the best week of the year to sit down altogether and exchange ideas and suggestions. The main topic is Automation. The goal is that many of these ideas might help various teams within SUSE to engage their business reasons better in defining key expectations and improve the quality of our software products. No fear of change -- the aim is to propose a modern pipeline in a less-invasive manner. Everybody has an idea, everybody has a voice! Brainstorming together can be useful to many different roles, including testers, analysts and developers. Let's have a chit-chat and write down some of those; Hopefully we will come up with plenty of tips on how to organise testing activities better.

> Coming together is a beginning. Keeping together is a progress. Working together is a success. - Henry Ford

Continuous Delivery is a scientific and systematic driven approach, that requires a small team of people with the same beliefs and passion and values the small changes as much as large changes. It’s all about getting good at getting better.

I am available the whole Hackweek -- also ping me on IRC on the #Maintenance channel (nickname: panos).

Feel free to come by my office and discuss :-)

So, what we will talk about?

Define a shared big-picture view of quality

Quality is a difficult concept to pin down. This is because everyone has its own interpretation.

“I am not sure if this is a bug, or an undocumented feature”

Something one person considers critical might not even register on the scale of importance for someone from a different department. That is why a seemingly critical defect can be left in a software product for months, sitting in Bugzilla, often serving just to prevent people sending more notifications about the same issue. A good solution for this misunderstanding is to create a multi-layered, multi-faced view of quality that different departments can agree on.

One model that works relatively well in many situations is based on Maslow’s hierarchy of human needs.The famous Maslow’s hierarchy lists human needs as a pyramid with those necessary for basic functions on the bottom level, safety, love and belonging, and esteem on the middle levels and self-actualisation at the top. The premise of the hierarchy of needs is that when a lower-level need is not met, we disregard higher-level needs. As with any other models, this one is an abstraction, but in general it captures the situation relatively well. We can do something similar for software:

Drawing parallels between the different levels of needs, we can create a pyramid of software quality levels:

Does it compiles at all? What is the status of the unit tests? Is it packaged for all products and architectures? Does it work at all? What are the key deployment tests? Does it work well? What are the integration, performance, regression and security tests? Is it useful? What production metric will show that it is used in real work? Is it successful? What business metrics will show that this was worth doing? Is it operating within financial constraints?

Such a pyramid can help teams to define acceptance criteria at each of the levels, and create shared agreements on what the entire group means when they think about quality.

Key benefits

A shared visualisation of different aspects of quality helps to paint the big picture for everyone. People can avoid spending too much time improving aspects that are already derisked to a satisfactory level. At the same time, everyone will understand what constitutes a critical problem -- if any of the things in the pyramid is broken, it’s pretty serious, regardless of what level it is on.

Although higher levels of the pyramid -- typically characterised by production metrics -- might not be fully testable during development, they provide a useful context for designing the right solution as a whole.

How to make it work

The whole purpose of creating a shared big picture among the different departments in SUSE is to align expectations across different groups, so it only makes sense to do it in a cross-functional setting, with representatives of developers, packagers, testers, analysts, devops, security engineers and business managers. I welcome representatives from each of those groups to come into my office, and draw the pyramid on a large whiteboard. Then let the people add their expectations with sticky notes to each of the levels. After twenty (20) minutes, or when the discussion stops and people have no more ideas, go through the notes and try to quantify expectations. For example if someone says ‘test quickly security updates’ to the third level, define together what ‘quickly’ actually means. Avoid trying to capture all possible test cases, and focus on the broad understanding that will allow you to make better low-level decisions later.

Decrease overhead and rework

DevOps is considered as a movement or the philosophy to help Development and Operations work better together. Better collaboration, better communication and most importantly to build trust to each other and reduce the misunderstanding we see in these handoffs between Dev and Ops. It’s all about how do we get an idea, a feature, a bug fix, or a customer’s request, all the way up to our production.

The process, it’s not as simple as it sounds. There’s a whole number of people in between who are involved in creating, testing, and delivering the software. These people might be some business analysts, designers, architects, obviously some coders, testers, release engineers, deployment engineers, operations and infrastructure people, cloud admins ... they are all in this process. The goal is to be able to take the software from one environment into the next automatically.

The improvement comes in two ways, such as reducing any kind of rework we might have to do, and secondly is to reduce any kind of overhead. Overhead is something we would like to reduce. If the developer reproduces the bug using a complex setup, and then the QAM Engineer has to recreate this setup from scratch, this is overhead.

What's the current status?

I don't know and that's what I am expecting to learn and share. I welcome the people who are working on this field, to come by my office and have a chit-chat on that matter.

Some related videos to get yourself inspired: