Photo: Brett Jordan

What are the 4 metrics?

The four key metrics are Deployment Frequency (the frequency at which new releases go to production), Lead Time For Changes (the time until a commit goes to production), Mean Time to Restore (the time it takes to resolve a service impairment in production) and Change Failure Rate (the ratio of deployments to production ...

infoq.com - Experiences from Measuring the DevOps Four Key Metrics - InfoQ

How long before bed should I take ashwagandha?

While effects can vary person to person and on the dosage, taking Ashwagandha in capsule form at least a half hour before bedtime should allow the...

Last updated Aug 1, 2020

Is it OK to eat cranberries raw?

Cranberries are more than just the makings of a sweet sauce. You can add them to smoothies or salads, or eat them whole. Eating raw cranberries is...

Last updated Oct 2, 2018

Promotion

Measuring the four key metrics of DevOps, which were original specified within the Accelerate book, helped a company to assess the performance of their software delivery process. Continuous observation of these metrics supports decisions on where to invest and guides performance improvements. Nikolaus Huber, a software architect at Reservix, shared his experiences from measuring the software delivery process of their SaaS product at DevOpsCon Berlin 2021. The four key metrics are Deployment Frequency (the frequency at which new releases go to production), Lead Time For Changes (the time until a commit goes to production), Mean Time to Restore (the time it takes to resolve a service impairment in production) and Change Failure Rate (the ratio of deployments to production that leads to errors and successful deployments).

These metrics have a strong scientific background, Huber explained:

The four key metrics have been described in the book Accelerate (see the Q&A on Accelerate on InfoQ) and further improved in the State of DevOps Reports. The authors applied cluster analysis to their survey data and found out that these metrics can be used to categorize the performance of the software delivery process. For example: elite performers deploy on demand whereas low performers deploy between once per month and once in six months. Measuring these metrics can help you to assess the performance of your software delivery process, Huber mentioned. As software delivery performance has a positive impact on your organizational performance, you can also work on improving your software delivery process by improving these metrics. Huber described what their deployment frequency looks like and what they have learned from analyzing the data: Between January and May 2021, I observed a mean deployment frequency of 13.04 deployments per day, which is pretty good. Our fully automated deployment process gives us the ability to deploy quickly and on demand. And it’s good to have small batch sizes and integrate frequently. This is why we have such a good deployment frequency. Measuring the change failure rate turned out to be rather challenging, as Huber explained: The change failure rate turned out to be the most complicated metric to measure for us since we currently have no link between a deployment and a system impairment. So I had to find workarounds and ended up analyzing the deployment job log and the Git history for keywords like "revert" and "hotfix" to calculate the change failure rate.

What are the 12 dimensions of health?

The continuum includes: 1) Self-responsibility & Love; 2) Breathing; 3) Sensing; 4) Eating; 5)Moving; 6) Feeling; 7) Thinking; 8) Playing &...

Last updated Nov 14, 2020

How do I survive a sexless wife?

How to cope with a sexless marriage Pick your moment to talk. ... Pick your moment to listen. ... Be honest with yourself and each other. ......

Last updated Aug 8, 2021

Promotion

Huber mentioned that they need to invest in tool improvements to get a better understanding on which deployment caused an error: In particular, we need a link between deployment and potential system impairment, e.g. by linking deployment job log and monitoring data, or by logging information about the released version on errors. InfoQ interviewed Nikolaus Huber about their experience in measuring the software delivery process.

InfoQ: What methods and tools have you used to obtain the four key metrics for the software delivery process of your SaaS product?

Nikolaus Huber: I examined different data sources (JIRA reports, Git history, deployment job logs, incident reports, and Gitlab analytics data) and ended up using simply the Git history and the deployment job log as data sources. I wrote R Markdown scripts to import and analyze the data and calculate the metrics described above. The approach is not yet fully automated (MTTR is calculated manually from incident reports) and needs some tooling improvement, but it’s simple and convenient. I also had a look at other tools like Gitlab Analytics or the Four Keys Project, but they were not applicable to our situation. The reasons are that with Gitlab, you need to use the full feature set (e.g. the issue tracker) which we don’t, and the Four Keys Project requires dedicated infrastructure which I don’t want to maintain.

InfoQ: What did you learn from measuring the lead time for changes?

Huber: We have a mean lead time for change of 1.25 days, which is a good value (high performer category). Actually, this value was already this good when I started working on measuring these metrics. So far, we haven’t worked on improvements. Conversely, measuring this value helped us to not slow down the tempo as we also had discussions on whether the tempo was too fast. But comparing our metrics to others gives us the confidence to keep the tempo. If we experience too many errors, then we should rather work on improving the stability metrics (MTTR and Change Failure Rate).

InfoQ: How did you analyze the mean time to restore data? What insights did that bring you?

What's the rarest eye color?

a day ago Green Green is the rarest eye color in the world, with only 2% of the world's population (and fewer than one out of ten Americans)...

Last updated Jul 14, 2022

What is the best milk for high cholesterol?

The best dairy milk for people with high cholesterol is fat-free or skim milk. Plant-based milks, such as soy milk, almond milk, or oat milk, are...

Last updated Dec 23, 2018

Imagine Your Moment Of Freedom!

Fluxactive Complete is conveniently packed with over 14 essential prostate powerhouse herbs, vitamins and grade A nutrients which work synergistically to help you support a healthy prostate faster

Learn More »

Promotion

Huber: Currently, analyzing Mean Time to Restore is difficult since I don’t have monitoring data over a long period of time, which is why I had to fall back on writing incident reports when we have a system outage. So here my insights were that we needed to improve our tooling to get more reliable and automated data analysis. From the pure metrics perspective, I can say that the root cause of the longest system outages was the infrastructure provider. Here, we have already taken countermeasures by moving services to the cloud. It would be interesting to analyse the effect of these countermeasures on MTTR, which is part of future work.

InfoQ: What have you learned from measuring and analyzing the four key metrics?

Huber: There are a lot of learnings. First of all, measuring software delivery is possible but also more time-consuming than I thought. So if you’re interested only in a quick comparison of your software delivery performance, start by estimating the metrics for your process. Nevertheless, investing more time into analysing these metrics helped us to get a better understanding of the software delivery process and what is important. For example, for us it is better to invest in stability rather than reducing tempo. This was a very valuable discussion and insight which we would not have if we hadn’t invested in examining these metrics. And in general, we as a team learned that the software delivery process we have is performing very well and that this is a great success.

InfoQ: What will be the next steps?

infoq.com - Experiences from Measuring the DevOps Four Key Metrics - InfoQ

What illness makes you itchy?

Itching on the whole body might be a symptom of an underlying illness, such as liver disease, kidney disease, anemia, diabetes, thyroid problems,...

Last updated Jan 14, 2022

Is 100% pure dark chocolate healthy?

Dark chocolate is loaded with nutrients that can positively affect your health. Made from the seed of the cacao tree, it's one of the best sources...

Last updated Dec 24, 2021

Promotion

What race is naturally blonde?

Naturally-occurring blond hair is primarily found in people living in or descended from people who lived in the northern half of Europe, and may...

Last updated Nov 9, 2022

Promotion

How many cranberry pills equal a glass of cranberry juice?

A cranberry capsule is much more concentrated, and is the equivalent of drinking nearly 28 ounces of juice, which is why we recommend them to our...

Last updated Sep 17, 2019