When it comes to building quality software, why is Observability so critical? Does it help deal with the “unknown unknowns” of software quality today? How does observability relate to monitoring, testing, and chaos engineering? Let’s find out in this article, featuring Federico Toledo and Lisa Crispin’s voices.

By Natalie Rodgers

Have you ever stopped to think about the relevance of applying Observability on a daily basis? Some time ago, Federico Toledo, COO at Abstracta, sat down for a while and got to work. He interviewed Lisa Crispin for his podcast Quality Sense. She co-founded the Agile Test Fellowship with Janet Gregory and is the author of numerous influential books on agile testing. 

Maybe you didn’t listen to the episode or maybe you are one of those who prefer to read. But Observability is getting more and more attention. That’s why we will go over the main concepts Federico Toledo and Lisa Crispin discussed.

Observability: What does it Really Mean?

According to Lisa Crispin, even people working in operations who have done monitoring and log-in, don’t really know what observability is about. Sometimes they get confused and think that it’s just a new name for monitoring. There is a lot of debate around observability! But if something is sure is that observability is definitely different from monitoring.

Lisa explained, “Observability is instrumenting more things in our code, more events in our system, so that if something goes wrong that we totally could not have imagined that could happen (even with all of the risk assessment and things that we did in advance), we can still delve into what has gone wrong and see what to do about it. We can also trace what users did. So, even though we really  have a lot of complex distributed systems now, we can see what path our user took through our application.”

Lisa then went deeper: “Observability allows us to trace what users did across all the different services and APIs to see exactly all the steps they took so we can reproduce or directly fix the problem. It’s a way to explore production using really sophisticated tools that are available to us and also because we’re able to store huge amounts of data at an affordable cost now.”

Federico Toledo remarked that observability is key to having the possibility of “analyzing more things in different parts of the system because the systems have more components and more things to monitor or analyze.” 

“Basically, we have to get ready for possible problems that we don’t know about,” outlined Federico. These are known as “Unknown Unknowns,” and they are undoubtedly one of the biggest challenges when it comes to building quality software. 

Observability and Software Testing

With continuous delivery, continuous integrations, and agile teams come a lot of challenges in the digital era. Software is becoming more and more complex and it’s getting harder and harder to test everything.

In this context, Federico Toledo and Lisa Crispin agreed that the goal of observability and testing is basically the same. “Because it’s like getting information about the usage of the system or the quality, or considering different aspects of the application with the goal of providing this information to someone who is going to make a decision”, he said.

“We still want to do all the testing we can before we release. But we have to take advantage of all these extra approaches like observability, testing in production and using feature toggles, chaos engineering”, specified Lisa.

“I think it’s all part of the package and testers really need to be involved on both sides of that continuous delivery loop or DevOps loop. We need to be involved all the way around because we need that firsthand knowledge of what’s going on in production as well,” she added.

Matías Fornara, leader of the automation hub at Abstracta, explained in this article that with the popularization of the DevOps concept, Shift-Right practices have also become linked to application release, configuration, and monitoring processes. And, undeniably, a key concept associated with this is observability.

Federico Toledo commented that chaos engineering is helpful to check how observable our system is. By injecting errors the whole team can understand if they are capable or not of analyzing and fixing the issue with the information they have.

Lisa Crispin exclaimed, “Chaos engineering is one way to discover the unknown unknowns.”

Long story short: testers should be really involved because observability is actually a property of their systems. They need to know the architecture and the technology behind their systems to be able to understand where to pay attention to be able to test how observable their systems are.

Some Stats

Recent research published by Splunk explained that as the evolution to hybrid, multi-cloud infrastructure continues, organizations are encountering escalated complexity, lower visibility, and intensifying burnout. 

“Fluid observability is the antidote… As observability becomes essential, leaders extend multi-cloud visibility and dramatically improve digital experiences,” it held. “Organizations that build a rich observability practice into everything they do can expect to see compounding benefits in security, performance, and cost savings each year.”

The research includes a survey of 1,250 observability practitioners, managers, and experts on this topic. Some of their results revealed that:

✔️Observability leaders are 2.1 times as likely to say that they can detect problems in internally developed applications in minutes.
✔️Leaders report a 69% better mean time to resolution for unplanned downtime or performance degradation.
✔️ Leaders’ average annual cost of downtime associated with business-critical internally developed applications is $2.5 million, versus $23.8 million for beginners.

Observability remains a huge challenge. There is no doubt that there’s still much to come!

“The future of observability is a continued expansion throughout the IT organization. For all but the youngest digital-native businesses, traditional IT operations have bigger teams and bigger budgets than DevOps and observability. But the proliferation of clouds and the constant pressure to move faster will drive observability forward,” concluded the report. 

Would you like to learn more about observability? We recommend you hear this whole episode of Quality Sense!

Are you looking for your ideal software testing partner? Abstracta is one of the most trusted companies in software quality engineering. Contact us to discuss how we can help you grow your business. Follow us on Linkedin & Twitter to be part of our community!