Save yourself some trouble, here’s how to create an effective test report—no frills required—that stakeholders will thank you for

We test software in order to gather information about the quality and risks, the product, its users, and the conditions of its use. This information helps our clients to make empirically informed decisions about the product, project, or business. One way of giving visibility to our test results is to share them through test reports.

So, you’ve thoughtfully planned a test strategy and carefully carried out your performance, exploratory, manual, or automated tests, etc. What good is the information you uncover if it’s not clearly presented in a way that allows for it to be acted upon?

This is where writing a good test report comes into play. It doesn’t have to be a tedious process, and although it may seem more polished or impressive, please forget the frill. Time is of the essence and stakeholders want testers to get to the point. So, here’s how to create an effective test report that’s concise, that your team will be happy to read.

Keep in mind, testers don’t always have to write test reports. In agile methodologies it is not often required or useful. In our context, working with teams with different methodologies, and in most cases, working with distributed teams, depending on the context, we find ourselves writing reports. So, the purpose of our work is not on documenting, but on delivering working software as the Agile Manifesto says, we want to get the most out of our time spent in reporting.

Tips for a More Effective Test Report

The Medium

In my opinion, sending a plain text document is the best. It can be read from any device: laptop, tablet, and smartphone. Include no attachments, so it’s very convenient and does not consume a lot of bandwidth. Of course, any reports you can add that are automatically generated from your testing tools are valuable to share as well.

What to Include in a Test Report

Your test report may be lean, but it should include:

1. What has been tested:

• What areas of the product (or products)
• What version(s)
• What environment/configuration
• What test scenarios (including automated tests, if applicable 

2. What has not been tested and why

It’s important to show your thought processes and simply state the reasons why you tested what you did and what you did not. You can point out that due to time constraints, X,Y, and Z were left for future testing. We must talk about risks, and not having test something points to a risk.

3. The results:

• Which scenarios have passed
• Which scenarios have failed (attach bug numbers if applicable)
• What is left to test (if nothing and the task is completed as requested, great, include that information too 🙂 )

4. Conclusions from the tests

Be careful here, use safe language, don’t assume responsibility that does not correspond to you (i.e. don’t say something “There are no bugs.” or “The product is fine to be released.”)

Did the product/version/feature reach the defined acceptance criteria?

If not, why? For example: X number of critical/serious bugs must be fixed or some more tests must be completed.

Are there any risks involved? For example: the security tests have not been run due to time and resource constraints.

5. Any relevant research carried out prior

If your project/contract is paid per hour, I would also suggest to list any kind of research/investigation you have done (and if applicable, what came from it). For example, “4 hours of research to check what test framework would be the best to automate some test scenarios and add them to Jenkins.”

Leave Room for Discussion and Feedback

At the very end of each report add this sentence, “Please let me know if you have any questions or concerns or if you need more information.” While rare, if a client does not like something, then they will let you know, and you can take it into account for future testing and reporting.

You may also write, “Tell me if the information provided in this report gives you enough visibility on the quality process and state, or if you would like to see any other data.”

Keep it Simple

I would strongly suggest to first come up with the above scheme of a report and wait for the feedback from a client and NOT ask what kind of report or format they want. Chances are that they will accept the simplest, most concise report once they see it, and won’t demand any “bells and whistles” forms, charts etc.

In your experience, what are some of the fundamental things to include in an effective test report?

Thank you Federico Toledo for collaborating with me in this post.

Recommended for You

How to Make a Performance Test Plan
When Do You Need to Hire a Software Tester?