In our recent webinar, you asked… Lisa answered!

This month, we had the opportunity to have Lisa Crispin present her and George Dinwiddie’s ideas about BDD and CD; mainly how BDD can shorten feedback loops, how CD impacts BDD and things your team can try.

There were so many questions from the audience that we ran out of time. Fortunately, that didn’t mean we’ve left your questions unanswered!

Webinar Recap

Watch the webinar below or check out Ali Hill’s live tweets from the webinar (thanks Ali!), which we summarized here:

• What is BDD? BDD involves structured conversations to help elicit rules and examples within your team.
• What’s Continuous Delivery/Deployment (CD)? CD allows you to deploy small low-risk changes to production. This allows you to gain faster feedback.
• BDD helps slice features into small stories, which helps with CD as CD works well with small changes. It allows us to explore how a capability will be used, therefore allowing us to build the “right” thing.
• Principles of the Continuous Delivery book support BDD. Build quality in, work in small batches, everyone is responsible, etc. Fast feedback is gained from automated tests and information gathered from production, from the real usage of the system. Feature toggles and canary releases in order to make adjustments to what users have according to what you learn from usage metrics and errors from production.
• What if you are practicing CD but not BDD? Try a small experiment, she says. Have a”three amigos” meeting (or even more than three) before your next planning meeting. Discuss goals, rules and examples of each story. Before coding begins, write scenarios based on these examples. This will guide development.
• Build a quality culture. Psychological safety is a pre-requisite to having a high performing team. Try small experiments when creating this culture. Focus on quality and not speed when you’re first starting out. TDD takes time to learn, for example, but helps improve quality.
• Talk about it with your whole team. How can you gain shared understanding? How can you shorten the feedback loop? How can you continuously deliver value to customers?

BDD and CD with @lisacrispin. Lisa will talk about how BDD can shorten feedback loops, how CD impacts BDD and things your team can try. pic.twitter.com/NC64xKt4Zc

— Ali Hill (@ali_hill91) May 5, 2020

Q&A

Question:

You mentioned that it would be great to have accessibility as a topic in the three amigos meeting. What do you think about including an automated accessibility tool into our CI?

Answer:

What my team did for this was agree on accessibility standards. For example, we decided to have aria labels for all UI elements. We could verify those were in place with our static analysis tool, a linting tool. That was fast and simple. There may be other tools I’m not aware of, it’s worth researching.

Question:

I’d like to know what is your best advice for a small team starting with BDD and CI?

Answer:

If you don’t have your CI in place, that’s your first priority.

You can start introducing BDD by having three amigos discussions before your iteration planning meetings. I find example mapping is easy to introduce and doesn’t take much extra time. You can start with BDD from there, turning those concrete examples into test specifications. Don’t worry if you don’t automate them right away – the shared understanding matters most. As far as automation, focus on automating unit tests first (preferably by doing test-driven development, which will prevent a huge percentage of bugs).

Question:

Where does BDD fit in the deployment pipeline with TDD?

Answer:

BDD comes before continuous integration, it’s not in the visualization of a deployment pipeline that I showed in the talk. It helps ensure that small changes, still valuable for customers, go into the pipeline one commit at a time, to help succeed with CD.

Question:

We often use the story splitting cheat sheet, but still have stories that go for 1 or even 2 iterations. Do you have any more suggestions on how to split smaller?

Answer:

I don’t know which story splitting technique you’re using (I like story mapping from Jeff Patton, also Richard Lawrence’s technique). It’s worth giving example mapping a try.

If you come up with a lot of rules and/or a lot of examples for a story, you know it’s too big, and the rules and examples indicate where you can slice the story.

Question:

What about security testing?

Answer:

Security tests can also come out in pre-planning sessions and expressed in business rules and examples. You can even write BDD-style test specifications.There are tools to automate security testing. So it can all be part of your deployment pipeline.

Question:

80% less bugs with TDD whenever compared with other teams. That’s quite a number! I wonder what would be the impact of BDD?

Answer:

“The impact of BDD, in my experience, is that we deliver the right thing, what the customer really wanted. I’ve experienced the frustration of delivering a ‘bug-free’, reliable piece of software, only to be told it isn’t what the customer wanted.“

LISA CRISPIN

Question:

In the majority of teams I have worked in as an SDET, test automation is always playing catch up, let alone complete before dev. Have you worked in a team that has been able to get tests automated before dev or even after the sprint ends? Any tips?

Answer:

The key is to first, slice the stories up small enough – something that can be finished in a day or two, including the automated tests – and, the team committing to focusing on finishing one story at a time.

It doesn’t work for many people to be working in the same part of the code base, but different team members can volunteer to do different testing activities, so that it all gets done faster. Working on stories with a “steel thread” approach, where you start with the simplest end to end implementation, automate tests for that, and add to it incrementally

Question:

What are some tips for building that whole team mentality for quality in a small team where it may not “seem” necessary?

Answer:

I’ve found it makes a difference to get the whole team together and discuss the level of quality you want to achieve. Make a commitment to delivering that, because you know you’ll run into problems.

Question:

When mixing BDD and CD, teams tend to confuse BDD as an automation tool for programming abstraction and forget about sharing knowledge and communication. Any thought on how to avoid it?

Answer:

Yes, I know the folks at Cucumber constantly try to dispel the notion that it is a test automation tool! I found that introducing example mapping helps. It is so simple and fast, and builds shared understanding.

Question:

How can we do 3 amigos or 4 etc when a user story is to setup infra projects?

Answer:

If the work to be done is well understood, no need for these conversations. But if there’s any chance of misunderstanding or uncertainty, get the appropriate people together – maybe an operations expert, a developer, the PO and a tester?

Question:

How to BDD for user stories like upgrade library, separate components, create swagger file…?

Answer:

I’m going to repeat the answer to a similar question: If the work to be done is well understood, no need for these conversations. But if there’s any chance of misunderstanding or uncertainty, get the appropriate people together. An operations person, a test automation specialist, whoever has the in-depth knowledge, along with the product person, tester, dev.

Question:

If we define BDD in user story then FE and BE have to write only those as their unit tests?

Answer:

Unit tests can be written with a BDD style format, such as given/when/then, but they are technology-facing tests that guide development at the code design level. What I was talking about in this context is business-facing tests that guide development at the story level. If it’s possible to push a test down to the unit level, usually that’s a good plan. However, if non-coders need to be able to read the tests, or if you want useful “living documentation”, you may want them in higher-level tests such as API tests or UI-level tests.

Question:

How would you promote the value of exploratory testing in a CD environment?

Answer:

I explain it this way:

“BDD covers the ‘knowns’, the desired behavior of the code, including what should happen when known misbehaviors occur (required fields missing, for example). Automating those tests gives us time to do exploratory testing, which is where we uncover the unknowns, the things we didn’t think about before we delivered the story.”

LISA CRISPIN

I’ve had success with holding short workshops to let non-testers on the team practice exploratory testing, also with doing mob exploratory testing, and pair exploratory testing. Also, you can introduce exploratory testing charters (based on Elisabeth Hendrickson’s Explore It book) to bug bash sessions to give people a taste of it.

Question:

How can you organize collaborative work on CD and BDD in case your organization has several developmental teams which release together?

Answer:

Communities of practice can help. My current organization has people from teams that have successfully done some practice or technique go work with other teams temporarily to help them learn.

Our quality team also is able to spread good ideas and techniques. Having “share and tell” sessions where teams can give a brief overview of what they’ve done and let people know who to talk to for more information is helpful.

Question:

How would you test ML software i.e. statistically models using BDD approaches?

Answer:

I don’t have experience testing ML models, but I don’t see why BDD approaches wouldn’t work. I know there’s a lot more to testing ML models and algorithms, for example, it’s recommended that you use 2/3 of the training data to train the model and keep 1/3 to test the model.

Angie Jones has some great content on testing AI and ML.

Question:

How to distribute the tests in the test pyramid, teams practicing BDD, they define scenarios in Given/When/Then. Then, they assume all tests will be at the same level , perhaps UI based tests. But most of tests can be lower in the test pyramid, therefore, for tests at lower levels, we won’t have the reports or living documentation. Any advice?

Answer:

Whether non-coders need to be able to read the tests or you want tests as living doc is an important consideration in deciding at what level to automate. If you do need readable tests, you could still automate at the API level if that’s feasible. See Seb Rose’s Test Automation Iceberg model: http://claysnow.co.uk/the-testing-iceberg/

Question:

How do you go about documentation in agile projects, usually at big corporations, they have all kind of documents for code design, test design, architecture, test results, and so on. You need to manually maintain all these word documents, cumbersome. How to find the right balance and convince organizations, people usually says, well this is the company template, we need to follow that..?

Answer:

Documentation is a thing whether or not your project is agile. I see a lot of approaches for documenting architecture, code design and the like, including wikis, a repo such as GitHub. Automated tests with easily-readable results make great living documentation. I’ve seen these approaches used even in regulated environments. If you have auditors to think about, you can talk to them and show them your ideas, they are usually amenable.

Question:

Micro services testing has many more layers than normal testing pyramid – how could we apply BDD for all layers, especially integration testing?

Answer:

Micro services are basically APIs, and a BDD approach works at the API level. I’m not even talking about automating the tests – I’m talking about getting the shared understanding before or at least during coding, through conversations where you get concrete examples to illustrate the business rules. If you can also automate the tests, so much the better, but that’s not the main goal of BDD.

Question:

Sounds like BDD is meant for realizing new features and new business ideas. How about old rusty features that need changes? I think of many things can go wrong when trying to add a change to an existing/legacy feature. Documenting BDD scripts for a feature that has been there for five or more years doesn’t sound much fun. People tend to drown with tech debt and tech guilt (nice one!) – any recommendations to get it started?

Answer:

I’d get together with my team and think of some small experiments to try. If you’re updating an existing feature, it’s a nice opportunity to do TDD and/or BDD to help document it for your future self or the other future people who have to change the code. The Scout rule applies – leave the campground cleaner than you found it. Saves lots of time in the long run.

Question:

What tools & approaches do you recommend? For example, to communicate and automate test cases: Gherkin, API Contracts, Snapshots/Screenshots… And how to distribute them? End2End, Integration, Unit Tests?

Answer:

This is something for the team to decide together. I like to use visual models like the test automation pyramid to generate good conversations amongst team members with different specialties. Tools are a big investment, hopefully you’ll use them a long time, so invest the time to think about what fits your context best, try out different possibilities, and find what works bets for you.

Question:

Is it possible to access these webinar slides?

Answer:

https://www.slideshare.net/lisacrispin/bdd-and-cd-better-together

Thanks Lisa so much for sharing with us!

---

Recommended for You

Webinar: BDD and CD with Lisa Crispin
A Guide to Good Cucumber Practices