Welcome to the 4th season of the Quality Sense podcast! In this first episode we bring you this interesting conversation between our host and COO, Federico Toledo and our guest, Paul Holland. In this episode, Paul tells Federico how we went from studying computer science, to airplane pilot, to then tester. Quite the ride, huh? This, brainstorming, creativity and much more in this new episode!
Episode Highlights
- How Paul went from computer science, to airplane pilot, to tester.
- Testing workshops and its mental exercises. Why are they useful?
- How testing works when trying to assess and find risks.
- Paul’s opinion on how to know if my brainstorming with my peers/team is good enough.
- Creativity: how to think in different ways.
- Typical mistakes testers do when doing a risk analysis.
Relevant Links:
- Follow Paul on Twitter: @PaulHolland_TWN
- Check out Paul’s website: https://testingthoughts.com/
Listen Here
Episode Transcript
Federico:
Hello, Paul. Finally, we did it. We are here.
Paul:
We are here.
Federico:
Yeah.
Paul:
Fourth attempt.
Federico:
Fourth attempt. You’re right. But yeah, now we are here. So excited to have you here in the show. It’s been a while since the last time we had a chance to have a chat. I think it was in…
Paul:
France.
Federico:
… in France in the [Whopper], right?
Paul:
Yeah. In Marseille.
Federico:
It was an amazing week over there.
Paul:
It was a very nice week.
Federico:
And I was looking forward to more conferences, in-person conferences, right?
Paul:
I am going to be going to Germany in November. So I’m doing the agile testing days. I’m doing a tutorial with [Hibe Schutz].
Federico:
Amazing. That’s awesome.
Paul:
What is amazing is how I just badly pronounced his name actually because I’m sure I didn’t do it right.
Federico:
Paul, one of the first questions I have for you is how did you end up working in software testing? How did that happen?
Paul:
How did that happen? Well, I graduated with a science degree in 1987. And the thing that I learned after four years of doing a computer science degree is that I do not want to be a developer as my job. I wanted to work with computers. I like computers. I understand computing, I understand coding, I feel I’m a good coder. But I just found it boring. The design part wasn’t enough of the job for me. It was mostly a typing exercise. And so I was not overly thrilled with the idea of being a coder or a developer for my career. So I became an applications engineer at Xerox working on high-speed, laser printers. I did pre and post-sales support. So I needed to use my technical skills, but I wasn’t coding as a primary job. I did some coding on the side.
And then I became a pilot in the Canadian military as a complete tangent off of what I was doing. And I flew, seeking helicopters for a couple of years. So five years in the military full-time through training to be a pilot, and then about two years flying seeking helicopters off the back of ships. And then in 1995, I got out of the military and was just about to marry my current wife. And I reached out to old friends in high tech and one of them said that he had mentioned my name to a friend of his who was a director at a company called Newbridge. And that person, his name was [Gee], asked my friend, Sean, “What’s Paul doing?” And he said, “Oh, he’s just getting out of the military. He’s looking for a job.” And Gee said, “Hey, I have a job for him. Send him to me”. So without even applying, I ended up going in talking with Gee who was interestingly a friend of my brothers, so I actually knew him. And the job interview when something like, “The job that I want you to do is this,” and he described it, and he said, “I need three things from you. You need to know Unix,” and I said, “I don’t know Unix,” and he said, “You can learn.” He said, “You need to know telecom,” and I said, “I don’t know telecom,” and he said, “It’s okay, you can learn. And I need you to be able to learn. And I know you can do that because you were a pilot in the military.” So he hired me on a three-month contract to do some testing. And then at the end of that, I got flipped to full-time. And I’ve been in testing ever since. So that’s how I started as a tester. It was because a friend of mine stopped in the hallway to talk to Gee because I had told him a funny story about Gee which I’m not going to repeat here. But he didn’t think it was true. So he told to Gee whereas he passed him in the hallway at work. And he said, “I just heard from Paul this story,” and Gee laughed and said, “Yes, that’s true. What’s Paul doing?” And that was how that part started. So it was because I told my friend Sean a funny story about Gee that I’m a software tester.
Federico:
And so, being a pilot was part of the decision of Gee, so it influenced the final.
Paul:
I did later find out, once I was no longer reporting to Gee, that he had actually applied to be a pilot in the Canadian military and didn’t make it through the selection process. And Gee is a very smart guy so he, I guess, made the decision that because I made it through, that I must be smart or able to learn or something.
Federico:
“I want to really assess that,” right?
Paul:
Exactly. So that ended up great, that Gee had said, “I want to be a pilot,” and then went through the selection process and didn’t make it. So that helps.
Federico:
Okay, perfect. Also now, as part of your career in software testing, you’re also a rapid software testing trainer, right?
Paul:
I am. So yeah, that’s a fairly long story. I’ll try to hit just the highlights. So when I first became a test manager in 2000, I realized that nobody at my company had received any training on being a tester. It was all essentially, “Oh, you know something about computers, we need testers, you’re a tester.” And the vast majority went from being a tester and then they moved into being a software developer. This particular company used testing as a stepping stone. You would go in, write automation, learn the systems and then move into development. And I was asked, “When do you want to move into development?” And I said, “I don’t.” I like being a tester. And they were, “What? Why wouldn’t you want to be a developer?” And I said, “Because I enjoy testing too much. It’s far more interesting to me.” And I ended up being the first test manager. And I looked for somebody who could do reasonable testing education that wasn’t something like the certification training because just at a glance, I could see that that wasn’t overly helpful. So I found a gentleman named Ross Collard who was likely if I looked back at my career, the single best phone call I have ever made. Ross agreed to do the training, we became friends. He modified his training to be pertinent to our teams.
And about a year and a half after he had provided the training, he provided two or three courses for testing for us. And then he contacted me to say, “I’m hosting this workshop on performance and reliability testing, Whopper.” And I was invited to go to the first one. It was a pure conference. I had never heard of the lost style, the Los Altos workshop on software testing, style of pure conferences. It was intriguing. And he was saying, “It’s invitation only.” And I asked my boss if I could apply and then he said yes. So I applied, and much to my surprise, I got accepted. So I went to the first Whopper. And the facilitator was James [Bach].
So James and I met in October 2003 in New York City at the first Whopper conference. And that went fine. I was completely overwhelmed and out of my league. There was Rob [Sadler] was there. There was a guy named Roland [Stents] from Vancouver who was fairly well known. And Scott Barber, Ross Collard. So there was quite a few well-known testers in this conference. And I was fairly new, I’d only been testing at this point for eight years and I felt out of my league.
But the second one was held in May the next year, and James facilitated that one as well. And I got invited back and that time, it felt more comfortable and I actually participated quite a lot. And then in the fall of 2004, there was supposed to be another conference in Chicago and [Kim Caner] was going to be the facilitator. But Hurricane Ivan was going over his house at the time that he was supposed to fly out. And so we didn’t have a facilitator. I volunteered to be a facilitator and I have facilitated almost every Whopper since. We’re up to Whopper 28 and I’ve facilitated all but two of them, from Whopper 3 through Whopper 23, I facilitated all of them and then I had to skip two, and then I went back to being the facilitator.
So all of that to say that’s how I met James Bach. Having met James in that way and becoming friendly with him. I had him come in and teach rapid software testing at Alcatel-Lucent and he did two courses of that and on the second course, I said to him, “I would like to teach rapid software testing.” And I thought he was going to say, “Yeah, so what? Forget it,” because at the time, it was literally just him and Michael Bolton who were the instructors. But he didn’t, he actually said, “I think you’d be an excellent instructor of rapid software testing, here’s what you would need to do.”
And he listed around 16 steps that I would have to do off the top of his head. And so I wrote them all down. And I just started to go through them. And it took me two, maybe three years to go through the entire list. It was a long list and it was a lot of work. It involved co-teaching with him, co-teaching with Michael, going through every single exercise with both of them. There was a bunch of stuff that I had to do.
And once I finished that list, he literally said, “Okay, you can now teach rapid software testing.” And I started to teach it internally at Alcatel. I taught about five or six classes internally at Alcatel. And then after I’d been there for 17 years, they were eliminating my department. And they offered me a good severance package or I could get another job internally. So I took the severance package and became independent and actually started to teach rapid software testing for James. So I did that for a year and a half. And then I moved to New York and started another part of my career.
Federico:
Amazing.
Paul:
I told you it was a long answer.
Federico:
Yeah. And you tried to make it short but-
Paul:
Yeah yeah but-
Federico:
… but you failed.
Paul:
I failed. That was probably a medium-length answer. You can edit out most of that. I met Ross Collard, I became an instructor. There you go.
Federico:
I had the chance to participate in that workshop in Uruguay a couple of years ago when we invited Michael to go to testingUY, the conference in our small country. We had a great time with Michael. It was a lot of things to learn. Even when we went out for beers-
Paul:
The learning never stops.
Federico:
Yeah, it never stops. It’s all the time asking questions and thinking about different things, everything related to software testing. I remember we were hanging out in a bar and Michael said, “Let’s play that set game.” He had the game in his pocket. And it was like another exercise, similar to those in the workshop, right? And it was amazing.
Paul:
Set is great for pattern matching. We also do art shows, the dice game, the pen game, there’s just so many, “Games” that aren’t really games. Plenty of questions is another good one where you come up with a bizarre scenario. And with just 20 questions, yes-no type questions, you have to figure out what happened. So anyway, that’s fine. I won’t give an example. But yeah, there are lots of testing games.
There are some that you can buy in the store that ends up being very applicable to testing. One of them has colored triangles, and I can’t remember the name of it but they’re different-sized colored triangles and it’s kind of like a mastermind but a lot more complicated. And on the dice game, if you think you’ve done the dice game, make up a new rule and do it again. So it’s a never-ending game, you’re only limited by your imagination.
Federico:
So it’s like you could take the workshop more than one time. And it’s worth it because the same exercises are different every time, right?
Paul:
They can be. Yeah. So the way that I would recommend if you are taking RST multiple times is to try to take it from different instructors. Because James, Michael, me, Hibe Schutz is another instructor and Griffin Jones is the fifth instructor. So there’s the five of us, we all teach it differently. I’ve seen everybody do the exercises except for Griffin. Because I just haven’t had an opportunity to watch Griffin step through those exercises. But I do know that all five of us teach very differently even though it’s the same underlying concepts. The way we teach, what we stress is very different based on our own experiences.
So, when I teach it, I tend to focus a lot on the implementation of the theory and how I have now, I’m on my fifth time, implementing it at different companies, the challenges that I faced and things that have helped me, things that had worked at one company, not worked at another company. And so stressing that element of it while going over really the same underlying pillars, I guess, of knowledge. Plus, if you took RST over three years ago I think, it’s very different now. About three years ago, for the first time, James, Michael, and I all co-taught a course. And it was the first time where we were teaching a slightly more agile version of the course, where there’s a product that James and Michael created. And we go through more of the development of that product and testing it as part of how you can approach testing with an RST methodology.
So it is a different course. And it’s always good. Plus, James and Michael have RSTM. So rapid software testing for managers. There’s rapid software testing applied. RSTE, I can’t remember what the E stands for. There are lots of different focuses that you can take. So it works out pretty well. And most of the time, if you take just RST, it’s two or three-day classes, depending which one you… It’s enough to open your mind and open your thought processes, but it is difficult to then apply that to change how your work is doing it. So I would recommend taking multiple classes just to become better at it.
Federico:
Interesting. One of the topics that I remember that is very important in the workshop is about the risks, right? It’s like everything is connected to risks and how testing can help to identify and maybe address the different risks that can be around the approach. Right?
Paul:
Yeah.
Federico:
And I know that you also have a specific workshop for risk analysis, right?
Paul:
Yeah, there was an interesting workshop that James held on Orcas Island in the northwest of the United States where he lives. There was 15 of us who went to this risk analysis workshop. And at one point, at the very beginning of the workshop, James asked, “How many people here know how to do a risk analysis. And I didn’t put up my hand. And James noticed and called me out on it. And he said, “Paul, I know you know how to do a risk analysis.” And at that time, which was only four years ago now, I still thought that there was some magical procedure that everybody should follow when they’re doing a risk analysis.
And I was, I guess, surprised even then, even though I was at the stage of my career where I was, that the risk analysis essentially can be as simple. There are some things you can do. But it can be as simple as brainstorming with other members of your team saying, “What could go wrong?” And identifying things that can go wrong. “There. I’ve now done a risk analysis. What? It’s that easy? Yeah, it’s that easy.” Thinking about what could go wrong in a way that might impact the company, our customers, the integrity of our data, anything like that. And just thinking about, “Well, what if that link goes down? What if people, can they hack us?”
All of these different things that you can think of that can go wrong. And then how can I test to mitigate that risk? So if you’re at risk of people hacking you, then you do more security testing and you look for SQL injections and cross-site scripting and things like that. But there is no magical process. Individual companies might introduce them but I was quite surprised that yeah, I not only knew how to do a risk analysis, but I’ve been doing it for years. And that’s really all there was to it.
And then being able to focus your testing around the risks that you think are the most important. A friend of mine [Jordi Kit], who worked with me at, I guess two companies ago now, [Doren Jones], used to do his test plans by identifying risks. And then he would come up with test charters or test objectives to test each risk. That was how he came up with his testing.
I came up with my testing thinking about the charters first and I never officially identified the risks that the charters were testing for, even though I guess underlying it, there was always a reason or a risk behind every test or charter that I was coming up with. But Jordi would identify them and then say, “Because of this thing that could go wrong, I’m going to execute this test charter,” which for him helped him focus the charter on the particular risks. So, lots of different ways of doing it.
Federico:
Yeah. Maybe this is a specific way to make it more explicit. Because, from my perspective, everything you think as a test or something that you want to test, you are thinking, maybe unconsciously, about a related risk that you want to cover with that.
Paul:
Correct. Why would I be saying test the login procedure for various types of input? So in other words, can I do a SQL injection? Can I enter too many characters? What if I enter one username and someone else’s password, a bug I’ve actually seen, that lets you log in. Things like that. Why am I thinking of testing those things? Because, in the way too long I’ve been testing, 26 years, I’ve seen failures. I’ve seen risks, things that can break. So that’s what I’m doing. But you’re right. I don’t explicitly say, “I’m doing this for that risk,” as Jordi does. But that is underlying, every single thing that I’m doing is a result of a risk that I’m either aware of personally, from personal experience, or a risk that I’ve heard about. Something that I know could go wrong. So, you’re right, it makes it explicit.
Federico:
And maybe for those testers starting their career with less experience, maybe sometimes it’s good to do that connection. Because, for instance, I’m thinking of an example. If you are trying with different characters in an input that receives only numbers, and you say, “Why should I test with characters? Why should I try with this?” An example of usage of a mobile application for a person may be running or doing some activity that under this context, you cannot pay much attention or you cannot be precise on when you are tapping in the form. This increases the chances that you miss the number and put a character. And you don’t want the app to close and lose all the data that you’re already put.
Paul:
A heuristic that I use is, “What could a good user do by mistake?” As you just indicated. I didn’t mean to hit that. But I was walking while I was trying to enter it, and I hit the wrong character and didn’t notice. So what might a good user do by mistake? But also, what might a malicious user do on purpose? I’m not going to accidentally type in the drop table. But a malicious user is going to intentionally try to drop the table. So, the SQL injection, that makes sense to me that you’re thinking both, what might somebody do by mistake and what might somebody else do on purpose?
A bug that I’ve seen in the past is, I thought I was pasting in my username and instead I pasted in three pages of a Word document. I was working on a Unix machine that had a Windows emulator on it that cross when I did a copy in the windows emulator or on the Unix box, they would fill the copy buffer of both. And on the Unix machine, I had highlighted my username, which meant it was in the copy buffer. But before I pasted it, I got interrupted and in the Word document, I cut and paste three pages of the Word document because I was just moving into another section. And then when I went back to the Unix page, my username was highlighted.
So my brain said, “That’s in your copy buffer,” and so I pasted three pages of a Word document into a command-line login, which caused a whole bunch of beeping and upon further investigation when I was logged into the control card, it wasn’t too bad. But when I was logged into a line card on the system, it actually rebooted the system, or the line card itself rebooted. So, “Oh, okay. So something else that started sort of benign now became this.” And so I thought, “Aha, a good user might do this on purpose or by mistake because that’s literally what I just did.” So that’s the type of thing when you say no user would do this, the answer to that is no user that you can think of would do this on purpose.
Federico:
Yeah, totally. But, you mentioned that doing a research analysis is just brainstorming about the possible risks, but maybe it’s my mind of engineer that thinks, “I need a process to know that I’m following the steps and getting to unexpectedly good results.” Right? So is there-
Paul:
Do you want me to come up with a process right now?
Federico:
I don’t know, maybe some guidelines to know that I’m not missing. How do I know that my brainstorming with my peers or with my team is good enough?
Paul:
Well, you will never identify all the risks. There are just too many risks that we don’t know about, too many failures that have not been identified. However, the whole methodology of brainstorming, there’s a pretty intriguing sort of science as to how our brains work behind that. And simple steps that I like to follow in a brainstorming session, is present the issue. In this case, what are the risks that we might face with this feature? Two, everybody in advance of the meeting. Let’s say a day in advance, at least a day in advance of the meeting, and ask everybody to write out the risks that they can think of. So your brain has now consciously tried to identify risks that you’ve thought of for, say 20 minutes, half an hour, you have a set period of time.
For your creativity aspect, having a set period of uninterruptible time with a known start time and a known end time helps your creativity. If you want more information on that, look up anything by John Cleese, the Monty Python comedian. He’s done a huge amount of research into creativity. And he has some videos that are on YouTube. He did a commencement speech at Vimeo which is incredibly insightful. Anyway, so having a known, set, start time and a known, set, end time. You want at least half an hour, probably even longer, where you’re sitting there and just trying to think of something. Then stop thinking about it, go on with your life. And in the meantime, your subconscious is going to keep thinking about it. When you then have the meeting, do a round-robin format where I identify a risk that I’ve thought of and we can have a discussion about it or if somebody has a similar one, they can add to it so that the risks get, I guess, better-defined kind of like in an agile process where you’re trying to define your requirements better.
And then you go to the next one and the next one. And during that entire process, keep your mind open because you will likely hear somebody say something that will make you think of another risk that you didn’t identify. Once, in one of these sessions, somebody with an accent, a fairly unique accent as a matter of fact. He was born in Pakistan but grew up in the middle of England, so he has a Midlands accent but it also has an underlying Pakistani note to it as well. But the way he said a particular word reminded me of a risk that I had thought of. It had nothing to do with what he said. It was just how he pronounced a word that triggered a thought in my mind. And so, I added that to my list and it ended up being a risk that was a good one to add to our risk list.
Also when you’re doing brainstorming there is no… Remember that if you say something even nonsensical. So, birds might not know how to use our product properly because they don’t have fingers, they only have claws and beaks. Okay, that’s a pretty nonsensical thing, but even saying nonsensical things helps create creative responses in other people’s heads. So someone might just, “You said birds. Oh birds, I knew a guy named Larry Bird,” whatever. Who knows what happens in our brains. I’m definitely not suggesting you just say nonsensical things but keeping humor involved and keeping the ability to say nonsensical things without judgment will increase the diversity of the risks that you identify.
I ran a workshop I’d like to test in Sweden on brainstorming. And we, informally obviously, it wasn’t a scientific process. But we had three groups. One group that had to pretend the CEO was in the room and that they were looking to fire people because they were not doing well financially. Another group was allowed to say whatever the heck they wanted. And they were encouraged to say stupid things as well. And then the third group was the same as the second group, but also had a random word generator that showed in the middle of their table. So it would just show random words.
I was really hoping the random word generator one would be better than the second one and that the second and third ones were both going to be better than the first. Unfortunately, the random word generated didn’t noticeably change anything. And very few people commented that they saw a random word and expanded on it. But both, we cycled through with three different scenarios. And every single group said that they did better when they were allowed to say whatever they want without judgment, versus having to only say serious answers and it was noticeably better. 50% to 60% more ideas that came out, that was legitimate. You obviously take the nonsensical ones out.
But that type of brainstorming, and then the other key aspect is you’re not done. You’ve had the brainstorming session, you’ve identified 20 risks just to pick a number, you’re not done. As you’re going through your testing, you’re going to think of new risks. As you’re going through stand-ups, as you’re having a bath at home, you’re going to think of new risks. Write them down, bring them up, get them added to the list, see if you can test to mitigate the risks. Sometimes you can, sometimes you can’t.
But the biggest thing to not miss risks is to keep [inaudible] because your subconscious will continuously be thinking about risks whether you know it or not. And you’ll see something and you’ll say, “Oh jeez, that could happen to us. That’s a risk that we need to identify.”
Federico:
Yeah, the mental process, I guess that continues.
Paul:
Yeah. And I do encourage people to listen to the John Cleese videos because he talks about creativity in a very easy-to-digest and amusing way. He uses his humor and creativity to teach people about creativity.
Federico:
Talking about creativity and how to incentivize people to think in different ways, right? I really like an approach that Valentina in my team has, which is asking, “What would you do if you had a magic wand? What would you change? What could be different?” In order to eliminate your restrictions. Because you typically try to think of things that are possible and you eliminate the things that you say, “No, this is a stupid idea. I don’t have to try to go this path.” Right?
Paul:
Yeah, that’s a good way. Again, yeah. Well, I think it was Jerry Weinberg who said, “Testers know that things can be different.” So, knowing that there’s a lot of the things that people say, “That would never happen, that’s impossible.” Know that they’re quite probably wrong. A really good trick to look like you know what you’re doing as a tester, especially for junior testers who don’t know what’s happening is, in a meeting where you’re going over the architecture, there’s two things you can do. So, one of my favorites is, if you have an architecture diagram that shows things talking to other things and links, just say, “What happens if this link goes down?” And just point to a random link. It doesn’t matter which one.
And regularly, some will say, “Well, that can’t go down. It’s a database connection,” or something, “It can’t go down,” and you’re going, “Really? Well, what if the server lost power? Well, there’s a backup. The backup is not in this diagram. So oh, the diagram isn’t accurate. And what if it goes down in the middle of a transaction? So I’d sent the data but I haven’t gotten the acknowledgement yet and it goes down. What happens to your system as it’s sending that request and not getting an acknowledgment back? What happens? You always get an acknowledgment back. But when you don’t? Well, we always do. But what if you didn’t?” And so that’s the one thing I do.
Another thing I do is just ask the developers, “What’s the worst thing that can happen?” Or, “What are you most scared about that could happen?” Because when you ask them that question, they start to think about negative things instead of what’s going to go right. Because the developer minds… Feelings, yeah. But a developer mindset is, “How am I going to make this work? And how can I show that it does work?” By asking them, “What could go wrong?” Or, “What’s your biggest fear?” They start to think about, “Oh, what might go wrong?” And it gets them thinking because if they’re the ones developing it, they really should be able to identify more risks than you. Because they understand what they’re developing and what things could go wrong. So asking them, “What’s your biggest fear?” Or, “What could go wrong?” Just to try to switch their mindset into a negative one instead of a positivity one helps identify risks, not that you want to work with negative people.
Federico:
Interesting. Another thing. Yes, most of your students when you teach the workshop, they are typically testers. Am I right?
Paul:
Mostly, yeah.
Federico:
Mostly?
Paul:
I’d say probably 75%.
Federico:
Okay, so what’s the typical mistake that testers do when doing a risk-
Paul:
Performance testing
Federico:
… risk Analysis?
Paul:
Wow. That’s a really good question. I don’t know if I have an answer as to what typically happens. I know a lot of people are scared to bring up risks that they think might be construed as impossible or as an example saying, “What if the database doesn’t respond?” Or saying, “What if somebody pastes a million characters into that field?” I did corrupt the database once which was interesting. It was my first day on a three-month contract. And I asked for my own test environment because they were an expensive environment. It was essentially a single server. And it was an aircraft maintenance software. And I asked for my own version of it.
And it was cumbersome and complex. I did a mind map of it and it printed out on 35 sheets of paper. It was huge. It’s by far the biggest mind map I’ve ever made. And the person who I was working for, who knew the system really well, estimated that I was about 60% complete on that 35 sheet mind map. Anyway, it was big and complicated. But there was this one page that had an advanced search on it. And they had no input constraints on any of the fields on the advanced search page. So I put a million characters into six of the fields. And I clicked search, and it didn’t respond right away. And so I clicked search two more times, not realizing when I clicked search the first time that it was actually transmitting six meg to the database, which was then being processed and taking the time to respond.
So I clicked it two more times. By overwhelming the database with 18 megabytes of search data. It actually corrupted the database. And then I got logged out and I thought, “Oh, it recognized that it was under attack and it logged me out.” I was super impressed. It actually didn’t. It just logged me out because the database had been corrupted and it didn’t know what to do so it responded with, “I have a corrupt database. Let’s get all the users off the system.” So I logged back on and everything seemed fine. And I was still impressed. I didn’t know yet.
But then as soon as I touched the database, I got logged out. And I went, “Okay, this is wrong.” So I asked the senior tester, I said, “What’s going on? And he looks and he goes, “You have corrupted the database. I’ve been here five years and I’ve never corrupted the database. You’ve been testing for less than an hour and a half and you’ve corrupted the database.” So again, I know things can be different. I know some idiot like me is going to try to do it. Could you imagine that an aircraft maintenance software that techs who fix aircraft go in, they want the afternoon off? If the software is down, they’re going to get the afternoon off.
If they knew that back door to just corrupt the database. All maintenance would stop, they would get the afternoon off while they restored the database. Sounds like a really bad bug to me. Anyway, it was an easy fix. All you need to do is put input constraints on the input and obviously, on the back end, put some-
Federico:
Put some gates, yeah. Just in case some user learns [crosstalk].
Paul:
Yeah, exactly. But at least have input constraints. I wasn’t doing anything tricky. I just put in a million characters into six fields, anyway.
Federico:
Oh well, man, I think we can continue talking about risks for hours. I have a couple of final questions. One is if you have any recommendations about productivity hacks. If you have some habits to suggest people to form?
Paul:
Stop writing test cases, stop following scripts. Might be controversial. Yeah. And if you’re not allowed to do that, and I understand that. There’s quite a few places that you have to have your scripted tests and they have to be executed. And they’re all tracked in a test management system. If you want to move towards exploratory testing, which is a good way of finding more bugs. Just do some exploratory testing on your own time. Stay an hour late at work, test through lunch hour, skip a meeting that you know you don’t need to go to and do testing during that, and just try poking around.
And if somebody says, “How did you find that bug? What script were you executing?” And you can say, “I was just doing some exploratory testing,” you will likely, hopefully, maybe better than likely, enlighten them that the way they’re doing testing is potentially not as efficient as they would hope. There’s a lot of words that you would need ready because people say, “If you say we should stop doing the scripts, but we need to do the scripts because we need to know the regression is working and I don’t have time to go into it now.” But being able to talk intelligently about that and saying, “Well, if we’re testing the scenarios or if we have automation that tests those basic scripts, we don’t need the people to be doing the same thing, the people can be doing different testing and burying it. Anything that’s scripted, why don’t you just automate it and have the testers use their brains and take the responsibility for their testing?” It’s not really a hack. But that’s what I got for now.
Federico:
That works. And what about books? Do you have any books to recommend?
Paul:
Lessons Learned in Software Testing by Kaner, Bach and Pettichord is a very good book. It also is good if you have attention deficit disorder like I do. Because each lesson is very short. So you can get through the book without getting bored or distracted. It’s a whole bunch of short little books. Perfect Software by Jerry Weinberg. Your Brain at Work by David Rock, I have found interesting. It’s not about testing, but it definitely helps figure out how your brain is working. It’s a collection of research that he compiled into a more easily on the brain, that he compiled into a fairly easy-to-digest book.
It’s one of the few books I’ve read twice. Thinking, Fast and Slow by Daniel Kahneman. Again, not a testing book, but a book about your brain and being aware. Most of the books about your brain don’t teach you how to fix or workaround limitations of your brain, but just being aware of them and what the limitations are, helps you plan your day so that you’re spending the good part of your day that your thinking is at its peak doing thinking stuff. In other words, if you get to work and the first thing you do is go through emails for an hour, that’s likely the most productive brain time you have and wasting it on emails is not a good idea. So again, suggestions like that come out of these types of books. Pretty much any book by Jerry Weinberg is good. General Systems Thinking. The-
Federico:
The Secrets of Consulting, have you read that one?
Paul:
Secrets of Consulting, yeah. And More Secrets of Consulting. Yeah, he’s written 20 books and they’re all pretty amazing. But yeah, those are the books that I would recommend.
Federico:
Perfect. And the last thing, do you want to invite people to something in particular? Follow you on social media?
Paul:
You want to follow me on social media, my Twitter handle is paulholland_twn as in the whole nation. I do have a web page, but I haven’t updated it in years. But it’s testingthoughts.com. There’s a couple of blogs up there which are kind of interesting. And I’m going to be at Agile Testing Days in November doing a tutorial on how to talk about testing.
Federico:
In-person?
Paul:
In-person. Well, we hope. I’m fully vaccinated. I’ve done my civic duty, get vaccinated.
Federico:
Paul, thank you so much for your time, for all your thoughts, for everything you’ve shared.
Paul:
It was fun.
Federico:
I really appreciate that. Yeah, it was fun.
Paul:
Thanks for inviting me.
Federico:
See you soon. Bye-bye.
Paul:
Take care. Bye.
3 Keys to a Successful Continuous Testing Implementation
Unless you’ve been hibernating for the past decade or two, I’m sure you have already become well aware of the benefits of continuous testing: reduced cost of development, less waste, improved system reliability, reduced risk upon release, and so on. Of course, putting it into…
Quality Sense Podcast: Refael Botbol – Optimizing Performance Testing Costs
Interviewing a leading expert in performance engineering and open source tools Recently for his Quality Sense Podcast, Federico Toledo interviewed Refael Botbol, the BlazeMeter testing domain expert at Broadcom, where he enables developers to achieve higher-quality applications by injecting testing throughout the software development lifecycle….
Abstracta
