Finding new solutions for testing mobile performance on the client side

In today’s Quality Sense episode, Federico Toledo sits down for a chat with a colleague and friend, Sofia Palamarchuk. She’s a Director and Board Member of Abstracta and the co-founder and CEO of Apptim, a tool that helps you to test and analyze native mobile app performance.

After beginning her career as a performance engineer at Abstracta, she led our expansion to the United States – heading up business development. After seeing the challenges that mobile development teams face, in 2019, she embarked on a mission to transform the way global mobile teams create quality apps.

When she’s not busy leading a startup, you can find her kite surfing or supporting projects that promote more women in technology.

What to Expect?

• Sofia shared her journey from tester to Abstracta CEO, Board Member and then Apptim co-founder
• The two discussed what affects mobile app performance (server side, network and client side)
• Challenges in native mobile apps (Android and iOS) to monitor and gather information to optimize their performance
• Typical types of issues in native apps (like memory leaks, rendering time, excessive CPU and battery usage)
• Observability for mobile apps
• A book not about testing, but one that everyone should read

Listen Here:

• Listen on Soundcloud
• Listen on Spotify
• Listen on Apple Podcasts

Episodes Transcript:

(Lightly edited for clarity.)

Federico:

Hello, Sofia. Nice to see you again. And it’s an honor, a pleasure for me to have you here in the show. Welcome.

Sofia:

Thank you Fede, the honor is mine. I’ve been following your podcast for some time. I’m happy to be here.

Federico:

Oh, wow. I really appreciate that.

Can you start by giving us a summary of your career in testing, because I find it really interesting (from many points of view)?

Sofia:

Okay. It’s a really fun story, but I’m going to try to do my best to summarize it.

I started working in performance testing specifically.

I actually never did any functional testing, which is something that many people can’t believe, but I learned a lot.

After my first years in performance testing, I moved to more like pre-sales engineering roles at Abstracta… which is a company that you are very familiar with… because you co-founded it 🙂

Federico:

Yeah. It rings a bell.

Sofia:

Yeah. I think you might have heard of that one.

So, I was in a technical role working with clients and then I changed to what many people say is the dark side: sales, but I really like it!

Just being in direct contact with customers.

And that’s when I learned about the other part of testing, actually understanding the needs, where it makes sense to add more testing and coming up with strategies.

So I really liked that role and fast forward to the present… I’m working right now on a new startup, which is Apptim. 

So my current role is understanding what are the needs in mobile testing for today’s teams and we’re building tools to help them with these challenges.

I’ve been kind of moving around different roles, but everything related to software testing.

Federico:

And I can say that we have been sharing a lot of time working together in different projects, specifically at the beginning.

As you mentioned, we were doing some performance testing specifically on the server side and now you’re working mainly on the client side for the mobile devices. 

So my question is, are there problems in the client side aspect of mobile apps?

As performance testers, we typically focus on the server side, why should we pay attention to the client side?

Sofia:

Yeah, that’s a great question. And I asked this question myself as well, because that’s where I was coming from. And I never thought about this. 

If you asked me a couple of years ago, what was the most important? I was trying to push for more performance testing early in the development cycle, but focusing on services, backend APIs and what most people are familiar with when we talk about performance testing. 

You also have the network, which is something that must be taken into consideration. Although it’s not in our power as testers to know where our users are going to be connected to, or what will be the quality of the network. I think we should include that in our test.

The client side performance is very important because it’s directly associated with the user experience, which is something that many people talk about.

And it’s becoming really important for companies to manage their digital channels and how they interact with their customers through mobile apps, which are growing really fast every year. 

So that’s why four years ago, we started thinking about the mobile client side, what were the tools available, why this wasn’t something that was being tested early on as well. 

So a lot of monitoring tools in the market today, analytics, crashlytics, in production. 

But what about during development or testing? Do we know what the performance of our app is in real devices?

So starting to think about that and how we can actually help teams from a testing perspective to include this type of metrics and tests early on in their development cycle.

Federico:

And what are the typical issues that we may face in the client side, in the native applications?

Sofia:

Yeah, exactly. I was going to talk about that. So you have different types of apps. We have native apps, which are built directly, two platforms today on Android and iOS.

Then you have hybrid apps which actually have web use embedded on the native app. They have both web and native, and then you have fully web mobile apps, which is basically opening the browser on your mobile device and you can navigate the website.

Federico:

I have to clarify because maybe my question was specifically for native applications, because when you have a website, it doesn’t matter that much, if you are accessing from a mobile device or from a browser, because there are tools, developers tools or some things like this that allow you to access and gather a lot of information regarding the performance of the web application. 

But my question was specifically for native applications where we have maybe less options, right?

Sofia:

Yeah. So you have less options and you have more challenges and the reason why is because first there are a lot of devices on the market today, (they’re growing 20% every year) which makes it really hard for a development or testing team to have coverage of all the devices that they need to test.

Cell phones vary in OS, in screen size, in hardware and all that affects the performance of a native app. 

It’s a big challenge in terms of cost. And the risk is that you don’t know how your app is going to behave. So that’s why for native apps, we need specific tools to capture data. 

So what type of problems can we identify?

One of them, which is very typical, probably many of our listeners who are mobile users have experienced is when an app freezes. So the screen doesn’t respond and we just close it thinking it’s going to work again. That’s not how it’s supposed to work!

As developers, we have to understand how our app should behave in terms of how fast it will draw different screens and how the experience will be.

So identifying when a screen freezes is one of the problems that we have to test on the client side to figure it out, same for things like memory leaks.

We have to take into consideration something that is very important: Our app is actually installed on a device that’s sharing resources with all the other apps that are installed in the same device, right? 

So we can’t think our app is something isolated and we have unlimited memory and CPU and battery in that device. So one thing that we can actually detect is when there’s a memory leak, which means that the app is not actually releasing memory properly. When it reaches the limit of the memory available in that device and it crashes, right? So these are examples of problems that you can identify when you test a mobile performance on the client side.

Federico:

What you’re mentioning, I see it being very related to a term that I’ve been hearing a lot about lately, which is observability… how easy or how hard it is to gather information about the performance of the native app?

Sofia:

Well, it is hard because as I mentioned, there are two different platforms today in the world, and you need specific tools for each platform.

For Android, you have tools provided by Google for Android developers and for iOS, you have no other option that using the tools provided by Apple and all these tools have a learning curve.

You actually need to understand what you’re measuring in order to gather the data properly. And that’s one of the biggest challenges also for testers, being able to familiarize oneself with both platforms and the tools that are available.

There are tools for debugging purposes or profiling purposes, which are the most common today. And that gives us visibility on how different metrics and how the app actually behaves on the device.

The other big challenge for mobile, which is something that we’re really excited about at Apptim… is that we hope that in the future, we can be one of the first tools to release this; being able to actually measure the exact time that it takes for a user to finish an event, but from the client side.

This will mean I click on a button in the app and I can actually know exactly what happened in the network, rendering time, server times… and see right away where you can know why it’s taking so long and where there is the possibility for optimization. And that’s just really hard for mobile native specifically and something that we’re looking forward to solving.

Federico:

Even the APMs the application performance management tools don’t allow you to gather this information to drill down from an action to the database invocation that imp-

Sofia:

Exactly. Yeah. Some APM tools allow you to see all this data. It’s just really hard to know in mobile when an actual event is finished.

So you can try to see separately, specifically what’s happening throughout the whole test. But if you want to, then let’s say, I want to optimize the login of my app.

It’s just really hard… I’m probably going to be looking at many other requests that are being sent and things are happening, but I’m not going to be only looking at the action of logging in.

So there are some tools that help you capture both, server response times, database times, and the rendering times, but mostly separately.

We want to make it super easy for the user to know and be able to measure by just clicking on a button… How long did it take without using scripts and without using current automation practices.

Federico:

Seems promising, excellent Sofi!

A different type of question I have for you, if you have any book to suggest our listeners to read?

Sofia:

Well, I actually do, and it has nothing to do with performance testing which has been our topic of conversation…

Federico:

I love it.

Sofia:

But it’s a book that I’m reading right now. I’m almost finished, but I feel comfortable sharing it. It’s called Emotional Intelligence and I think the author is Daniel Coleman.

I might have to warn that the book is dense, most of all in the beginning. Why? Because there’s a lot of science on how our brain works related to our emotions.

Then after the first one or two chapters, it just gets really interesting and it has a good balance of theory and practical examples that you can apply in real life.

And it talks about relationships, teamwork, everything regarding emotional intelligence and why it is so important today. So I totally recommend it.

Federico:

Amazing, excellent. Thank you. Is there anything you want to invite the listeners to try or to do?

Sofia:

Of course, if there’s any mobile app developer or mobile app tester in the audience or listening to this, and you’re thinking about mobile performance, maybe this is the first time that you heard about testing on the client side. So you’re already used to measuring performance on the backend. 

I will invite you to go to Apptim website. We have a free tool that you can download. It’s a desktop tool for Windows and Mac, and you can start capturing this performance data when you’re testing your app.

We just want to make it super easy and simple for anyone to start gathering this data on mobile app performance. So I invite you to use it and send us feedback!

Federico:

Excellent. Sofi, thank you so much, have a nice day. Bye. Bye.

Sofia:

Thank you Fede, thank you as well.

Recommended for You

Quality Sense Podcast: Refael Botbol – Optimizing Performance Testing Costs
Quality Sense Podcast: Paul-Henri Pillet – Why We Made Gatling