How we used SIPp for VoIP performance testing over the SIP protocol

Not too long ago, some of my colleagues and I helped to run performance tests for a new mobile app with a hotline called “Yo Te Escucho” (which translates in English to “I hear you”). It was created during the COVID-19 crisis as a non-profit, voluntary initiative in Uruguay. The hotline connects anyone who wants to simply speak and be heard (safely and anonymously) by a trained person. The main objective is to be able to provide people with a channel of communication and empathetic listening. Sparked as a response to the negative impacts of the virus, the initiative is expected to continue in the long-term, even when it’s no longer around.

In this post, we want to share how we went about performance testing the mobile app’s VoIP technology as well as everything we learned about the tool, SIPp, for performance tests over the SIP protocol (which is basically known as VoIP, Voice Over the Internet). We’ll go through the basics of the protocols involved and then see how to use the SIPp tool to simulate concurrent calls on the system. All this of course, from the experience of a real volunteer project, which makes it much more interesting! 

Thanks to my colleagues, Pablo Richieri, who also worked on the project, for helping me write this post and to Andréi Guchin for leading the effort. 

Project Background

Our role in this project was to conduct performance tests on the mobile app in order to observe the system’s performance when it experiences the maximum number of currrencurent calls that it was designed to support. 

The hotline application has two types of users: those who make the calls (‘callers’) and those who are available to answer them (‘listeners’), using the SIP protocol to establish communication. 

We soon learned that the typical tools we use for performance testing such as JMeter or Gatling don’t support this protocol by default. So, we had to investigate what the SIP protocol consists of, how to capture it and which tool we could use to automate the flow of the call from both the caller and listener side. We achieved this with the SIPp tool, which was recommended by 3CX, the VoIP service provider for the hotline.

What is VoIP?

The VoIP (Voice Over IP) protocol is used to make and transmit telephone calls over an IP network. Its goal is to packetize audio streams in order to transport them.

Most of these calls use the SIP protocol (Session Initiation Protocol), either through the use of physical telephones as well as softphones.

Physical SIP phones look a lot like traditional phones, however they’re developed for use on IP networks. (Think of the clunky phone you typically see on everyone’s desk in an office!)

Softphones on the other hand, are software applications used on computers, tablets or smartphones and behave exactly like a normal IP phone, allowing to perform calls as well as additional features like video conferencing, presentations, etc.

What is SIP – Session Initiation Protocol?

SIP is a protocol used to establish a “session” between two or more participants, modify it and eventually terminate the session. It’s found its greatest use in the world of IP telephony. This protocol bears a significant resemblance to the HTTP protocol. The messages are text-based and the request-response mechanism makes error resolution very easy.

We have two types of SIP messages, requests and status messages. Among the most common requests are: REGISTER, INVITE, ACK (acknowledge), INFO, NOTIFY, UPDATE, BYE, etc. And in the states we will find response codes in the range between 100 and 600, for example: 100 (Trying), 180 (Ringing), 200 (OK), 407 (You need to authenticate), etc.

SIP messages describe the identity of the participants in a call and how the participants can be reached over an IP network. Once the exchange of configuration messages is complete, communication can be exchanged using another protocol, typically RTP (Real-Time Transmission Protocol).

SIP was developed by the IETF and published as RFC 3261, and its flexibility has made it possible to almost completely replace the H.323 protocol in the VoIP world.

How to Capture Traffic with the SIP Protocol?

The tool we recommend to capture this protocol is Wireshark, a protocol analyzer that allows you to see all the traffic that passes through a network and filter it by protocols.

The different sections that we will find in Wireshark that are very useful when it comes to automating the flow are shown below:

1. In this section you can filter both by protocols and by IPs
2. The IP from which the messages are sent is displayed
3. The IP that receives the messages is displayed
4. In this section you can see what protocol is being used
5. In this section we can see if the message is a request or a status message (status)
6. By selecting one of the captured packets, we can view in this section the information regarding the protocols of each of the network layers. If we select “Session Initiation Protocol” in “Message Headers” we will be able to see all the SIP protocol headers that are sent in that message that we will then use to automate the call.

What is SIPp? 

SIPp is an Open Source tool that allows us to generate one or more calls using the SIP protocol, emulating both clients and servers.

By default it provides some basic scenarios that are ready to run, but it also allows us to import our own scenarios that we will create in XML format.

In these scenarios we will describe call flows from the simplest to the most complex, being able to regulate the number of calls per period of time, the use of authentications, sending audio or video using the RTP protocol, among others.

In this link, you can download SIPp and find its documentation, which easily explains how to install it (we recommend doing it in Linux).

Using SIPp 

If you want to use the tool, all you have to do is perform the following steps: 

To begin, it’s run from the command line using “sipp”.

# ./sipp

Then we indicate the IP address to which our calls will be directed.

# ./sipp 127.0.0.1

And finally the “-sf” flag is used to execute a scenario created by us or “-sn” to use one of those already included, followed by the scenario file that we are going to execute.

# ./sipp 127.0.0.1 -sf my_scenario.xml

With this we could already execute a simple scenario without problems, but we have a great variety of options that will allow us to execute more complex scenarios.  

Next we will see some of these options that we believe will be very useful when making your scenarios more robust. 

-h: Shows help information on the screen

-aa: Activates an automatic response with the code “200 OK” for all INFO, UPDATE and NOTIFY requests. This is useful to be able to avoid all these messages that generally do not contribute much to the user, they only inform.

-m: Determines the maximum number of calls to execute and stops the test when that number of calls is processed

-r: Configures the number of calls per time period that we want to generate. By default, 10 calls are generated per second. This setting is dynamically editable during the test using the following keys:

• “+” increases calls per time period by 
• “-” decreases calls by 1 per time period
• “*” Increases calls by 10 per time period
• “/” Decreases calls by 10 per time period 

-rp: In addition to the previous flag, this specifies the period of time (in milliseconds) in which this number of calls will be generated (set by default at 1000 = 1 second). 

-inf: Allows us to import files, for example, CSV files, and work with their data in our scripts. Later when we see how to generate our scenarios we will look at this option in more detail.

-oocsf: SIPp uses a single “call-id” for each execution of the automated flow. This means that if different communications are automated in the same script, the tool will use the same call-id for both. In the case of this application, all communications (calls, listener registration on the server, etc.) are made with a different call-id. This flag allows us, whenever SIPp receives a request with a “call-id” different from the initial one, to execute a second particular scenario to resolve this second communication. To use this functionality but with the scenarios already integrated in the tool we will use the “-oocsn” flag.

Scenarios in SIPp

Scenarios in SIPp are files in XML format. They have the following format:

Next we will see some of the commands that we can place in our script.

Commands

These are the most used tags:

<send> – Used to send requests

<recv> – Used to receive requests

<pause> – Generates pauses of a certain duration (in milliseconds)

<label> – Allows us to identify a specific part of our scenario and then if we want to be able to go directly to that request if any condition is met

<nop> – Has no effect at the SIP protocol level, but allows us to perform specific actions that we will see below

For a complete and more detailed list of these commands with their use examples, see here. 

Actions

In any instance of a “recv” or “nop” command we can use actions of different types, among them we have: the use of regular expressions, play audio or video, assign values ​​to variables, jump to another point on the scenario, etc.

For a complete and more detailed list of these actions with their use examples, see here. 

Keywords

When we put together our messages, SIPp provides us with very useful keywords that are used to tell SIPp that it has to do something with them. Among the most used are:

[remote_ip] and [remote_port] – IP and port values ​​with which we want to communicate

[local_ip] and [local_port] – Values ​​of our IP and local port

[call_id] – Saves the value of the call identifier

[authentication] – Creates authentication header (We will see examples later)

[fieldX] – Uses values ​​obtained from external files (We will see examples later)

The following figure shows a typical SIP request. The first block of information represents all of our headers. Then, separated by a blank line, we find at the end of the request, the block that contains the information that belongs to the body. 

In the headers, we find information about the communication itself, such as: the VIA that we are going to use, who sends (FROM) and who receives the message (TO), what is the CALL ID, etc. And in the body, we usually find the configuration of what the content of that call will be, such as: where it will be sent from, what type it will be (audio or video), what format it will have, etc.

In both the headers and the body we can see examples of the use of the keywords we mentioned above.

On the other hand, a typical status message in SIP looks like this. 

You will notice that the keywords change from those in our previous order message. Now the keywords are preceded by the word “last”. This is because all the status messages are the consequence of a previous request and share the vast majority of their headers with that request. SIPp allows us to place this word “last” to access the last known configuration of that header (in the example it would be the one received in the INVITE request) and reuse it in this header.

For a complete and more detailed list of these keywords with their usage examples see here. 

Results in SIPp

Now we’ll go over what we will see on screen when executing SIPp, using the following image as a reference:

1. This is the script configuration
2. This is the flow that the call will have. The arrows indicate whether the message is sent or received
3. This is the status of each message (Correct, retransmissions or errors)
4. These are the warnings or errors that occurred during the execution, if any

NOTE: Retransmissions occur when a message does not receive a response in a period of time set by the user (by default 500 milliseconds) or when the response received is not what is expected.

Other Elements Used

In our testing project for Yo Te Escucho, we had to make use of other elements that we’ll also explain below.

Variables

In more complex scenarios, we will need to store information that will be reused in other parts of the script or even in other calls. To do this, we will use variables. 

In this particular project, we used variables for the different users and defined them as follows, 

User is the name of the variable and 002 its value. To make use of this variable, you can call it: [$ user]

Regular Expressions

The use of regular expressions in SIPp allows us to obtain and store specific information about a message to be able to use it later.

If you do not know the basics of regular expressions, we recommend this website, which explains how they work through exercises.

As shown below, the use of the “ereg” action in SIPp requires several data:

regex – the regular expression

search_in – the place where we will look for that regular expression

header – name of the header, we will only put it when we search in a header

check_it – If the expression is not found, it marks the call as failed

assign_to – name of the variable where we will store the found value

In this case, we looked for the regular expression “;. *” to be found in the “hdr” (header) that has the name “To”. It will not mark the call as failed if it does not find that expression (“false”) and if it finds it, it will save its value in a variable named “3”.

For more information on the use of regular expressions in SIPp, go to this link.

CSV files

To insert values in​​to the scenarios from a CSV file, the following command can be used in the console: -inf file.csv

The first line of the file must specify how the data will be read:

• SEQUENTIAL: If we want it to be read sequentially
• RANDOM: If we want them to be read randomly

Each line corresponds to a call and is delimited with “;”. Wherever in the script the keyword “[field0]” appears, it will be replaced by the value “042” on the first call, “043” on the second call, and so on. Upon reaching the end of the file, SIPp will start reading the file from the beginning.

For more information on this topic, visit this link.

Audio Files

SIPp allows us to send audio or video files which is done by simply using the action “rtp_stream” as shown below.

After the name of the file to be played, we can place a “,” and a number that will indicate the number of times that file will be played (by default it is 1 and if we put -1 it will play it without stopping).

On the other hand, it is somewhat complex to meet the requirements regarding the configuration of the audio file so that SIPp can play it. This website is very useful if you need to transform audio files to different formats so that they can be played correctly in the call (we recommend using the “a-law” option).

To know more about how to send audio or video using SIPp see the following link.

Authentication

SIPp supports two types of authentication algorithms, MD5 and AKA. This authentication is easily done using SIPp. When we receive a 401 or 407 message with an authentication request, we must set the auth = ”true” configuration as shown below, with this SIPp will know where to obtain the necessary information to perform the algorithms.

<recv response = “407” auth = “true” />

In the next request, we will be able to send between our headers the generated authentication using the keyword [authentication]. In our case, we had to authenticate against the server using MD5, adding the username and password values ​​in the keyword as follows.

With this, SIPp will automatically generate the header with all the necessary information to authenticate against the server.

Next and Label

It’s possible to execute a scenario in a non-sequential way, being able to jump from one part of the scenario to another when some condition is met.

For this we will use the “label” command that will indicate a reference point in our scenario and the “next” parameter that placed in the command of our choice will cause that when this is fulfilled the scenario will jump to the point where the indicated label is located.

In the following image, we see an example of a call that is made and then waits for a 486 busy call code that may or may not arrive, that is why the “optional” parameter is activated.

When the busy call code is received, the scenario returns to the “busy” label, as indicated by the “next” parameter and retries the call until it is able to communicate.

Final Thoughts on SIPp for VoIP Performance Testing

This VoIP performance testing project was a challenge that involved a lot of learning, but it was also very motivating, due to what we could learn and the fact that it meant we could help create an amazing tool for those in need. 

SIPp is a very powerful and simple tool to use. Making good use of keywords takes care of practically all the script configuration, leaving us in charge of building the call flow. It’s important to point out how it lets you bring information from external files for more versatility when setting parameters and that you can transmit audio and video files to make the call as real as possible.

On the other hand, we believe that it does have some room for improvement. SIPp has a GUI that tries to make script assembly easy. At the outset, we didn’t find it intuitive, since we didn’t already know the logic behind the assembly of the scripts. For this reason, we chose to write the scripts by hand. If you have prior knowledge of the SIP protocol, the interface can be beneficial to you. Otherwise, you have to read the documentation that explains the use of this GUI or the SIPp documentation to be able to use it correctly. 

Another thing that we do like about it though is the possibility of working with more than one call-id per script. This greatly facilitates the work in cases like ours, in which the user registration and the communication itself are carried out in two different call ids.

I should also mention that we didn’t go into complete detail in this post about all of SIPp’s functionalities (as we only used some of them), but they should be considered if you are going to evaluate SIPp’s capabilities for yourself! 

Want to try it for yourself? Here you can find the repository with the scripts that we used for this VoIP performance testing project so that you can use them as a reference!

Recommended for You

Load Testing a Video Streaming Service for 85,000 Concurrent Viewers
How to Make a Performance Test Plan