Azure Load Testing is a fully managed load testing service that uses JMeter under the hood. It’s really easy to get started, and in this post I’m going to show how we can load test a simple App Service instance.
Create an Azure App Service Instance
Azure Load Testing allows us to load test any arbitrary URL but it might be a little rude to test someone else’s page so we’ll create a simple app service instance and just keep the default page.
Additionally when we are load testing an Azure resource we can capture server side metrics such as memory and CPU usage (RU consumption for Cosmos etc.) as opposed to just client side metrics like response time, this is where the real power of Azure Load Testing comes from IMHO.
Search for ‘app services’ in the portal search bar and then click the ‘+ Create’ button to get to the below screen…
I won’t go into too much detail here as we just want a simple URL so you can pretty much configure the app service as you please. For the tests below I’ve created one in North Europe on the P1V2 plan with .NET 6 as the runtime stack.
It’s a good idea to add the app service into a new resource group into which we’ll also put our Azure Loading Testing instance and tests so we can easily delete all resources after we are done. Note Azure Load Testing is not a free service.
After you’ve created the app service, note the URL for the next step.
Create an Azure Load Testing instance
Search for ‘azure load testing’ and click on the result to get to the listing page…
… click ‘+ Create’ to open the ‘Create a load testing resource’ page. Fill in your details such as resource group and location. Note… only a limited number of locations are currently supported.
Location does not have to match the location of the app service created earlier. Similarly the resource group selected below does not have to match that of the app service created above, but as mentioned it might be useful to put everything into a temp resource group so you can delete it all afterwards if you wish.
After your loading testing resource is created you can now create a test.
Create an Azure Load Testing Test
We have two options to create a test in the Azure Load Testing service. One is to upload a JMeter script and the other is to create a ‘quick’ test using a URL. In this case we are just going to create a quick test and plug the app service URL from earlier into the setup…
After adding the app service URL into the ‘Test URL’ box click ‘Run test’. Since we can’t track server metrics without having first run one test we’ll actually be ignoring the first run.
After the test run is complete, navigate back into the test run listing page (which will have only run one at this stage) and click the Configure option and then ‘App components’…
This will bring up the Configure app components screen from which you will select the app service you created earlier (or a DB, queue etc.). After adding it the Metrics button in the Configure option (shown disabled above) will become enabled. It is from here you can choose to monitor things like CPU, memory, data in, data out, thread counts etc.
Note… the particular metrics available to monitor will depend on the service.
In this case I’ve added the CPU Time and Average memory working set metrics. Note, Requests and Response Time are defaults added by Azure.
Run two tests (50 and 200 virtual users)
At this stage we can run tests as everything we want to track is now configured. When on the test run listing page, click Run in the menu bar to open the below pop out screen…
After entering a helpful description and the number of virtual users (threads_per_engine) click the Run button at the bottom on the dialogue.
For this post I’ve ran two tests against my app service, one with 50 users and one with 200 users. Wait until one run is complete before starting the next to ensure the two runs don’t interfere with each other.
Comparing results of two test runs
Comparing the results of two test runs is easy. In the test listing page simply select two tests and click compare. Now lets look and see what some of the graphs look like…
In the client side metric section we can see response time graph shows us the response time for the 200 users (orange) test hovered mostly between 0.5 and 1ms. The test with only 50 users never had a client side response time above 0.5ms.
The response time chart on the server side isn’t very exciting but shows again the much shorter response time when the app service is hit with 50 virtual users rather than 200 (orange).
Since I also added metrics for CPU Time and Average Working Memory Set they are also shown in the comparison…
The graphs above aren’t very exciting as all we are doing is hitting a static HTML only page.
If we had a normal application hooking into an Azure hosted resource such as a Cosmos DB for example we could also get server side metrics for it across multiple test runs to try and determine where our apps breaking point would be by gradually increasing the number of virtual users we test with.
Alternatively we can of course keep the virtual user count consistent across multiple tests while we deploy performance related changes to our app to see how things improve.
The Azure Load Testing service is still in preview but it has a huge amount of potential, so I would definitely encourage you to check it out.
