This article is an introduction to C# Stress Testing which walks you through a step-by-step, easy-to-follow example that helps you write your first stress test using the NBnech library. Stress testing is an important aspect of software development that helps ensure an application’s stability and reliability under extreme conditions. Stress testing is the process of putting software under heavy loads to identify performance issues and bottlenecks. In this article, we will discuss how to perform stress testing in .NET. In this article, we will use NBench library for stress testing a scenario in a Shop application.
What is Nbench?
Nbench is a benchmarking tool that measures the performance of a computer’s CPU and memory subsystem. It is particularly useful for stress testing because it can put a heavy load on the CPU and memory, simulating a real-world workload. Did you know that we offer a unique online course that boosts your C# career? Check it out here!
Demo Project
In this article, we want to stress test the checkout of a simple shop application. Our demo project contains two classes. Shop and Product.
|
1 2 3 4 5 6 7 8 9 10 11 |
public class Shop { public double Checkout(IEnumerable<Product> products) => products.Sum(p => p.Price); } public record Product(string Name,double price) { public double Price { get; set; } = price; } |
You can create a new class library project with the name Shopand add the classes to follow along. So create a new folder named Shop and run the following command in it to create the production demo project:
|
1 |
dotnet new classlib |
Next, we’ll set up our test project.
Step 1: Create a Test Project
Create a new console application with the name: ShopStressTests. You can create the project by creating a new folder with the name ShopStressTests and running the following command in the terminal in that path.
|
1 |
dotnet new classlib |
We’ll add the shop tests to this app.
Step 2: Add a Reference to the Production App
Next, add a reference to the production application. You can do that by running the following command in your test application:
|
1 |
dotnet add reference ../Shop |
Step 2: Install NBench
To use NBench, you first need to install the NBench NuGet package in the console application. You can do this by running the following command in the test project’s path:
|
1 |
dotnet add package NBench |
Step 3: Create the Test Class
Create a new class with the name ShopStressTests in the test project.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
using NBench; public class ShopStressTests { Counter testCounter; Shop shop; [PerfSetup] public void Setup(BenchmarkContext context) { shop = new(); testCounter = context.GetCounter("CheckoutCounter"); } [PerfBenchmark(Description="Gauge high traffic checkouts", NumberOfIterations = 5, //Number of times this test should run RunMode = RunMode.Throughput, RunTimeMilliseconds = 2000, // How long to run the benchmark TestMode = TestMode.Test)] [CounterThroughputAssertion("CheckoutCounter", MustBe.GreaterThan, 11000000)] //The shop shold be able to perform more than 11000000 checkouts per second [MemoryAssertion(MemoryMetric.TotalBytesAllocated, MustBe.LessThanOrEqualTo, ByteConstants.SixtyFourKb)] // Checking out shouldn't consume more than sixteen kb of memory [GcTotalAssertion(GcMetric.TotalCollections, GcGeneration.Gen2, MustBe.ExactlyEqualTo, 0.0d)] //Checking out shouldn't generate gen 2 allocations public void Checkout_test() { shop.Checkout(new Product[]{ new Product("Speakers",1000) }); testCounter.Increment(); } [PerfCleanup] public void Cleanup() { // shop.Dispose(); } } |
Checkout_test is the test method. The testCounter.Increment() in the method’s body collects performance data. The attributes that decorate the method are also a set of assertions to measure the checkout operation’s performance under the load parameters the attributes specify. I’ve included some comments on what each attribute does.
Step 4: Set the Test Project Up
Add the following line of code to the Program.cs file of the test project to run the test in the main method.
|
1 |
return NBenchRunner.Run<Program>(); |
Step 5: Running the Test
Build the test project by running the following command in the test project’s folder:
|
1 |
dotnet build |
Then run the tests by running the following command:
|
1 |
dotnet run ShopStressTests/bin/Debug/net7.0/ShopStressTests.dll -c release |
Step 6: The Results
This is the result of running the test on my machine:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
Executing Benchmarks in ShopStressTests, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null ------------ STARTING ShopStressTests+Checkout_test ---------- --------------- BEGIN WARMUP --------------- Elapsed: 00:00:00.0106236 TotalBytesAllocated - bytes: 2,212,160.000 ,bytes: /s 208,229,281.132 , ns / bytes: 4.802 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 10,623,674.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 13,826,196.098 , ns / operations: 72.326 --------------- END WARMUP --------------- --------------- BEGIN WARMUP --------------- Elapsed: 00:00:00.0124891 TotalBytesAllocated - bytes: 2,211,976.000 ,bytes: /s 177,111,288.349 , ns / bytes: 5.646 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 12,489,187.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 11,760,973.713 , ns / operations: 85.027 --------------- END WARMUP --------------- --------------- BEGIN WARMUP --------------- Elapsed: 00:00:00.0118637 TotalBytesAllocated - bytes: 2,220,168.000 ,bytes: /s 187,138,079.583 , ns / bytes: 5.344 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 11,863,796.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 12,380,944.514 , ns / operations: 80.769 --------------- END WARMUP --------------- --------------- BEGIN WARMUP --------------- Elapsed: 00:00:00.0112031 TotalBytesAllocated - bytes: 2,220,168.000 ,bytes: /s 198,174,292.348 , ns / bytes: 5.046 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 11,203,108.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 13,111,093.814 , ns / operations: 76.271 --------------- END WARMUP --------------- --------------- BEGIN WARMUP --------------- Elapsed: 00:00:00.0107393 TotalBytesAllocated - bytes: 2,220,168.000 ,bytes: /s 206,731,160.046 , ns / bytes: 4.837 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 10,739,397.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 13,677,211.113 , ns / operations: 73.114 --------------- END WARMUP --------------- --------------- BEGIN RUN --------------- Elapsed: 00:00:00.0117813 TotalBytesAllocated - bytes: 2,220,168.000 ,bytes: /s 188,448,073.540 , ns / bytes: 5.307 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 11,781,325.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 12,467,612.938 , ns / operations: 80.208 --------------- END RUN --------------- --------------- BEGIN RUN --------------- Elapsed: 00:00:00.0119733 TotalBytesAllocated - bytes: 2,220,168.000 ,bytes: /s 185,425,613.959 , ns / bytes: 5.393 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 11,973,362.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 12,267,648.802 , ns / operations: 81.515 --------------- END RUN --------------- --------------- BEGIN RUN --------------- Elapsed: 00:00:00.0114746 TotalBytesAllocated - bytes: 2,220,168.000 ,bytes: /s 193,483,987.274 , ns / bytes: 5.168 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 11,474,686.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 12,800,786.008 , ns / operations: 78.120 --------------- END RUN --------------- --------------- BEGIN RUN --------------- Elapsed: 00:00:00.0109759 TotalBytesAllocated - bytes: 2,220,168.000 ,bytes: /s 202,275,076.203 , ns / bytes: 4.944 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 10,975,984.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 13,382,399.245 , ns / operations: 74.725 --------------- END RUN --------------- --------------- BEGIN RUN --------------- Elapsed: 00:00:00.0119199 TotalBytesAllocated - bytes: 2,220,168.000 ,bytes: /s 186,256,110.961 , ns / bytes: 5.369 TotalCollections [Gen2] - collections: 0.000 ,collections: /s 0.000 , ns / collections: 11,919,974.000 [Counter] CheckoutCounter - operations: 146,885.000 ,operations: /s 12,322,593.992 , ns / operations: 81.152 --------------- END RUN --------------- --------------- RESULTS: ShopStressTests+Checkout_test --------------- Gauge high traffic checkouts --------------- DATA --------------- TotalBytesAllocated: Max: 2,220,168.000 bytes, Average: 2,220,168.000 bytes, Min: 2,220,168.000 bytes, StdDev: 0.000 bytes TotalBytesAllocated: Max / s: 202,275,076.203 bytes, Average / s: 191,177,772.387 bytes, Min / s: 185,425,613.959 bytes, StdDev / s: 6,950,361.729 bytes TotalCollections [Gen2]: Max: 0.000 collections, Average: 0.000 collections, Min: 0.000 collections, StdDev: 0.000 collections TotalCollections [Gen2]: Max / s: 0.000 collections, Average / s: 0.000 collections, Min / s: 0.000 collections, StdDev / s: 0.000 collections [Counter] CheckoutCounter: Max: 146,885.000 operations, Average: 146,885.000 operations, Min: 146,885.000 operations, StdDev: 0.000 operations [Counter] CheckoutCounter: Max / s: 13,382,399.245 operations, Average / s: 12,648,208.197 operations, Min / s: 12,267,648.802 operations, StdDev / s: 459,831.816 operations --------------- ASSERTIONS --------------- [PASS] Expected [Counter] CheckoutCounter to must be greater than 11,000,000.000 operations; actual value was 12,648,208.197 operations. [FAIL] Expected TotalBytesAllocated to must be less than or equal to 65,536.000 bytes; actual value was 2,220,168.000 bytes. [PASS] Expected TotalCollections [Gen2] to must be exactly 0.000 collections; actual value was 0.000 collections. ------------ FINISHED ShopStressTests+Checkout_test ---------- |
Look at the ASSERTIONS section. Two out of the three expectations I specified by the attributes were passed. One of them failed.
Whenever you detect a performance bottleneck in your application consider using the algorithms design techniques to fix the issue. You can find more information in this and this article. You can also learn how to load test your web applications using Apache JMeter in this article.
Conclusion
Stress testing is an important part of software development that allows developers to identify and fix performance issues before they become a problem for end-users. By simulating high-traffic conditions and running benchmarks, developers can ensure that their code performs well under stress and is able to handle the demands of real-world usage. Tools like NBench make it easy to create and run these benchmarks, and by integrating stress testing into your development process, you can ensure that your code is always optimized for performance. If you want to skyrocket your C# career, check out our powerful ASP.NET full-stack web development course that also covers test-driven development.