How to benchmark different versions of the same library with BenchmarkDotNet

If you want to benchmark the performance of two or more versions of the same library against each other you can do this easily using BenchmarkDotNet.

Below I’m comparing the performance of a simple insert for two different versions of Entity Framework…

Click on the image for a larger view in a new tab…

You need to enter the official NuGet package name and version of what you’re trying to benchmark against into the WithNuGet method as shown above.

The WithId method shown above allows you to choose an arbitrary label for display in the results, while the WithBaseline(true) method tells BenchmarkDotNet to present results relative to those of a particular library version. In this case I’ve made EF 6.0.7 the baseline.

Note that versions of libraries you wish to benchmark against cannot be lower than what’s referenced in the benchmark project. In my case EF 6.0.7 is referenced so I can benchmark against this version or higher… BUT not EF 5 for example…

Then assuming your project.cs file has a call to BenchmarkRunner.Run similar to below…

BenchmarkRunner.Run<MulitpleVersions>();

you can go ahead and run your console app without debugging and in release mode to start the benchmarks. When everything is finished you should see output like below…

Click on the image for a larger view in a new tab…

Leave a Reply

Your email address will not be published. Required fields are marked *