Testing, Benchmarking & Sanitizers
Write unit tests with GoogleTest, benchmark with Google Benchmark, catch bugs with sanitizers (ASan, UBSan, TSan), and integrate static analysis into your CI/CD pipeline.
The Testing Pyramid for C++
Professional C++ development relies on multiple layers of quality assurance: unit tests verify individual functions and classes, integration tests check component interactions, sanitizers catch memory errors and undefined behavior at runtime, static analysis finds bugs without running the code, and benchmarks ensure performance meets requirements. Together, these tools form a safety net that catches bugs before they reach production.
The two most popular C++ testing frameworks are GoogleTest (from Google, widely used in industry) and Catch2 (header-only, BDD-style syntax). Both integrate well with CMake and CI/CD systems.
Unit Testing with GoogleTest
GoogleTest provides macros for assertions (EXPECT_* for non-fatal, ASSERT_* for fatal), test fixtures for shared setup/teardown, and parameterized tests:
#include <gtest/gtest.h>
#include <vector>
#include <stdexcept>
// Function under test
int factorial(int n) {
if (n < 0) throw std::invalid_argument("negative input");
if (n <= 1) return 1;
return n * factorial(n - 1);
}
// Basic test — TEST(TestSuite, TestName)
TEST(FactorialTest, HandlesZero) {
EXPECT_EQ(factorial(0), 1);
}
TEST(FactorialTest, HandlesPositive) {
EXPECT_EQ(factorial(1), 1);
EXPECT_EQ(factorial(5), 120);
EXPECT_EQ(factorial(10), 3628800);
}
TEST(FactorialTest, ThrowsOnNegative) {
EXPECT_THROW(factorial(-1), std::invalid_argument);
}
// Test fixture — shared setup/teardown
class StackTest : public ::testing::Test {
protected:
std::vector<int> stack;
void SetUp() override {
stack.push_back(1);
stack.push_back(2);
stack.push_back(3);
}
void TearDown() override {
stack.clear();
}
};
TEST_F(StackTest, TopElement) {
EXPECT_EQ(stack.back(), 3);
}
TEST_F(StackTest, PopReducesSize) {
stack.pop_back();
EXPECT_EQ(stack.size(), 2);
EXPECT_EQ(stack.back(), 2);
}
TEST_F(StackTest, IsNotEmpty) {
ASSERT_FALSE(stack.empty()); // fatal — stops test if false
EXPECT_GT(stack.size(), 0u); // non-fatal — continues on failure
}Integrating Tests with CMake
Set up GoogleTest with CMake and run tests via CTest, the built-in test runner:
# CMakeLists.txt for testing
cmake_minimum_required(VERSION 3.20)
project(MyProject LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
# Fetch GoogleTest
include(FetchContent)
FetchContent_Declare(
googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG v1.14.0
)
FetchContent_MakeAvailable(googletest)
# Main library
add_library(mylib src/math.cpp src/utils.cpp)
target_include_directories(mylib PUBLIC include)
# Test executable
enable_testing()
add_executable(unit_tests
tests/test_math.cpp
tests/test_utils.cpp
)
target_link_libraries(unit_tests PRIVATE mylib GTest::gtest_main)
# Register tests with CTest
include(GoogleTest)
gtest_discover_tests(unit_tests)
# Run tests:
# cmake -B build && cmake --build build
# cd build && ctest --output-on-failure --parallel $(nproc)Sanitizers: Runtime Bug Detection
Sanitizers are compiler-instrumented runtime checks that catch entire classes of bugs. Enable them with compiler flags — they add ~2x slowdown but catch bugs that no amount of testing can find without them:
# AddressSanitizer (ASan) — catches memory errors
# Buffer overflows, use-after-free, double-free, memory leaks
cmake -B build-asan \
-DCMAKE_CXX_FLAGS="-fsanitize=address -fno-omit-frame-pointer -g" \
-DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address"
# UndefinedBehaviorSanitizer (UBSan) — catches UB
# Signed overflow, null dereference, misaligned access, shift errors
cmake -B build-ubsan \
-DCMAKE_CXX_FLAGS="-fsanitize=undefined -fno-omit-frame-pointer -g" \
-DCMAKE_EXE_LINKER_FLAGS="-fsanitize=undefined"
# ThreadSanitizer (TSan) — catches data races
# Data races, lock-order inversions, deadlocks
cmake -B build-tsan \
-DCMAKE_CXX_FLAGS="-fsanitize=thread -g" \
-DCMAKE_EXE_LINKER_FLAGS="-fsanitize=thread"
# MemorySanitizer (MSan) — catches uninitialized reads (Clang only)
cmake -B build-msan \
-DCMAKE_CXX_FLAGS="-fsanitize=memory -fno-omit-frame-pointer -g" \
-DCMAKE_EXE_LINKER_FLAGS="-fsanitize=memory"
# Combine ASan + UBSan (commonly done together)
cmake -B build-asan-ubsan \
-DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -g" \
-DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address,undefined"
# IMPORTANT: ASan and TSan cannot be combined — use separate builds
# IMPORTANT: Always compile ALL code with sanitizer flags (including libraries)Static Analysis & Code Coverage
Static analysis examines your code without running it, catching bugs, style violations, and security issues:
clang-tidy — The most powerful C++ linter. Checks for modernization opportunities, performance issues, readability, and bug-prone patterns:
``bash``
clang-tidy src/*.cpp -- -std=c++20 -I include
# Or with CMake:
cmake -B build -DCMAKE_CXX_CLANG_TIDY="clang-tidy;-checks=*,-fuchsia-*"
cppcheck — Complementary to clang-tidy, catches different bug classes:
``bash``
cppcheck --enable=all --std=c++20 src/
Code Coverage — Measure which lines your tests actually execute:
``bash``
cmake -B build-cov -DCMAKE_CXX_FLAGS="--coverage -g -O0"
cmake --build build-cov && cd build-cov && ctest
gcovr --html-details coverage.html -r ../src
Aim for 80%+ line coverage on critical code paths. 100% coverage is rarely worth the effort — focus on testing logic, edge cases, and error handling.
Build a comprehensive testing strategy:
1. Write tests alongside code — not after. Test-driven development (TDD) works well for C++: write a failing test, implement the minimum code to pass, refactor.
2. Run sanitizers in CI — have separate CI jobs for ASan+UBSan and TSan. These catch bugs that tests alone miss.
3. Use test fixtures for setup/teardown of shared resources (database connections, file handles, complex objects).
4. Test edge cases and error paths — null inputs, empty containers, integer overflow, allocation failure.
5. Keep tests fast — unit tests should run in seconds. Move slow tests (network, disk) to integration test suites.
6. Benchmark before optimizing — measure first, then optimize. Don't guess where the bottlenecks are.
Testing implementation, not behavior: Tests that depend on internal data structures break when you refactor. Test the public API and observable behavior, not private methods.
Ignoring sanitizer findings: Sanitizer reports are NOT false positives (with very rare exceptions). Every ASan or UBSan report indicates real undefined behavior that WILL cause problems on some platform or optimization level.
Not testing with optimizations: Some bugs only manifest with -O2 or -O3 (especially UB that the optimizer exploits). Run your test suite with both Debug and Release builds.
Flaky tests: Tests that pass sometimes and fail sometimes are worse than no tests — they erode trust in the test suite. Fix or delete flaky tests immediately.
- GoogleTest provides
EXPECT_*(non-fatal) andASSERT_*(fatal) macros, fixtures, and parameterized tests - Use
gtest_discover_tests()in CMake to auto-register tests with CTest - ASan catches memory errors, UBSan catches undefined behavior, TSan catches data races — run all in CI
- Static analysis (clang-tidy, cppcheck) catches bugs without running the code
- Test behavior, not implementation — and always test edge cases and error paths
- Run tests with both Debug and Release builds to catch optimization-dependent bugs
Quiz — Test Your Knowledge
(15 XP)1. What is the difference between `EXPECT_EQ` and `ASSERT_EQ` in GoogleTest?
2. Which sanitizer detects use-after-free and buffer overflow bugs?
3. Why should you run tests with both Debug and Release builds?