From 25924d61e549fc87be931b5e4d67ab860b733b76 Mon Sep 17 00:00:00 2001 From: Annie Tallund Date: Fri, 29 May 2026 13:00:41 -0700 Subject: [PATCH] Technical review of QuantLib LP --- .../quantlib/1-overview.md | 30 +++- .../quantlib/2-setup-environment.md | 152 ++++++++---------- .../quantlib/3-build-quantlib.md | 25 ++- .../quantlib/4-run-benchmarks.md | 49 +++--- .../quantlib/5-analyze-results.md | 41 ++--- .../quantlib/_index.md | 2 + 6 files changed, 155 insertions(+), 144 deletions(-) diff --git a/content/learning-paths/servers-and-cloud-computing/quantlib/1-overview.md b/content/learning-paths/servers-and-cloud-computing/quantlib/1-overview.md index fd3e25e277..c5db95775c 100644 --- a/content/learning-paths/servers-and-cloud-computing/quantlib/1-overview.md +++ b/content/learning-paths/servers-and-cloud-computing/quantlib/1-overview.md @@ -10,24 +10,22 @@ layout: learningpathall QuantLib is an open-source C++ library for quantitative finance. It provides tools for pricing, modeling, trading, and risk management, and is widely used as both a development library and a representative financial computing workload. -Because QuantLib is a substantial C++ codebase with realistic compute behavior, it is also useful as a benchmark when evaluating cloud systems and processor architectures. - -In this Learning Path, you will build QuantLib from source and run its benchmark executable on an Arm-based Azure Cobalt virtual machine. +Because QuantLib is a substantial C++ codebase with realistic compute behavior, it is also useful as a benchmark when evaluating cloud systems and processor architectures. In this Learning Path, you will build QuantLib from source and run its benchmark executable on an Arm-based Azure Cobalt virtual machine. ## Why use Azure Cobalt? Azure Cobalt provides Arm64 virtual machines for cloud-native development and performance evaluation. Running QuantLib on Azure Cobalt gives you a practical way to measure how a real C++ finance workload behaves on Arm-based cloud infrastructure. -The workflow in this Learning Path uses: +The workflow uses: -- Ubuntu Server 22.04 LTS +- Ubuntu Server 22.04 LTS (also tested on 24.04 LTS) - an Arm64 Azure Cobalt virtual machine - a source build of QuantLib - QuantLib's benchmark executable for repeatable performance testing -## What you'll do +## Benchmark workflow -This Learning Path follows a simple workflow: +The steps follow a practical benchmark flow: 1. Create and connect to an Arm64 Azure Cobalt virtual machine 2. Install the tools needed to build QuantLib @@ -39,6 +37,22 @@ This Learning Path follows a simple workflow: This Learning Path focuses on building and benchmarking QuantLib on Azure Cobalt. It is not a general introduction to quantitative finance or QuantLib development. {{% /notice %}} +## What the benchmark tests + +The benchmark executable runs approximately 85 tests drawn directly from QuantLib's own test suite, covering five domains: + +- **Equity and FX**: American and European option pricing, Heston and Bates model calibration, convertible bonds, Andreasen-Huge volatility interpolation +- **Interest rates**: Short rate models, Bermudan swaptions, Libor market model, piecewise yield curves, overnight indexed swaps, Markov functional models, SABR and ZABR volatility +- **Credit derivatives**: Nth-to-default pricing and credit default swap calibration +- **Energy**: Swing options and virtual power plant pricing +- **Math**: Gaussian quadratures, low-discrepancy sequences, statistics, and special functions + +Each test has a fixed iteration count built in. Some run once per task, others run hundreds or thousands of times to produce a measurable signal. The `--size` argument multiplies the entire set: `--size=2` runs each test twice, `--size=5` runs it five times, and so on. Doubling `--size` doubles runtime while leaving throughput unchanged — this is the expected weak scaling behavior of the benchmark. + +The `--nProc` argument controls the number of worker processes. Because QuantLib is not thread-safe, the benchmark uses separate processes rather than threads, coordinated through Boost IPC. Before timing begins, the benchmark runs every test once through the Boost unit test framework to verify correctness — this is what produces the `*** No errors detected` line in the output. + +**System Throughput** is calculated as `(size × number_of_tests) / total_runtime`. It is the primary metric for comparing runs across thread counts and system configurations. + ## Benchmarking goals When benchmarking a workload such as QuantLib, the goal is not just to obtain one runtime number. You want a repeatable process that lets you compare runs across system sizes, thread counts, software versions, and compiler settings. @@ -48,4 +62,4 @@ For that reason, this Learning Path emphasizes: - using a known VM configuration - keeping the software environment consistent - changing one benchmark variable at a time -- recording commands and results so runs can be reproduced later \ No newline at end of file +- recording commands and results so runs can be reproduced later diff --git a/content/learning-paths/servers-and-cloud-computing/quantlib/2-setup-environment.md b/content/learning-paths/servers-and-cloud-computing/quantlib/2-setup-environment.md index 7fc025b528..4ee7f53127 100644 --- a/content/learning-paths/servers-and-cloud-computing/quantlib/2-setup-environment.md +++ b/content/learning-paths/servers-and-cloud-computing/quantlib/2-setup-environment.md @@ -6,21 +6,24 @@ weight: 3 layout: learningpathall --- -## Create an Arm64 Azure Cobalt virtual machine +## Create and connect to an Arm64 Azure Cobalt virtual machine To run QuantLib on Azure Cobalt, first create an Arm64 Ubuntu virtual machine in the Azure portal. Use the following settings: -- **Virtual machine name:** `quantlib-cobalt-vm` -- **Region:** a Cobalt-supported region such as **West US 2** -- **Availability options:** **No infrastructure redundancy required** -- **Security type:** **Standard** -- **Image:** **Ubuntu Server 22.04 LTS** -- **VM architecture:** **Arm64** -- **Size:** **Standard_D4ps_v5** -- **Authentication type:** **SSH public key** -- **Username:** `azureuser` +| Setting | Value | +|---|---| +| Virtual machine name | `quantlib-cobalt-vm` | +| Region | a Cobalt-supported region such as **West US 2** | +| Availability options | No infrastructure redundancy required | +| Security type | Standard | +| Image | Ubuntu Server 22.04 LTS | +| VM architecture | Arm64 | +| Size | Standard_D4ps_v5 | +| Authentication type | SSH public key | +| SSH public key name | `quantlib-cobalt-vm_key` | +| Username | `azureuser` | For storage, a `64 GB` OS disk is sufficient for this workflow. @@ -28,48 +31,22 @@ For networking, allow inbound SSH on port `22`. Restricting the source to **My I After creating the VM, download the generated private key in `.pem` format if Azure provides one during setup. -## Connect to the virtual machine - -On your local machine, update the permissions on the private key: +Before connecting, update the permissions on the private key from your local machine. SSH refuses to use keys that are readable by other users: ```bash chmod 600 ~/Downloads/quantlib-cobalt-vm_key.pem ``` -Then connect using SSH: +Connect to the VM using the key and the public IP address shown in the Azure portal: ```bash ssh -i ~/Downloads/quantlib-cobalt-vm_key.pem azureuser@ ``` -Replace with the public IP address of your VM. - -### (Optional) Reconnect to cobalt frequently - -If you’ll reconnect often, add a shortcut entry to your SSH config: +Replace `` with the public IP address of your VM. -```bash -nano ~/.ssh/config -``` -Add: - -```bash -Host quantlib-cobalt - HostName - User azureuser - IdentityFile ~/Downloads/quantlib-cobalt-vm_key.pem -``` - -Then connect with: - -```bash -ssh quantlib-cobalt -``` - -## Confirm that the system is Arm64 - -After logging in, verify the architecture: +After logging in, verify the architecture before installing packages. The rest of this Learning Path assumes you are on an Arm64 system: ```bash uname -m @@ -83,9 +60,9 @@ aarch64 If you do not see aarch64, check that you created the VM with Arm64 architecture and selected an Azure Cobalt-compatible instance type. -## Install build dependencies +## Install dependencies and download QuantLib -Update the package index and install required packages: +Update the package index and install the required packages: ```bash sudo apt update @@ -94,29 +71,16 @@ sudo apt install -y build-essential cmake curl libboost-all-dev These packages provide the compiler toolchain, build system support, download tools, and Boost libraries needed to build QuantLib. -## Optional: use tmux for remote builds - -If you want the build to continue even if your SSH session disconnects, install tmux: - -```bash -sudo apt update -sudo apt install -y tmux -tmux -``` - -## Download QuantLib - -Set the version and download the release archive: +Set the QuantLib version as an environment variable, then download the release archive. Keeping the version in `QL_VER` makes later commands easier to repeat or adapt for a newer release: ```bash export QL_VER=1.41 - cd ~ curl -L -o QuantLib-$QL_VER.tar.gz \ https://github.com/lballabio/QuantLib/releases/download/v$QL_VER/QuantLib-$QL_VER.tar.gz ``` -Check that the file exists. Run: +Check that the file exists and has a non-zero size: ```bash ls -lh QuantLib-$QL_VER.tar.gz @@ -130,61 +94,85 @@ You should see output showing the file name and size, for example: If the file is missing or has size 0, re-run the curl command. -## Verify the file type +## Verify and extract the source archive + +Use the `file` command to confirm that the archive is a gzip-compressed tar file: -Use the file command to confirm that the archive is a valid gzip-compressed tar file: ```bash file QuantLib-$QL_VER.tar.gz ``` -Expected output is simlar to: +Expected output is similar to: + ```bash QuantLib-1.41.tar.gz: gzip compressed data, max compression, from Unix, original size modulo 2^32 42721280 ``` -### (Optional) Test archive integrity +Once confirmed, extract it and move into the extracted directory: -To check that the archive is not corrupted, run: ```bash -tar -tzf QuantLib-$QL_VER.tar.gz > /dev/null +tar -xzf QuantLib-$QL_VER.tar.gz +cd QuantLib-$QL_VER ``` -If the command completes without errors, the archive is valid. -If you see errors such as: + +List the contents to confirm that the source code is ready to configure and build: + +```bash +ls +``` + +You should see files and directories such as: ```bash -gzip: stdin: unexpected end of file -tar: Unexpected EOF in archive +configure +Makefile.am +ql/ +test-suite/ ``` -the download is incomplete or corrupted.l Delete the file and download it again. +This confirms that the source code has been unpacked correctly and is ready to configure and build. -## Extract the archive +{{% notice Optional Setup %}} +## Reconnect to Cobalt frequently + +If you'll reconnect often, add a shortcut entry to your SSH config: -Once the archive is verified, extract it: ```bash -tar -xzf QuantLib-$QL_VER.tar.gz +nano ~/.ssh/config ``` -Then move into the extracted directory: +Add: + ```bash -cd QuantLib-$QL_VER +Host quantlib-cobalt + HostName + User azureuser + IdentityFile ~/Downloads/quantlib-cobalt-vm_key.pem ``` -## Confirm the extracted contents +Then connect with: -List the contents: ```bash -ls +ssh quantlib-cobalt ``` -You should see files and directories such as: +## Use tmux for remote builds + +If your SSH session disconnects during the build, the compile job will be killed. To prevent this, install tmux and start a session before running `make`: + ```bash -configure -Makefile.am -ql/ -test-suite/ +sudo apt update +sudo apt install -y tmux +tmux ``` -This confirms that the source code has been unpacked correctly and is ready to configure and build. +Run the build commands from inside the tmux session. If your connection drops, reconnect to the VM and re-attach with: + +```bash +tmux attach +``` +{{% /notice %}} + +With your environment set up, move on to the next section to build QuantLib. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/quantlib/3-build-quantlib.md b/content/learning-paths/servers-and-cloud-computing/quantlib/3-build-quantlib.md index d7397f1755..dab74bdb99 100644 --- a/content/learning-paths/servers-and-cloud-computing/quantlib/3-build-quantlib.md +++ b/content/learning-paths/servers-and-cloud-computing/quantlib/3-build-quantlib.md @@ -6,15 +6,15 @@ weight: 4 layout: learningpathall --- -## Configure the QuantLib build +## Configure the build -From the QuantLib source directory: +Return to the QuantLib source directory. This uses the `QL_VER` variable you exported when downloading the source archive: ```bash cd ~/QuantLib-$QL_VER ``` -Run the configure script: +Run the configure script with benchmark support enabled: ```bash ./configure \ --prefix=/usr/local \ @@ -32,35 +32,30 @@ This configuration: - applies CPU-specific optimization flags -## Build QuantLib +## Install QuantLib -Compile using all available cores: +Compile using all available cores. The `nproc` command returns the number of processing units visible to the VM, so `make -j$(nproc)` keeps the build command portable across VM sizes: ```bash make -j$(nproc) ``` {{% notice Note %}} -The build may take 30–45 minutes on smaller instances. Use tmux to avoid losing progress if your SSH session disconnects. +The build may take 30–45 minutes on the Standard_D4ps_v5. If your SSH session might disconnect, set up tmux before running `make` — see the optional setup steps in the previous section. {{% /notice %}} -## Install QuantLib - -After the build completes: +After the build completes, install QuantLib into `/usr/local` and refresh the dynamic linker cache: ```bash sudo make install sudo ldconfig ``` -## Verify the build +Move to the test suite and check that the benchmark executable was created: -Move to the test suite: ```bash cd ~/QuantLib-$QL_VER/test-suite -``` - -Check that the benchmark executable exists: -```bash ls quantlib-benchmark ``` + +You should see `quantlib-benchmark` in the output. You will use this executable in the next section. diff --git a/content/learning-paths/servers-and-cloud-computing/quantlib/4-run-benchmarks.md b/content/learning-paths/servers-and-cloud-computing/quantlib/4-run-benchmarks.md index 45695b5e36..6dd7165391 100644 --- a/content/learning-paths/servers-and-cloud-computing/quantlib/4-run-benchmarks.md +++ b/content/learning-paths/servers-and-cloud-computing/quantlib/4-run-benchmarks.md @@ -7,23 +7,16 @@ layout: learningpathall --- ## Run a baseline benchmark -After building QuantLib, move to the test suite directory: +After building QuantLib, move to the test suite directory and run the benchmark with its default settings: ```bash cd ~/QuantLib-$QL_VER/test-suite -``` - -From the test suite directory, run a baseline benchmark: - -```bash ./quantlib-benchmark ``` This confirms that the benchmark is working correctly on your system. -## Vary thread count - -To understand how performance scales, run benchmarks with different numbers of threads: +To understand how performance scales across the Arm cores in the VM, keep the workload size constant and change only the number of worker processes: ```bash ./quantlib-benchmark --size=80 --nProc=1 @@ -31,11 +24,11 @@ To understand how performance scales, run benchmarks with different numbers of t ./quantlib-benchmark --size=80 --nProc=4 ``` -These runs keep the workload size constant while changing the number of threads. +The Standard_D4ps_v5 virtual machine has a limited number of cores. Start with 1, 2, and 4 workers. Larger values such as 12, 24, or 48 are better suited to larger machines and can oversubscribe this VM, which makes the results harder to interpret. ## Vary workload size -Next, vary the problem size while keeping the thread count fixed: +Next, keep the thread count fixed and vary the problem size. This shows how runtime changes as the benchmark does more work: ```bash ./quantlib-benchmark --size=1 --nProc=1 @@ -43,21 +36,35 @@ Next, vary the problem size while keeping the thread count fixed: ./quantlib-benchmark --size=8 --nProc=1 ``` -This shows how runtime changes as the workload increases. +## Interpreting the output -## Choose appropriate thread counts +The `--size=1 --nProc=1` run is the quickest of the three, completing in roughly 2 minutes 25 seconds on this configuration. It's a good first check before committing to longer runs. Adding `--verbose=2` prints a per-test runtime breakdown alongside the summary. Without it, only the summary block is printed. Verbosity levels run from 0 (summary only) to 2 (per-test detail); level 3 adds internal debug output. -The Standard_D4ps_v5 virtual machine has a limited number of cores. +The first run with `--size=1 --nProc=1 --verbose=2` produces output similar to the following. The benchmark first confirms that all tests passed, then lists the runtime for each individual test, and finishes with a summary: -Start with: +```output +*** No errors detected -* 1 thread -* 2 threads -* 4 threads +------------------------------------------------------------------------------------ + Total Runtime spent in each test +------------------------------------------------------------------------------------ +NthToDefaultTests/testGauss : 7.98616s +MarketModelCmsTests/testMultiStepCmSwapsAndSwaptions : 8.77406s +ZabrTests/testConsistency : 5.86461s +MarkovFunctionalTests/testVanillaEngines : 6.88742s +... +FunctionsTests/testFactorial : 0.004909s +------------------------------------------------------------------------------------ -Using larger values can oversubscribe the system and lead to inconsistent results. +Benchmark Size = Custom (1) +Number of processes = 1 +System Throughput = 0.598422 tasks/s +Benchmark Runtime = 145.382s +Num. Worker Processes = 1 +Tail Effect Ratio = 1 +``` -Your notes include larger values such as 12, 24, and 48 threads, but these are better suited for larger machines. +The **System Throughput** and **Benchmark Runtime** values are what you'll compare across runs. The individual test lines show which computations dominate total runtime — longer-running tests such as `testMultiStepCmSwapsAndSwaptions` and `testGauss` reflect the most numerically intensive parts of the workload. ## Keep benchmark runs controlled @@ -67,4 +74,4 @@ For meaningful comparisons: * keep the environment consistent * repeat runs if results vary -This helps ensure that differences in runtime reflect real performance changes. \ No newline at end of file +This helps ensure that differences in runtime reflect real performance changes. diff --git a/content/learning-paths/servers-and-cloud-computing/quantlib/5-analyze-results.md b/content/learning-paths/servers-and-cloud-computing/quantlib/5-analyze-results.md index e78a4809c0..d16692cb08 100644 --- a/content/learning-paths/servers-and-cloud-computing/quantlib/5-analyze-results.md +++ b/content/learning-paths/servers-and-cloud-computing/quantlib/5-analyze-results.md @@ -14,17 +14,17 @@ The following results were collected on a Standard_D4ps_v5 Azure Cobalt virtual | Command | Size | Threads | Throughput | Runtime | |--------|------|---------|------------|---------| -| `--size=1 --nProc=1` | 1 | 1 | 0.401534 tasks/s | 216.669s | -| `--size=5 --nProc=1` | 5 | 1 | 0.370434 tasks/s | 1174.3s | -| `--size=8 --nProc=1` | 8 | 1 | 0.401196 tasks/s | 1734.81s | +| `--size=1 --nProc=1` | 1 | 1 | 0.598422 tasks/s | 2m 25s | +| `--size=5 --nProc=1` | 5 | 1 | 0.370434 tasks/s | 19m 34s | +| `--size=8 --nProc=1` | 8 | 1 | 0.401196 tasks/s | 28m 55s | ### Thread scaling (fixed workload size) | Command | Size | Threads | Throughput | Runtime | |--------|------|---------|------------|---------| -| `--size=80 --nProc=1` | 80 | 1 | 0.372445 tasks/s | 18687.3s | -| `--size=80 --nProc=2` | 80 | 2 | 0.775048 tasks/s | 8980.08s | -| `--size=80 --nProc=4` | 80 | 4 | 1.55115 tasks/s | 4487s | +| `--size=80 --nProc=1` | 80 | 1 | 0.372445 tasks/s | 5h 11m | +| `--size=80 --nProc=2` | 80 | 2 | 0.775048 tasks/s | 2h 30m | +| `--size=80 --nProc=4` | 80 | 4 | 1.55115 tasks/s | 1h 15m | ## Understand workload scaling @@ -35,26 +35,26 @@ When increasing the workload size while keeping the thread count fixed: For example: -- `--size=1` completes in ~217 seconds -- `--size=8` completes in ~1735 seconds +- `--size=1` completes in ~2 minutes 25 seconds +- `--size=8` completes in ~28 minutes 55 seconds This shows that the benchmark is scaling the amount of work, not changing execution efficiency. ## Understand thread scaling -When increasing the number of processes: +When increasing the number of worker processes: - runtime decreases significantly - throughput increases almost linearly From the results: -- 1 → 2 threads: - - runtime drops from ~18687s to ~8980s +- 1 to 2 workers: + - runtime drops from ~5h 11m to ~2h 30m - throughput nearly doubles -- 2 → 4 threads: - - runtime drops again to ~4487s +- 2 to 4 workers: + - runtime drops again to ~1h 15m - throughput doubles again This indicates **near-linear scaling** on this system. @@ -65,9 +65,9 @@ Speedup compares performance relative to a single thread. | Threads | Runtime | Speedup | |--------|--------|---------| -| 1 | 18687.3s | 1.0× | -| 2 | 8980.08s | ~2.08× | -| 4 | 4487s | ~4.16× | +| 1 | 5h 11m | 1.0× | +| 2 | 2h 30m | ~2.08× | +| 4 | 1h 15m | ~4.16× | This shows slightly better than linear scaling, which can occur due to improved cache utilization or measurement variability. @@ -80,8 +80,13 @@ From these results: - Runtime grows with workload size, as expected - The system shows efficient utilization of available cores -## Practical guidance -{{% notice Note %}} +{{% notice Practical guidance %}} Large benchmark sizes such as `--size=80` can take several hours to complete on smaller virtual machines. For most use cases, smaller sizes such as 1, 5, or 8 are sufficient to demonstrate scaling behavior. {{% /notice %}} + +## What you learned + +You built QuantLib from source on an Arm-based Azure Cobalt VM, enabled its benchmark executable, and ran controlled tests that varied one parameter at a time. You also recorded enough context to compare runs later: VM size, workload size, worker count, runtime, and throughput. + +Use this workflow as a starting point for evaluating other C++ financial computing workloads on Arm cloud instances. For deeper comparisons, repeat the same benchmark process across VM sizes, compiler options, QuantLib versions, or cloud regions, and keep the command lines and environment details with the results. diff --git a/content/learning-paths/servers-and-cloud-computing/quantlib/_index.md b/content/learning-paths/servers-and-cloud-computing/quantlib/_index.md index bd619ed1a5..a92e62cf9a 100644 --- a/content/learning-paths/servers-and-cloud-computing/quantlib/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/quantlib/_index.md @@ -29,6 +29,8 @@ generate_summary_faq: true ### Tags skilllevels: Introductory subjects: Performance and Architecture +cloud_service_providers: + - Microsoft Azure armips: - Neoverse tools_software_languages: