Benchmark short Ada operations.

Define the claim to test.

A microbenchmark can estimate throughput, compare implementations under a shared local schedule, or expose unstable behavior. It cannot establish end-to-end service latency, production capacity, or a universal result for another host.

The crate uses mach_absolute_time on Darwin and CLOCK_MONOTONIC_RAW on Linux through a narrow C clock bridge. Adaptive calibration aims for useful batch durations while retaining the raw per-operation sample means and the clock characterization needed to audit them.

Measure one statically bound operation.

flyology_bench is currently version 0.1.0-dev and is published through the Flyology organization index. Keep the community index for compiler dependencies, add the Flyology development index ahead of it, then add the crate normally.

alr index --reset-community
alr index --add=git+https://github.com/flyology-ada/alire-index.git \
  --name=flyology --before=community
alr with flyology_bench

with Flyology_Bench;
with Flyology_Bench.Reporters;
with Interfaces;

procedure Bench_Mix is
   use type Interfaces.Unsigned_32;
   Value : Interfaces.Unsigned_32 := 1;

   procedure Mix is
   begin
      Value := Value * 1_664_525 + 1_013_904_223;
   end Mix;

   procedure Run is new Flyology_Bench.Measure (Mix);
   Result : Flyology_Bench.Measurement;
   Config : constant Flyology_Bench.Configuration :=
     Flyology_Bench.Reporters.Terminal_Mode (Name => "integer mix");
begin
   Run (Config => Config, Result => Result);
   Flyology_Bench.Reporters.Put_Console ("integer_mix", Result);
end Bench_Mix;

The Flyology_Bench package stores output in a Measurement and applies a Configuration. Terminal_Mode prepares terminal reporting, and Put_Console writes the result.

Measure is generic, so the operation is statically bound. Use Measure_Batched when one timed batch coordinates tasks or owns a fixture that the harness must not recreate per operation. Use setup and teardown hooks when preparation must stay outside each timed interval. The generated flyology_bench API reference lists the complete configuration, measurement, comparison, and reporter contracts.

Read the measurement report.

Terminal mode keeps labels and numeric columns stable during progress, then prints one row per kind of evidence. The capture below is from the maintained example on Darwin/AArch64. Results will change with the host, compiler, load, and run policy.

-- integer_mix -----------------------------------------
   latency    | median 1.498 ns/op  mean 1.633 ns
   tails      | p95 2.225 ns  p99 2.784 ns  range 1.454 ns .. 2.839 ns
   sampling   | 16777216 iter/sample x 55 samples  median batch 25.13 ms
   quality    | CV 18.60%  outliers 7  lag-1 correlation 0.47
   clock      | mach_absolute_time
   resolution | nominal 42.000 ns  observed 41.000 ns  median read 41.000 ns  floor 0.003 ps/op
   cpu        | ▇▇▇▇▇▇▇▇▁▇▇▇▇▇▇▇▇▇▇▇▇▇▁▇▇▇▇▇▇▇▇▇▇▇▇▇▁▇▇▇▇▇▇▇▇▆▃▄
              | average 96.8%  (1.0 cores)    peak 100.0%  (1.0 cores)
   memory     | ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
              | RSS change across own batches +0.0 KiB    largest batch +0.0 KiB
   elapsed    | 00:00:01.5 timed samples (hh:mm:ss)

Choose balanced rounds or sequential blocks.

Do not infer a comparison by running implementation A for a minute and implementation B later without recording that schedule. Compare alternates which of two implementations runs first. Compare_Many accepts two to sixteen enumeration values and, by default, rotates every implementation through each execution position.

type Candidate is (Current, Rewrite, Tasked);

procedure Batch
  (Which      : Candidate;
   Iterations : Flyology_Bench.Iteration_Count);

procedure Compare_All is new Flyology_Bench.Compare_Many
  (Case_Id => Candidate, Batch => Batch);
procedure Put_All is new
  Flyology_Bench.Reporters.Put_Multi_Comparison_Console (Candidate);

Result : Flyology_Bench.Multi_Comparison;
Config : Flyology_Bench.Configuration :=
  Flyology_Bench.Reporters.Terminal_Mode (Name => "parser shootout");
begin
Config.Measurement_Time := 3.0;
Config.Maximum_Sampling_Time := 2.0;
Config.Comparison_Batching := Flyology_Bench.Equal_Time;
Config.Shootout_Scheduling := Flyology_Bench.Sequential_Cases;
Compare_All (Config => Config, Result => Result);
Put_All (Result, Show_Individual_Details => True);

The batch receives an Iteration_Count. The generic Put_Multi_Comparison_Console renders a Multi_Comparison.

The first enumeration value is the reference. A speedup above 1.0x means the contender is faster. The elapsed-time column says the same thing in direct language: less is favorable, more is unfavorable. Color reinforces that verdict in a terminal, but the words and signs carry the meaning independently.

The confidence interval comes from deterministic circular-block resampling of paired log-ratios. A practical verdict appears only when the complete interval clears the configured effect threshold. Show_Individual_Details appends the full latency, sampling, clock, CPU, memory, and elapsed card for every implementation.

Measure resources with the same sample schedule.

Wall time remains the calibration and collection-budget clock. The Metrics field holds a Metric_Set. The predefined Process_Resource_Metrics and Linux_Hardware_Metrics values select additional counters. The harness captures them immediately before and after every retained batch, so time, CPU, memory, operating-system activity, and scheduler results use the same sample schedule.

use type Flyology_Bench.Metric_Set;

Config.Metrics :=
  Flyology_Bench.Process_Resource_Metrics
  or Flyology_Bench.Linux_Hardware_Metrics;

An unavailable axis reports why: unsupported platform, permission denial, unsupported event, unavailable counter resources, or probe failure. The API exposes the same value through Metric_Status. Console, CSV, and JSON output preserve the value instead of substituting zero.

perf_event_open returns EINVAL for an unsupported event and for a rejected attribute combination. The harness repeats that probe with only permission-related attributes before it classifies the result. This check distinguishes a missing counter from a harness defect.

All probes and counter controls stay outside the wall-clock timestamps, and setup or teardown hooks stay outside every selected axis. Probe families can still perturb one another at their boundaries. Adaptive batches amortize that fixed work, but a small resource difference should be repeated with only the relevant metric family selected.

Flyology scheduler axes include dispatches, poll batches, poll events, wakeups, and migration boundary crossings. They use Scheduler_Probe. A library-level adapter can copy cumulative values from one or more Flyology.Observability.Group_Snapshot records. The callback runs outside the timed interval, and the standalone benchmark crate keeps no dependency on the Flyology runtime.

Set a wall-time limit for collection.

Measurement_Time is the target amount of timed work across all cases. Maximum_Sampling_Time is a hard wall-time budget for the collection phase. A value of zero disables the limit.

Config.Measurement_Time := 3.0;
Config.Maximum_Sampling_Time := 2.0;
Config.Samples := 75;

The harness always completes at least ten samples and checks the budget only between complete samples, pairs, balanced rounds, or sequential case samples. One in-flight batch can therefore finish slightly after the boundary. A sequential shootout divides the collection ceiling among cases and analyzes the common sample count all cases completed. In every comparison, the measurement target is divided across timed implementations instead of being multiplied by their count.

Wait for sustained low CPU load.

CPU_Quiescence is an optional preflight check. Before clock characterization and warmup, it samples host CPU counters. The host average and busiest logical CPU must remain within their limits for one continuous interval. The check runs once for each measurement, comparison, or multi-way shootout.

Config.CPU_Quiescence :=
  (Enabled                     => True,
   Maximum_Average_CPU_Percent => 20.0,
   Maximum_Core_CPU_Percent    => 50.0,
   Stable_Time                 => 1.0,
   Poll_Interval               => 0.100,
   Timeout                     => 15.0);

The terminal progress bar represents the accepted portion of Stable_Time. It returns to zero when either utilization limit is exceeded. If the complete stable interval is not observed before Timeout, the benchmark raises CPU_Quiescence_Timeout without starting warmup or timed collection.

The maintained example enables the preflight with FLYOLOGY_BENCH_QUIESCENCE=1. It remains disabled by default so unattended runs do not acquire a new wait or failure condition.

Monitor the process outside timed intervals.

Terminal_Mode enables process telemetry and names every live progress line. During multi-way collection, a fixed-width field identifies the implementation that produced each sample. Its fixed width prevents the progress display from moving. The line also reports CPU percentage, occupied cores, current RSS, RSS growth, and elapsed wall time. Final reports retain CPU and RSS sparklines.

Record work controlled by the application.

Flyology_Bench.Recording is for a server, consumer, worker pool, or other long-lived process whose own control flow decides when work starts. A bounded Recorder stores identities of type Benchmark. Call Register before Start, then mark each span inside the application.

Begin_Sample initializes a Span. Finish it with the Success or Failure outcome. The benchmark runner remains available for calibrated microbenchmarks; recording does not force request handling through a harness callback.

package Recording renames Flyology_Bench.Recording;

Recorder : Recording.Recorder
  (Maximum_Benchmarks => 8,
   Retained_Samples   => 10_000);
Request : Recording.Benchmark;

Recording.Register (Recorder, "request", Request);
Recording.Start (Recorder);

declare
   Sample : Recording.Span;
begin
   Recording.Begin_Sample (Recorder, Request, Sample);
   begin
      Handle_Request;
   exception
      when others =>
         Recording.Finish (Sample, Recording.Failure);
         raise;
   end;
   Recording.Finish (Sample, Recording.Success);
end;

Registration allocates the bounded stores. The span path performs no Ada heap allocation. Finish captures its ending timestamp before entering the protected store, so retention work is outside recorded wall time. A snapshot reports observed, retained, omitted, active, abandoned, successful, failed, timed out, and cancelled spans. Each retained row keeps its observation number and outcome aligned with the status and optional value of every requested axis.

The live terminal rewrites one fixed display instead of printing an event log. It shows elapsed time, process CPU, occupied cores, and RSS. Each registered identity has one row for completed spans, active spans, errors, rolling median, and rolling p95.

fly recorder  elapsed 01.31 s      cpu  38.4% / 0.4 cores   rss 31.2 MiB
benchmark                    done  live  errors   median      p95
quick request                  66     0       2   42.0 ns     250.0 ns
cpu-heavy request              68     1       1    3.21 ms      4.08 ms
memory-burst request           70     0       2  140.3 us     358.2 us
wait-bound request             64     1       2   19.01 ms     64.03 ms

cd flyology_bench
alr exec -- gprbuild -p -P examples/flyology_bench_examples.gpr
./examples/bin/recording_service

FLYOLOGY_BENCH_RECORDING_OUTPUT=csv  ./examples/bin/recording_service
FLYOLOGY_BENCH_RECORDING_OUTPUT=json ./examples/bin/recording_service

The example does not use an HTTP server. Eight client tasks drive a long-lived rendezvous service with four workers. Instrumentation lives inside those workers, while fast, CPU-heavy, memory-burst, wait-bound, occasional-failure, and occasional-hiccup work makes the live fields move. CSV and JSON label the contracts as sample_semantics=individual_span and comparison_design=independent. Raw CSV and the JSON samples array retain observation identity and outcome across axes. Comparison output retains unavailable axes; when wall time is absent, wall-derived JSON fields are null rather than a fabricated 1× result.

Emit CSV or JSON without terminal output.

CSV and newline-delimited JSON reporters omit progress and ANSI sequences. The maintained example selects the reporter through one environment variable.

FLYOLOGY_BENCH_OUTPUT=terminal ./examples/bin/basic
FLYOLOGY_BENCH_OUTPUT=csv      ./examples/bin/basic > results.csv
FLYOLOGY_BENCH_OUTPUT=json     ./examples/bin/basic > results.ndjson
./examples/bin/basic --metrics=perf --require-perf
./examples/bin/basic --metrics=perf --require-perf=core

The existing CSV tables keep their latency schemas. Put_Metrics_CSV, Put_Comparison_Metrics_CSV, and the multi-way long-form reporter emit one row per requested axis with scope, unit, availability status, summaries, confidence bounds, and verdict. JSON measurement, comparison, and multi-way objects include the same status and metric arrays alongside clock and host/toolchain metadata. For regression history, Flyology_Bench.Baselines stores raw time samples and rejects a comparison when the clock backend or environment fingerprint differs.

Treat fractional nanoseconds as amortized throughput.

A result such as 0.878 ns/op does not mean the clock directly resolved one operation in 878 picoseconds. The harness timed a batch lasting milliseconds and divided that duration by the logical operation count. This is useful throughput evidence when the batch performs exactly that many comparable operations.

Repeat on the host you intend to describe.

The crate is maintained on Darwin and Linux, including native Linux/AArch64 Docker validation. Its default fingerprint records operating system, architecture, and GNAT version. Add compiler switches, revision, CPU placement and power policy, and other conditions that could change the result.

cd flyology_bench
alr test
./examples/bin/basic

alr test also checks the published machine-readable schemas. Every CSV row must contain the columns declared by its header. Long-form metric rows must agree between available and status. Each JSON object must parse. The test uses jq when available and reports when it uses structural checks instead.

From the repository root, FLYOLOGY_LINUX_PERF=1 ./scripts/test-linux-docker.sh grants Docker's PERFMON capability and makes the example require real hardware-counter samples. It also makes the smoke test compare a serial batch against one that starts four worker tasks, so inherited attribution is measured rather than assumed. A virtualized Linux host must also expose its PMU; the capability alone cannot provide missing virtual hardware. A guest kernel without one lists no CPU entry under /sys/bus/event_source/devices and rejects every hardware event, and the run then reports the specific unavailable status instead of passing.

For a shared report, keep the raw machine output, record the exact command and environment, repeat the full run, and describe only the tested host. Direct Compare or Compare_Many is preferable whenever all implementations can run in one process.