Description
Registration or identity contract violation.
Action
function Action (Options : Runner_Options) return Runner_Action
Return the selected top-level action.
Parameters
- Options
Parsed runner policy.
Return value
Run, list, or help action.
Argument_List
type Argument_List is array (Positive range <>) of Ada.Strings.Unbounded.Unbounded_String;
String array used by the testable parser. A null array represents no arguments.
Case_Index
subtype Case_Index is Positive range 1 .. Maximum_Cases;
One-based registered case index.
Comparison_Callback
type Comparison_Callback is access procedure (Config : Configuration; Result : out Comparison);
Callback around an already-instantiated Compare procedure.
Parameters
- Config
Effective shared collection policy.
- Result
Completed paired comparison.
Comparison_Value
function Comparison_Value (Result : Registered_Result) return Comparison
Return a paired result.
Parameters
- Result
Exact execution result.
Return value
Stored Comparison.
Raised exceptions
- Constraint_Error
If Kind is not Paired_Comparison.
Effective_Configuration
function Effective_Configuration (Options : Runner_Options) return Configuration
Return the effective shared benchmark configuration.
Parameters
- Options
Parsed runner policy.
Return value
Base configuration with explicit CLI overrides.
Error_Policy
type Error_Policy is (Continue_After_Error, Fail_Fast);
Response to a callback exception.
Enumeration literals
- Continue_After_Error
Report the error and invoke later cases.
- Fail_Fast
Stop before the next selected case.
Execute
procedure Execute
(Target : Suite;
Suite_Name : String;
Options : Runner_Options;
Summary : out Run_Summary;
Output : Ada.Text_IO.File_Type := Ada.Text_IO.Standard_Output;
Progress : Ada.Text_IO.File_Type := Ada.Text_IO.Standard_Error)
Execute selected callbacks serially and stream their results. Human output delegates detailed measurements to Flyology_Bench.Reporters. CSV prefixes the existing typed result/metric schemas with suite context; newline-delimited JSON extends each complete existing result object. Progress and exception diagnostics go to Progress, never to an explicit --output file. An explicit file is plain and contains no ANSI escapes. Output and Progress must remain open and must not be reset or closed, including through another view, until Execute returns.
Parameters
- Target
Registry to execute.
- Suite_Name
Stable suite identity segment.
- Options
Parsed selection, configuration, and reporting policy.
- Summary
Final bounded aggregate.
- Output
Default result stream when no path is explicit.
- Progress
Diagnostic stream excluded from an explicit result file.
Execute_One
procedure Execute_One
(Target : Suite; Full_Name : String; Config : Configuration; Result : out Registered_Result)
Invoke exactly one registration by full identity. This operation does not catch callback exceptions and performs no reporting or aggregation.
Parameters
- Target
Registry containing the callback.
- Full_Name
Exact stable identity.
- Config
Collection policy passed without modification.
- Result
Exact callback result.
Raised exceptions
- Constraint_Error
If Full_Name is absent or multi-way.
Execute_One_Multi
procedure Execute_One_Multi
(Target : Suite; Full_Name : String; Config : Configuration; Result : out Multi_Comparison)
Invoke exactly one multi-way registration without inflating every ordinary/paired Registered_Result to Multi_Comparison's storage size.
Parameters
- Target
Registry containing the callback.
- Full_Name
Exact stable identity.
- Config
Collection policy passed without modification.
- Result
Exact multi-way result.
Raised exceptions
- Constraint_Error
If Full_Name is absent or not multi-way.
Execute_Worker_Request
procedure Execute_Worker_Request
(Target : Suite;
Request : Flyology_Bench.Workers.Worker_Request;
Template : Configuration := Default_Configuration)
Execute one validated worker request through the same exact suite registry used by parent-side selection. The request kind and identity are checked before readiness is announced. Multi-way cases are not part of the current worker protocol. Benchmark exceptions propagate so the worker main can return their exact name and message.
Parameters
- Target
Registry containing the requested callback.
- Request
Validated worker-side protocol request.
- Template
Worker-local callback sources retained in configuration.
Raised exceptions
- Constraint_Error
If identity or kind does not match.
Execution_Order
type Execution_Order is (Registration_Order, Name_Order);
Ordering of selected cases.
Enumeration literals
- Registration_Order
Preserve explicit registration order.
- Name_Order
Sort by complete case-sensitive identity.
Final_Status
type Final_Status is
(Succeeded, No_Matching_Cases, Benchmark_Failed, Requested_Metric_Unavailable, Regression_Rejected);
Final runner classification. Inconclusive paired or multi-way results do not fail a run. A gated ordinary measurement fails only when its explicit baseline policy rejects the result.
Enumeration literals
- Succeeded
Every applicable success condition passed.
- No_Matching_Cases
Selection was empty without --allow-empty.
- Benchmark_Failed
At least one callback raised an exception.
- Requested_Metric_Unavailable
A required built-in axis was absent.
- Regression_Rejected
A baseline gate rejected at least one result.
Format
function Format (Options : Runner_Options) return Output_Style
Return the requested output format.
Parameters
- Options
Parsed runner policy.
Return value
Human, CSV, or JSON style.
Full_Name
function Full_Name (Target : Suite; Index : Case_Index) return String
Return one full identity in registration order.
Parameters
- Target
Registry to inspect.
- Index
One-based registration index.
Return value
Stable full identity.
Is_Dry_Run
function Is_Dry_Run (Options : Runner_Options) return Boolean
Return whether the fast validation policy is active.
Parameters
- Options
Parsed runner policy.
Return value
True for validation-only execution.
Is_Selected
function Is_Selected (Target : Suite; Options : Runner_Options; Index : Case_Index) return Boolean
Return whether one registration is selected by exact, filter, group, tag, and skip predicates. A --filter without '' or '?' is a substring match. Otherwise '' matches zero or more characters and '?' matches one character. Matching is case-sensitive and applies to full identity.
Parameters
- Target
Registry containing the case.
- Options
Parsed selection policy.
- Index
One-based registration index.
Return value
Whether every inclusion and exclusion predicate accepts it.
Kind
function Kind (Result : Registered_Result) return Result_Kind
Return the kind stored in one exact execution result.
Parameters
- Result
Exact execution result.
Return value
Ordinary or paired result kind.
Kind
function Kind (Target : Suite; Index : Case_Index) return Result_Kind
Return one registered result kind.
Parameters
- Target
Registry to inspect.
- Index
One-based registration index.
Return value
Ordinary, paired, or multi-way result kind.
Length
function Length (Target : Suite) return Natural
Return the number of registrations.
Parameters
- Target
Registry to inspect.
Return value
Registered case count.
List
procedure List
(Target : Suite;
Options : Runner_Options;
Summary : out Run_Summary;
Output : Ada.Text_IO.File_Type := Ada.Text_IO.Standard_Output)
List selected full identities without invoking callbacks. Registration order is retained unless --order=name was selected.
Parameters
- Target
Registry to inspect.
- Options
Parsed selection and ordering policy.
- Summary
Discovery aggregate with no completed callbacks.
- Output
Destination for one identity per line.
Maximum_Cases
Maximum_Cases : Positive;
Measurement_Callback
type Measurement_Callback is access procedure (Config : Configuration; Result : out Measurement);
Callback around an already-instantiated Measure procedure.
Parameters
- Config
Effective shared collection policy.
- Result
Completed ordinary measurement.
Measurement_Value
function Measurement_Value (Result : Registered_Result) return Measurement
Return an ordinary result.
Parameters
- Result
Exact execution result.
Return value
Stored Measurement.
Raised exceptions
- Constraint_Error
If Kind is not Ordinary_Measurement.
Option_Error
Option_Error : exception;
Invalid or conflicting command-line option.
Output_Path
function Output_Path (Options : Runner_Options) return String
Return the explicit output path, or an empty string for Output.
Parameters
- Options
Parsed runner policy.
Return value
Explicit output path or an empty string.
Output_Style
type Output_Style is (Human, CSV, JSON);
Output written by the suite runner.
Enumeration literals
- Human
Plain result cards and suite summary.
- CSV
Typed table sections using the existing reporter schemas.
- JSON
Newline-delimited JSON suite objects.
Parse
function Parse
(Arguments : Argument_List; Base_Config : Configuration := Default_Configuration) return Runner_Options
Parse explicit arguments against Base_Config. Options accept either --name=value or --name value where a value is required. Duration values use a nonnegative decimal followed by ns, us, ms, or s. Parsing is strict: signs and exponents are rejected for durations and percentages.
Parameters
- Arguments
Explicit argument values without a program name.
- Base_Config
Configuration retained where no override is present.
Return value
Parsed selection, execution, configuration, and output policy.
Raised exceptions
- Option_Error
On an unknown, malformed, out-of-range, or conflicting option.
Parse_Command_Line
function Parse_Command_Line (Base_Config : Configuration := Default_Configuration) return Runner_Options
Parse Ada.Command_Line arguments. This wrapper does no execution.
Parameters
- Base_Config
Configuration retained where no override is present.
Return value
Parsed selection, execution, configuration, and output policy.
Put_Help
procedure Put_Help (Output : Ada.Text_IO.File_Type := Ada.Text_IO.Standard_Output)
Print stable command-line help without parsing global state.
Parameters
- Output
Destination help stream.
Register
procedure Register
(Target : in out Suite;
Name : String;
Run : not null Measurement_Callback;
Group : String := "";
Tags : String := "")
Register one ordinary measurement. Name and Group are case-sensitive portable identity segments: ASCII letters or digits followed by ASCII letters, digits, '.', '_', or '-'. Full identity is Group & "/" & Name, or Name when Group is empty. Tags is a comma-separated list using the same segment grammar. Empty tags and surrounding whitespace are not accepted.
Parameters
- Target
Registry to extend.
- Name
Case identity segment.
- Run
Already-instantiated measurement wrapper.
- Group
Optional identity parent segment.
- Tags
Optional comma-separated selection tags.
Register_Gated
procedure Register_Gated
(Target : in out Suite;
Name : String;
Run : not null Measurement_Callback;
Baseline_Path : String;
Policy : Flyology_Bench.Baselines.Gate_Policy :=
Flyology_Bench.Baselines.Fail_Closed_Gate_Policy;
Fingerprint : String := "";
Group : String := "";
Tags : String := "")
Register one ordinary measurement with a read-only saved-baseline gate. The gate runs after successful collection, uses the effective suite confidence, resample, and random-seed policy, and never creates or updates Baseline_Path. An empty Fingerprint selects benchmark metadata.
Parameters
- Target
Registry to extend.
- Name
Case identity segment and baseline benchmark identity.
- Run
Already-instantiated measurement wrapper.
- Baseline_Path
Existing or policy-handled baseline artifact path.
- Policy
Regression threshold and exceptional-state actions.
- Fingerprint
Exact environment identity, or empty for metadata.
- Group
Optional identity parent segment.
- Tags
Optional comma-separated selection tags.
Register_Paired
procedure Register_Paired
(Target : in out Suite;
Name : String;
Reference_Name : String;
Contender_Name : String;
Run : not null Comparison_Callback;
Group : String := "";
Tags : String := "")
Register one paired comparison. Reference_Name and Contender_Name are reporter labels and follow the identity-segment grammar.
Parameters
- Target
Registry to extend.
- Name
Case identity segment.
- Reference_Name
Stable reference reporter label.
- Contender_Name
Stable contender reporter label.
- Run
Already-instantiated comparison wrapper.
- Group
Optional identity parent segment.
- Tags
Optional comma-separated selection tags.
Registered_Result
type Registered_Result is private;
Result of one exact, non-reporting callback execution. This is the backend seam for process-isolation adapters; the suite retains registry, policy, reporting, and aggregation ownership.
Registration_Error
Registration_Error : exception;
Registration or identity contract violation.
Result_Kind
type Result_Kind is (Ordinary_Measurement, Paired_Comparison, Multi_Way_Comparison);
Kind of result produced by a registered callback.
Enumeration literals
- Ordinary_Measurement
One Measurement result.
- Paired_Comparison
One order-balanced Comparison result.
- Multi_Way_Comparison
One comparison of two to sixteen cases.
Run_Summary
type Run_Summary is record
Discovered : Natural := 0;
Selected : Natural := 0;
Completed : Natural := 0;
Skipped : Natural := 0;
Failed : Natural := 0;
Inconclusive : Natural := 0;
Unavailable : Natural := 0;
Rejected : Natural := 0;
Dry_Run : Boolean := False;
Status : Final_Status := Succeeded;
end record;
Programmatically inspectable bounded aggregate. Completed counts callbacks that returned normally, including dry runs and inconclusive comparisons. Skipped counts unselected registrations and selected cases not reached after fail-fast.
Record fields
- Discovered
Total explicit registrations.
- Selected
Registrations selected before execution.
- Completed
Callbacks that returned normally.
- Skipped
Unselected or fail-fast-unreached registrations.
- Failed
Callbacks that raised an exception.
- Inconclusive
Completed paired results without a verdict.
- Unavailable
Completed cases missing a required built-in axis.
- Rejected
Results rejected by an installed baseline gate.
- Dry_Run
Whether validation-only collection was used.
- Status
Final process-oriented classification.
Runner_Action
type Runner_Action is (Run_Selected, List_Selected, Show_Help);
Requested top-level runner action.
Enumeration literals
- Run_Selected
Execute the selected callbacks.
- List_Selected
List identities without executing callbacks.
- Show_Help
Print command-line help.
Runner_Options
type Runner_Options is private;
Parsed selection, configuration overrides, and output policy.
Successful
function Successful (Summary : Run_Summary) return Boolean
Return whether the final status is successful.
Parameters
- Summary
Completed aggregate.
Return value
True only for Succeeded.
Suite
type Suite is tagged limited private;
Explicit bounded suite registry.