Flyology.Observability.Stall_Watchdogs

Description

Samples scheduler progress from a native monitor task to identify stalls.

Example:

Flyology.Observability.Stall_Watchdogs.Start (Monitor);

Group_Condition

type Group_Condition is
  (Not_Started,
   Group_Absent,
   Group_Starting,
   Idle,
   Waiting,
   Advancing,
   Runnable_Not_Progressing,
   Suspected_Stall,
   …,
   Monitor_Failed);

Interpretation of the latest pair of periodic samples.

Enumeration literals

Not_Started

Start has not yet created a monitor

Group_Absent

The observed group has never been created

Group_Starting

The scheduler thread is starting

Idle

No task is runnable or waiting

Waiting

Tasks are suspended and none is runnable

Advancing

Runnable work and scheduler counters advanced

Runnable_Not_Progressing

Runnable work has not crossed threshold

Suspected_Stall

Runnable work did not progress for the threshold

Group_Failed

The scheduler thread reports failure

Monitor_Stopped

Stop terminated the monitor normally

Monitor_Failed

The native monitor task raised an exception

Invalid_Configuration

Invalid_Configuration : exception;

Raised when a watchdog interval is nonpositive or threshold is shorter than the sample interval.

Is_Running

function Is_Running (Object : Watchdog) return Boolean

Query monitor task liveness; serialize this with Start and Stop.

Parameters

Object

Watchdog to inspect

Return value

True while its native monitor task has not terminated

Latest_Report

function Latest_Report
  (Object : Watchdog) return Watchdog_Report

Copy the latest protected report. A report compares periodic samples, not an atomic history; Suspected_Stall is evidence, not deadlock proof. Stall_Episodes increments once per observed episode.

Parameters

Object

Watchdog to inspect

Return value

Latest report, or default Not_Started report before Start

Start

procedure Start
  (Object : in out Watchdog;
   Config : Watchdog_Config := (others => <>))

Start sampling Config.Group from one native task. The operation creates no observed group. A stopped Watchdog may be restarted.

Parameters

Object

Stopped watchdog to start

Config

Positive sampling configuration

Raised exceptions

Invalid_Configuration

Intervals violate the stated rules

Program_Error

Object is already running

Storage_Error

Monitor state or task allocation fails

Tasking_Error

Native monitor task activation fails

Stop

procedure Stop (Object : in out Watchdog)

Stop and join the native monitor task. The final Monitor_Stopped report remains queryable. Calling before Start or after Stop is harmless.

Parameters

Object

Watchdog to stop

Watchdog

type Watchdog is new Ada.Finalization.Limited_Controlled with private;

Controlled owner of one native monitor task. Declaration is inert. Start and Stop must not run concurrently on the same object; callers must also serialize those lifecycle calls with Is_Running. Latest_Report is safe during monitoring. Finalize stops and releases the monitor.

Watchdog_Config

type Watchdog_Config is record
   Group           : Group_Id := 0;
   Sample_Interval : Duration := 0.050;
   Stall_Threshold : Duration := 0.250;
end record;

Periodic sampling configuration for one execution group.

Record fields

Group

Group to observe without starting it

Sample_Interval

Seconds between samples

Stall_Threshold

Seconds without progress before suspicion

Watchdog_Report

type Watchdog_Report is record
   Group              : Group_Id := 0;
   Condition          : Group_Condition := Not_Started;
   Sample_Sequence    : Counter := 0;
   Snapshot_Available : Boolean := False;
   Observed_For       : Duration := 0.0;
   Ready              : Counter := 0;
   Waiting            : Counter := 0;
   Running            : Counter := 0;
   Dispatches         : Counter := 0;
   Poll_Batches       : Counter := 0;
   Stall_Episodes     : Counter := 0;
end record;

Latest sampled condition and diagnostic counters.

Record fields

Group

Observed execution group

Condition

Interpretation of the sample pair

Sample_Sequence

Cumulative samples by this monitor run

Snapshot_Available

Whether the group snapshot existed

Observed_For

Seconds runnable work lacked observed progress

Ready

Ready count from the latest snapshot

Waiting

Waiting count from the latest snapshot

Running

Running count from the latest snapshot

Dispatches

Cumulative dispatches from the latest snapshot

Poll_Batches

Cumulative poll batches from the latest snapshot

Stall_Episodes

Distinct suspected-stall episodes in this run