Upgrade a running server with overlapping generations.

Reload can mean four different operations.

Before selecting a mechanism, decide which continuity property matters. Flyology implements the rolling replacement in the final row.

Updating the executable on disk does not change pages already mapped by a running process. A shared-library switch remains useful for a narrow, versioned plugin ABI used only after every task stops calling the plugin. Flyology keeps that separate from whole-server replacement.

Define continuity before designing the handoff.

Consider a TLS server with sessions that can remain open for several minutes. During deployment, clients need three different forms of continuity.

1. 01
   Admission continuity

   The service keeps a bound listener and a kernel backlog while generations change.

2. 02
   Connection continuity

   Each accepted connection stays with the generation that owns its TLS and protocol state.

3. 03
   Control continuity

   The coordinator uses transaction authority to act on the exact upgrade. That authority contains the coordinator id, upgrade id, and candidate generation.

These guarantees do not require execution continuity. The service does not need to move a task stack, protected object, parser buffer, or TLS session into the new process.

Follow one upgrade from generation 17 to generation 18.

Generation 17 is active and owns three established connections. The coordinator owns the original listener. The deployment supplies generation 18 with a launch description and a topology identity.

That identity contains an application signature, schema, epoch, digest, and candidate role. The role classifies the external effects that the candidate may perform. Generation 18 obtains the actual desired topology from its binary, configuration, or another durable source. It checks the reconstructed result against the supplied identity.

Preparation keeps admission disabled while generation 18 constructs its local owners. Canary evidence exists before the coordinator chooses the cancellation or promotion branch.

If the promoted generation later regresses, reversal starts a new generation from the retained prior launch description and provisioning data.

The complete coordinator example later follows this same path in Ada, with one comment for each lifecycle decision.

Make the application reconstructible before upgrading it.

Socket handoff transfers only the listener capability. The application must define how a candidate generation becomes correct from process-independent sources.

If the application cannot reconstruct its local topology or fence overlapping effects, it is not ready for this upgrade procedure. Use a stop-and-restart deployment until those boundaries exist.

Keep one stable owner outside both generations.

Flyology.Process_Generations.Coordinators owns the original listening socket. Each generation receives a listener duplicate as an operating-system capability. The generation lifecycle protocol authorizes the candidate to admit work separately.

The previous generation remains correct without knowing the candidate's layout. The candidate starts with its own executable, static data, heap, tasking runtime, and supervision scopes. When lightweight tasks require event machinery, that machinery also starts within the candidate process.

Transfer capabilities and descriptions, not live process state.

The Provisioning_Data record carries application identity, topology schema, epoch, digest, and role. Fixed-width messages avoid sending Ada record layouts across the wire.

Reconstruct the supervision tree from desired topology.

The coordinator sends a topology identity, not the topology itself. The candidate obtains the desired topology from application-controlled sources. It creates new supervisors, tasks, structured servers, and resource owners.

After canary authorization, the candidate starts the server task. Only after every required server accepts and the reconstructed topology matches the provision, return true from the application readiness hook. Flyology then validates the reported epoch and digest against the supplied identity.

Flyology.IO.Structured_Servers supplies the admission and drain boundary. The application defines the required supervised children and the digest calculation. Flyology does not inspect that application topology independently.

Declare the candidate generation's launch context.

A Flyology.Subprocesses.Command selects the candidate's executable, arguments, working directory, environment, and path-search policy. The following table identifies the properties that spawn stores, inherits, or resets.

Create the value with To_Command. Add each exact argument with Append_Argument. For a repeatable launch, call Set_Working_Directory, then use Clear_Environment and Set_Environment_Variable to build an explicit environment.

Command := Flyology.Subprocesses.To_Command ("/srv/releases/app-18");
Flyology.Subprocesses.Append_Argument (Command, "--managed-generation");
Flyology.Subprocesses.Set_Working_Directory (Command, "/srv/app");

Flyology.Subprocesses.Clear_Environment (Command);
Flyology.Subprocesses.Set_Environment_Variable
  (Command, "APP_CONFIG", "/etc/app/config.toml");
Flyology.Subprocesses.Set_Environment_Variable
  (Command, "LANG", "C.UTF-8");

Flyology.Subprocesses.Bootstrap.Spawn gives the child a framed control stream on descriptor 3 and a dedicated capability channel on descriptor 4.

Advance only across acknowledged boundaries.

1. 1Spawnfresh process
2. 2Provisiontopology identity
3. 3Preparecandidate holds listener capability
4. 4Canaryadmission enabled
5. 5Promotecommit candidate
6. 6Drainretire previous

Each transaction has a coordinator id, upgrade id, and candidate generation. The authority must match all three values. Stale commands cannot act on a later generation or upgrade.

Before calling Start_Upgrade, the deployer must verify the artifact identity and trust policy. The call returns after spawn, a child handshake bound to the transaction authority, provisioning, and listener handoff. The candidate still cannot accept.

Begin_Canary authorizes admission. It waits until the application reports admission readiness, then validates the reported epoch and digest against the provision.

Connect the generation lifecycle protocol to the application.

Flyology.Process_Generations.Agents is generic over an Application_Context and six lifecycle hooks. It owns bootstrap and the lifetime of one server task. Application code owns topology construction and effect semantics.

The hooks connect protocol transitions to application-owned scopes. Run adopts the inherited channels and drives those transitions for one generation.

package Image_Agent is new Flyology.Process_Generations.Agents
  (Application_Context => Server_Context,
   Prepare             => Prepare_Topology,
   Run_Server          => Run_Structured_Server,
   Ready               => Exact_Topology_Is_Accepting,
   Request_Stop        => Revoke_Then_Stop,
   Promoted            => Enable_Active_Only_Effects,
   Compensate          => Compensate_Canary_Effects);

Context : Server_Context;
Image_Agent.Run (Context);

Serialize upgrade decisions in the coordinator.

The coordinator owns listener escrow and tracks one active generation plus at most one candidate. Its operations are synchronous, so the caller must serialize them.

Prepare the candidate with Start_Upgrade, then authorize admission with Begin_Canary. After the canary passes, call Promote. If the canary does not pass, call Cancel.

The complete example below shows these calls in their first-generation and shutdown context.

Use Snapshot for point-in-time state. It reconciles terminal child state and exposes phase, generation identities, admission, rollback availability, compensation, and a bounded failure message.

Keep lifecycle policy in Ada.

Ada owns each acknowledged transition and every identity, validation, timeout, cleanup, compensation, and recovery decision.

The lifecycle policy is a pure SPARK unit. C does not decide when to admit, revoke, drain, compensate, promote, or reverse.

Start the first managed generation and release it explicitly.

Create one Coordinator in the stable owner process. Initialize transfers the original bound listener into escrow and closes the caller's socket owner.

Start_Initial starts, prepares, authorizes, and promotes the first generation under one total timeout. Use it to establish generation 17 before attempting generation 18.

At service shutdown, call Shutdown. It attempts cooperative drain and reaping for every managed generation. If cooperation is unavailable or fails, it hard-fences that generation and continues through the others. It releases listener escrow after all attempts, then reports the first cleanup failure. Each managed generation receives its own complete drain timeout.

Run the complete generation 17 to 18 upgrade.

What the demo server does.

The program being upgraded is a one-byte loopback request/reply server. For each accepted connection, the handler reads one byte and replies with the artifact's version byte. Generation 17 runs an artifact that replies with 1. Generation 18 runs a separate artifact that replies with 2.

The coordinator sends Q as a probe after the upgrade decision. A promotion run must receive 2, showing that a new connection reached the changed executable. A cancellation run must receive 1, showing that generation 17 remained the serving generation.

The server behavior is deliberately small. The example focuses on listener escrow, handoff, readiness, canary admission, promotion or cancellation, structured drain, and final shutdown. Already accepted connections remain with the generation that accepted them.

The application supplies a bound Listener, immutable commands, provisioning records, and its canary decision. The example uses Coordinators and Generations as package renames so each lifecycle call remains short.

Start_Initial and Start_Upgrade each return an exact Upgrade_Handle as transaction authority. Cancellation also reports a Compensation_Result. The following five fragments implement the lifecycle shown in the earlier story.

with Ada.Command_Line;
with Ada.Directories;
with Ada.Streams;
with Ada.Text_IO;
with Flyology.IO.Sockets;
with Flyology.Process_Generations;
with Flyology.Process_Generations.Coordinators;
with Flyology.Process_Generations.Messages;
with Flyology.Subprocesses;

procedure Guide_Process_Upgrade is
   package Coordinators renames
     Flyology.Process_Generations.Coordinators;
   package Generations renames Flyology.Process_Generations;
   package Messages renames Flyology.Process_Generations.Messages;
   package Sockets renames Flyology.IO.Sockets;
   package Subprocesses renames Flyology.Subprocesses;

   use type Ada.Streams.Stream_Element;
   use type Generations.Image_Generation;

   function Requested_Promotion return Boolean is
   begin
      if Ada.Command_Line.Argument_Count /= 1 then
         raise Program_Error with "expected promote or cancel";
      elsif Ada.Command_Line.Argument (1) = "promote" then
         return True;
      elsif Ada.Command_Line.Argument (1) = "cancel" then
         return False;
      else
         raise Program_Error with "expected promote or cancel";
      end if;
   end Requested_Promotion;

   Bin_Directory : constant String :=
     Ada.Directories.Containing_Directory
       (Ada.Directories.Full_Name (Ada.Command_Line.Command_Name));
   Command_17 : constant Subprocesses.Command := Subprocesses.To_Command
     (Ada.Directories.Compose
        (Bin_Directory, "process_generation_agent_v1"));
   Command_18 : constant Subprocesses.Command := Subprocesses.To_Command
     (Ada.Directories.Compose
        (Bin_Directory, "process_generation_agent_v2"));

   function Provision
     (Epoch : Messages.Nonzero_U64) return Messages.Provisioning_Data is
   begin
      return
        (Application_Signature => 16#F1#,
         Topology_Schema       => 1,
         Topology_Epoch        => Epoch,
         Digest                => (others => 0),
         Role                  => Generations.Canary_Safe);
   end Provision;

   Promote_Candidate : constant Boolean := Requested_Promotion;
   Canary_Evidence_Satisfies_Policy : constant Boolean :=
     Promote_Candidate;
   Coordinator_Identity : constant Generations.Coordinator_Id := 7_001;
   Provision_17 : constant Messages.Provisioning_Data := Provision (17);
   Provision_18 : constant Messages.Provisioning_Data := Provision (18);

   Manager      : Coordinators.Coordinator;
   Initial      : Generations.Upgrade_Handle;
   Candidate    : Generations.Upgrade_Handle;
   Compensation : Generations.Compensation_Result;
   Listener : Sockets.Socket_Type;
   Address  : Sockets.Endpoint;

   function Query return Ada.Streams.Stream_Element is
      Client : Sockets.Socket_Type;
      Reply  : Ada.Streams.Stream_Element_Array (1 .. 1);
   begin
      Sockets.Create_Socket (Client);
      Sockets.Connect (Client, Address, Timeout => 2.0);
      Sockets.Send_All
        (Client, [1 => Character'Pos ('Q')], Timeout => 2.0);
      Sockets.Receive_Exactly (Client, Reply, Timeout => 2.0);
      return Reply (1);
   end Query;

   procedure Reconcile_Cancellation
     (Result : Generations.Compensation_Result) is
   begin
      raise Program_Error with
        "cancellation requires reconciliation: " &
        Generations.Compensation_Result'Image (Result);
   end Reconcile_Cancellation;

   procedure Assert_Generation
     (Authority : Generations.Upgrade_Handle;
      Expected  : Generations.Image_Generation) is
   begin
      if Authority.Candidate /= Expected then
         raise Program_Error with "unexpected image generation";
      end if;
   end Assert_Generation;

   procedure Serve_Until_Test_Requests_Shutdown is
      Expected : constant Ada.Streams.Stream_Element :=
        (if Promote_Candidate then Character'Pos ('2')
         else Character'Pos ('1'));
   begin
      for Attempt in 1 .. 40 loop
         begin
            if Query = Expected then
               Ada.Text_IO.Put_Line
                 ((if Promote_Candidate
                   then "promotion: generation 18 served"
                   else "cancellation: generation 17 remained active"));
               return;
            end if;
         exception
            when others =>
               if Attempt = 40 then
                  raise;
               end if;
         end;
         delay 0.025;
      end loop;
      raise Program_Error with "selected generation did not serve";
   end Serve_Until_Test_Requests_Shutdown;
begin
   Sockets.Create_Socket (Listener);
   Sockets.Set_Socket_Option
     (Listener, Sockets.Socket_Level, (Sockets.Reuse_Address, True));
   Sockets.Bind_Socket
     (Listener,
      Sockets.Network_Endpoint (Sockets.Loopback_IPv4, Sockets.Any_Port));
   Sockets.Listen_Socket (Listener, Length => 16);
   Address := Sockets.Get_Socket_Name (Listener);

1. Escrow the listener and start generation 17.

The stable coordinator takes the bound listener. Start_Initial then constructs generation 17 and makes it active.

   --  Transfer the bound listener to the stable coordinator.
   Coordinators.Initialize
     (Item             => Manager,
      Identity         => Coordinator_Identity,
      Listener         => Listener,
      First_Generation => 17);

   --  Start generation 17 and make it the first active generation.
   Coordinators.Start_Initial
     (Item       => Manager,
      Executable => Command_17,
      Provision  => Provision_17,
      Authority  => Initial);
   Assert_Generation (Initial, 17);

2. Prepare generation 18 without admission.

Start_Upgrade starts the candidate, provisions its topology identity, and hands it a listener capability. It returns before the candidate can accept.

   --  Start generation 18. On return, it is prepared but cannot accept.
   Coordinators.Start_Upgrade
     (Item       => Manager,
      Executable => Command_18,
      Provision  => Provision_18,
      Authority  => Candidate);
   Assert_Generation (Candidate, 18);

3. Authorize the controlled overlap.

Begin_Canary grants admission and waits for the application readiness hook. Generation 17 remains active during this step.

   --  Authorize canary admission and wait for application readiness.
   Coordinators.Begin_Canary
     (Item      => Manager,
      Authority => Candidate);

4. Promote or cancel from explicit evidence.

Promotion commits generation 18 and drains generation 17. Cancellation retires generation 18, retains generation 17, and requires an immediate compensation decision. The demo accepts Nothing_To_Do and Compensated as completed outcomes; it sends any other result to reconciliation.

   --  Apply deployment policy to the observed canary evidence.
   if Canary_Evidence_Satisfies_Policy then
      --  Commit generation 18, then revoke and drain generation 17.
      Coordinators.Promote (Manager, Candidate);
   else
      --  Revoke and drain generation 18, then return compensation outcome.
      Coordinators.Cancel (Manager, Candidate, Compensation);
      if Compensation not in
        Generations.Nothing_To_Do | Generations.Compensated
      then
         Reconcile_Cancellation (Compensation);
      end if;
   end if;

5. Verify continued service, then shut down.

The mini showcase serves one verification request from the selected generation. A real service waits for its application shutdown request at this point.

   --  Serve a verification request before the test requests shutdown.
   Serve_Until_Test_Requests_Shutdown;

   --  At service shutdown, drain or fence every managed generation.
   Coordinators.Shutdown (Manager);
end Guide_Process_Upgrade;

The setup fragment and five story fragments form the complete main program. The extraction script joins them in numeric order with an empty separator. The two server artifacts are separate programs, shown next.

Build two server artifacts.

Both artifacts use the same generation agent and structured server. Their source differs in one observable behavior: generation 17 replies with 1, while generation 18 replies with 2. The coordinator launches both executables by name.

with Process_Generation_Demo;

procedure Process_Generation_Agent_V1 is
begin
   --  The generation 17 artifact replies with the byte "1".
   Process_Generation_Demo.Run (Character'Pos ('1'));
end Process_Generation_Agent_V1;

with Process_Generation_Demo;

procedure Process_Generation_Agent_V2 is
begin
   --  The generation 18 artifact replies with the byte "2".
   Process_Generation_Demo.Run (Character'Pos ('2'));
end Process_Generation_Agent_V2;

The promotion run must receive 2 after the cutover. The cancellation run must still receive 1. These checks confirm that the selected executable produces the expected reply in each branch.

with Ada.Streams;

package Process_Generation_Demo is
   procedure Run (Version : Ada.Streams.Stream_Element);
end Process_Generation_Demo;

with Flyology;
with Flyology.IO.Connections;
with Flyology.IO.Sockets;
with Flyology.IO.Structured_Servers;
with Flyology.Process_Generations;
with Flyology.Process_Generations.Agents;
with Flyology.Process_Generations.Messages;

package body Process_Generation_Demo is
   procedure Run (Version : Ada.Streams.Stream_Element) is
      package Connections renames Flyology.IO.Connections;
      package Generations renames Flyology.Process_Generations;
      package Messages renames Flyology.Process_Generations.Messages;
      package Sockets renames Flyology.IO.Sockets;

      use type Messages.Nonzero_U64;
      use type Messages.Topology_Digest;

      type Handler_Context is limited record
         Reply : Ada.Streams.Stream_Element := Version;
      end record;

      procedure Handle
        (Context      : in out Handler_Context;
         Connection   : in out Connections.Connection;
         Peer         : Sockets.Endpoint;
         Cancellation : not null access Connections.Cancellation_Token)
      is
         Request : Ada.Streams.Stream_Element_Array (1 .. 1);
      begin
         Connection.Receive_Exactly
           (Request, Timeout => 30.0, Token => Cancellation);
         Connection.Send_All
           ([1 => Context.Reply], Timeout => 5.0,
            Token => Cancellation);
         pragma Unreferenced (Peer);
      end Handle;

      package Structured is new Flyology.IO.Structured_Servers
        (Handler_Context => Handler_Context,
         Handle          => Handle,
         Handler_Model   => Flyology.Native_Task);

      type Application_Context is limited record
         Handler   : aliased Handler_Context;
         Server    : aliased Structured.Server (Capacity => 4);
         Provision : Messages.Provisioning_Data :=
           (Application_Signature => 1,
            Topology_Schema       => 1,
            Topology_Epoch        => 1,
            Digest                => (others => 0),
            Role                  => Generations.Canary_Safe);
      end record;

      procedure Prepare
        (Context : in out Application_Context;
         Data    : Messages.Provisioning_Data) is
      begin
         if Data.Application_Signature /= 16#F1# or else
           Data.Topology_Schema /= 1
         then
            raise Program_Error with "incompatible image provisioning";
         end if;
         Context.Provision := Data;
      end Prepare;

      procedure Run_Server
        (Context  : in out Application_Context;
         Listener : in out Sockets.Socket_Type;
         Role     : Generations.Candidate_Role) is
      begin
         pragma Unreferenced (Role);
         Structured.Serve
           (Context.Server, Listener, Context.Handler,
            Drain_Timeout => 2.0);
      end Run_Server;

      function Ready
        (Context : Application_Context;
         Data    : Messages.Provisioning_Data) return Boolean
      is
         Current : constant Structured.Snapshot :=
           Structured.Current (Context.Server);
      begin
         return Data.Topology_Epoch = Context.Provision.Topology_Epoch
           and then Data.Digest = Context.Provision.Digest
           and then Current.Accepting
           and then Current.Failures = 0;
      end Ready;

      procedure Request_Stop (Context : in out Application_Context) is
      begin
         Structured.Request_Shutdown (Context.Server);
      end Request_Stop;

      procedure Promoted
        (Context   : in out Application_Context;
         Authority : Generations.Upgrade_Handle) is
      begin
         pragma Unreferenced (Context, Authority);
      end Promoted;

      function Compensate
        (Context   : in out Application_Context;
         Authority : Generations.Upgrade_Handle)
         return Generations.Compensation_Result is
      begin
         pragma Unreferenced (Context, Authority);
         return Generations.Nothing_To_Do;
      end Compensate;

      package Agent is new Generations.Agents
        (Application_Context => Application_Context,
         Prepare             => Prepare,
         Run_Server          => Run_Server,
         Ready               => Ready,
         Request_Stop        => Request_Stop,
         Promoted            => Promoted,
         Compensate          => Compensate);

      Context : Application_Context;
   begin
      Agent.Run (Context, Ready_Timeout => 2.0, Drain_Timeout => 3.0);
   end Run;
end Process_Generation_Demo;

with "../../flyology.gpr";

project Process_Upgrade_Guide is
   for Languages use ("Ada");
   for Source_Dirs use ("src");
   for Object_Dir use "obj";
   for Exec_Dir use "bin";
   for Main use
     ("guide_process_upgrade.adb",
      "process_generation_agent_v1.adb",
      "process_generation_agent_v2.adb");

   package Compiler is
      for Default_Switches ("Ada") use
        ("-gnat2022", "-gnatwJ", "-gnatwa", "-gnatyg");
   end Compiler;
end Process_Upgrade_Guide;

./scripts/check-process-upgrade-guide-example.sh extracts every displayed file and verifies each file's complete, unique numeric order. It decodes HTML entities, joins fragments with an empty separator, builds the three executables, and runs both decisions. No Ada source is copied from a hidden fixture.

Control external effects during the canary.

The canary may accept real connections. It must therefore constrain its external effects, not only its traffic volume. Select an exact Candidate_Role.

The application defines whether it can compensate an effect. Compensatability is not another role. The compensation hook must use transaction authority and report an exact Compensation_Result.

Listener duplication does not provide weighted routing or an exact canary share. Use an external proxy or a separate listener when weighted routing is required.

Revoke, drain, compensate, then retire.

Cancel synchronously revokes and drains the candidate. During a canary, it calls compensation after quiescence. The result distinguishes success, failure, and uncertainty.

A Flyology.Cancellation.Token is a one-shot cancellation source. Another task may request the token while Start_Upgrade or Begin_Canary runs. That task must not call another coordinator operation concurrently.

After the coordinator fences the interrupted candidate, the operation raises Operation_Cancelled. A token can interrupt only Start_Upgrade or Begin_Canary. After Promote begins, Cancel cannot reverse it. A later failure produces the Rollback_Required phase.

Reverse by starting a new generation from the prior artifact.

Promotion changes which generation is active. The coordinator retains the prior launch description and provisioning data so that it can start a new replacement generation.

This guide calls the operational procedure reversal. The public API and lifecycle phase retain the exact terms Rollback_To_Previous and Rollback_Required.

Rollback_To_Previous retires every generation from that promotion that the coordinator still owns. A completed promotion normally leaves only the current active generation. An uncertain promotion can leave more than one generation under coordinator ownership. The procedure then starts a new generation from the retained prior artifact and provisioning data.

Observe authority, admission, and drain separately.

Record evidence for each upgrade decision.

• Authority: coordinator id, upgrade id, and candidate generation.
• Generations: active and candidate generation identifiers.
• Identity: artifact identity, topology epoch, and topology digest.
• Lifecycle: phase, admission state, and drain duration.
• Effects: compensation outcome and any external reconciliation record.

A PID identifies an operating-system process. It is not sufficient transaction authority.

Classify failure before choosing recovery.

After the phase becomes Rollback_Required, take a serialized Snapshot. If its Rollback_Available field is true, the caller may select Rollback_To_Previous. Otherwise, use Shutdown or an external replacement procedure. A forward repair is external; it is not another coordinator command.

Verify the protocol within its limits.

• The protocol manages one listener escrow, one active generation, and at most one candidate generation.
• Coordinator operations are synchronous. One caller must serialize them and ordinarily run them from a native task.
• Duplicated listeners provide kernel-mediated admission, not weighted routing or an exact canary share.
• The application hook decides readiness. Flyology validates the returned topology epoch and digest.
• Drain depends on application tasks reaching their structured stopping points.
• Compensation can be incomplete or uncertain. The application must reconcile durable effects.
• The protocol does not transfer active connections, shared heap objects, or Ada runtime state.
• Flyology is experimental. Validate the exact host, compiler release, service protocol, and failure model.

Use this mechanism only if all these conditions hold:

• The listener can remain in stable escrow.
• Each generation can reconstruct its topology.
• The application can classify and fence external effects.
• Structured servers can revoke admission before draining.

Otherwise, use a stop-and-restart procedure or an application-specific migration protocol.