Keep five lifetimes distinct.
The packages divide the problem by responsibility. Flyology.Shared_Memory answers where the bytes come from and who releases them. Shared_Memory.Segments answers how peers agree on what occupies those bytes. The Data_Structures packages answer how stored values remain valid when each process maps the bytes at a different address.
- Namespace
- A POSIX shm name or file path locates a backing object. Unlinking that name is explicit and does not remove a mapping or a registry entry.
- Backing object
- A limited owner closes exactly one descriptor. Closing it does not invalidate mappings already derived from it.
- Mapping
- A limited owner unmaps one process-local virtual range. The operating system chooses its base; public operations never request execute permission or a fixed address.
- Segment registry
- A process-local view validates the persisted header and coordinates exact-name allocation, publication, lookup, removal, and reuse.
- Relocatable leaf
- An arena, string, vector, map, slab, or ring stores offsets, indices, generations, counters, and bytes rather than native addresses.
The normal direction is backing object, mapping, segment, region, leaf. Teardown reverses the borrowed portion: detach the leaf, region, and segment before unmapping. Descriptor close may happen while the mapping is still live.
Choose the backing mode by discovery and persistence.
- Anonymous capability
- Use
Create_Anonymouswhen peers receive a descriptor rather than discover a name. Linux uses a size-sealedmemfd; Darwin immediately unlinks an exclusive mode-0600 POSIX shm object. - Named POSIX object
- Use
Create_Named,Open_Named, orCreate_Or_Open_Namedwhen an application already owns a POSIX name protocol. Size and type must match exactly. - Regular file
- Use
Create_FileandOpen_Filefor explicit filesystem persistence. Opens reject a final symlink where the host supportsO_NOFOLLOWand require a regular file of the exact expected size.
Creation permissions default to 0600. Every acquired descriptor must report FD_CLOEXEC. Linux anonymous storage also reports whether immutable size seals and the runtime-supported no-execute seal were applied. Set Require_No_Execute_Seal when absence of that Linux property must fail closed.
Map once, then attach local views.
with Flyology.Data_Structures.Regions;
with Flyology.Shared_Memory;
procedure Map_Shared_Bytes is
package Regions renames Flyology.Data_Structures.Regions;
package Shared renames Flyology.Shared_Memory;
Backing : Shared.Backing_Object;
Map : Shared.Mapping;
Region : Regions.View;
begin
Shared.Create_Anonymous (Backing, Length => 1_048_576);
Shared.Map (Map, Backing);
Shared.Attach_Region (Map, Region);
-- Attach relocatable structures through Region here.
Regions.Detach (Region);
Shared.Unmap (Map);
Shared.Close (Backing);
end Map_Shared_Bytes;
A Backing_Object owns the descriptor, while a Mapping owns the process-local mapped extent. Attach_Region initializes the relocatable region view. Unmap and Close release those owners independently.
Map borrows the descriptor only for the mapping call. The descriptor can close immediately afterward if the application no longer needs handoff, flush, or explicit unlink. Named objects and files must keep it open through Unlink, because the descriptor supplies the identity used to reject a detectable namespace replacement.
Each mapping has its own native base. Attach a distinct region, segment, and leaf view in each process or mapping. Stored offsets remain the same even when the virtual addresses differ.
Initialize one fixed-capacity registry.
An OS name identifies the complete backing object, not the values inside it. The segment registry is the shared directory that lets every mapping resolve an exact application name to the same offset, reserved length, and generation. The payload at that offset remains a relocatable data structure with its own header and validation rules.
The header persists its magic, layout version, application schema, complete mapping extent, slot capacity, maximum name length, slot geometry, and allocation alignment. Create_Or_Attach checks all of them exactly. Only a mapping derived from exclusive backing creation may claim an all-zero lifecycle.
Config : constant Segments.Configuration :=
(Schema => 16#4D59_4150_5000_0001#,
Registry_Capacity => 64,
Maximum_Name_Length => 96,
Allocation_Alignment => 64);
Segments.Create_Or_Attach (Segment, Map, Config, Segment_Open);
case Segment_Open is
when Segments.Initialized_New |
Segments.Attached_Existing =>
Segments.Attach_Region (Segment, Region);
when Segments.Initialization_In_Progress =>
-- Retry only at an application-selected scheduling point.
null;
end case;
Opened and received mappings that see zero report initialization in progress. They never reinterpret an abandoned object as virgin. A nonzero incompatible or corrupt header raises Segment_Error; attachment is validation, not migration or recovery.
Publish the leaf only after initialization completes.
Try_Find_Or_Create compares the hash, exact length, and every name byte. Hash collisions cannot alias different names. When several participants race one name, the registry designates exactly one creator. That creator receives a limited claim to reserved but unpublished storage; everyone else receives an explicit outcome rather than a partially initialized leaf.
Segments.Try_Find_Or_Create
(Segment, "status", Byte_Strings.Required_Storage (256),
Handle, Claim, Named_Open, Failure);
case Named_Open is
when Segments.Created =>
Segments.Claimed_Extent (Segment, Claim, Location, Extent);
Byte_Strings.Initialize
(Value, Region, Location, Maximum_Length => 256);
Byte_Strings.Assign
(Value, Ada.Streams.Stream_Element_Array'
(1 => 16#72#, 2 => 16#65#, 3 => 16#61#, 4 => 16#64#,
5 => 16#79#));
Segments.Publish (Segment, Claim);
when Segments.Attached_Existing =>
Segments.Resolve (Segment, Handle, Location, Extent);
Byte_Strings.Attach
(Value, Region, Location, Maximum_Length => 256);
when Segments.Registry_Busy |
Segments.Initialization_In_Progress =>
-- Yield or retry according to application policy.
null;
when others =>
-- Handle failure, mismatch, or bounded exhaustion explicitly.
null;
end case;
The creator calls Claimed_Extent before initializing and assigning the leaf. The untimed Byte_Strings.Assign overload replaces its bytes. An opener calls Segments.Resolve to recover the published location and extent.
Call Publish only after the nested leaf is ready. If initialization fails while the creator is still alive, call Publish_Failure with a nonzero application code. The failed name remains visible until explicit removal; no later participant silently overwrites it.
Removal invalidates the registry handle but does not destroy application bytes or unlink the OS object. A later fitting reuse advances a nonwrapping generation, so stale handles fail. The relocatable data-structures guide explains how to select and initialize the leaf placed in each extent.
Replace a quiescent segment instead of resizing it.
A live mapping cannot safely follow an in-place backing resize. Linux anonymous segments also have immutable grow and shrink seals. Flyology therefore prepares a larger replacement segment. The old mapping stays ready while the library copies its validated registry and allocated bytes into a distinct, exclusively created mapping.
Shared.Create_Anonymous (New_Backing, 2_097_152);
Shared.Map (New_Map, New_Backing);
Segments.Try_Prepare_Replacement
(Source => Segment,
Target => New_Map,
Config => Segment_Config,
Quiescence => Segments.Caller_Established_Quiescence,
Result => Migration);
case Migration is
when Segments.Replacement_Ready =>
-- Preparation publishes stored bytes. Attachment separately creates
-- this process's local view and cannot alias the source parameter.
Segments.Create_Or_Attach
(New_Segment, New_Map, Segment_Config, Open);
if Open /= Segments.Attached_Existing then
raise Program_Error with "published replacement did not attach";
end if;
-- Send New_Backing, wait for every peer to attach and acknowledge,
-- then direct participants to the new capability and retire the old.
when Segments.Registry_Busy |
Segments.Initialization_In_Progress =>
-- Quiescence is incomplete. Do not copy or cut over.
null;
end case;
Caller_Established_Quiescence is deliberately explicit. Flyology acquires the persisted registry guard and rejects every unpublished creation claim. However, the segment layer cannot discover application-owned leaf views or detect another process that is mutating a stored object. The caller must stop and acknowledge every participant before making this declaration. Do not migrate if the caller cannot establish quiescence.
The target must be a distinct mapping with a virgin lifecycle from exclusive backing creation. An opened or received mapping is rejected, even when its bytes are zero. The operation release-publishes the initializing lifecycle first, then zeroes the bytes after that field. It retains the configuration, validates each slot, copies through the aligned allocation frontier, changes the persisted extent, and resets the target guard. The ready lifecycle is release-published last.
If a later step raises, the target becomes poisoned and must be discarded. If the creator dies during a page fault or copy, observers see initialization in progress instead of virgin storage.
Try_Prepare_Replacement does not wait when the registry guard is busy, but successful preparation performs synchronous bulk work. It clears the target and copies through the frontier while holding the guard. Cost is linear in those byte ranges, and file-backed pages can fault or be written. Run migration from a native task unless event-loop occupation is explicitly acceptable.
Preparation and process-local attachment are separate on purpose. After Replacement_Ready, call Create_Or_Attach for the target mapping and require Attached_Existing. This prevents a source view from also serving as an output view and makes the cutover point visible.
Offsets and generation-stamped handles retain their values, so existing application metadata can resolve immediately in the replacement. Registry capacity and object geometry do not grow. Reserve enough registry slots initially, and use the new tail for later extents or arena allocations. After cloning, the old and new segments are independent snapshots. Writers must not resume against both segments.
The application still owns peer handoff and cutover. Send the replacement capability and wait for receiver validation and attachment acknowledgments. Then switch all participants and detach the old leaf and segment views. Finally, unmap and close or unlink the old backing. For a file-backed target, flush according to the persistence policy after migration. This operation is not a durable transaction, schema conversion, owner-death recovery, or namespace-atomic replacement.
Report progress from an exec'd worker.
Consider a coordinator that starts a helper process to index a collection of images. The coordinator needs a small status value that survives different virtual addresses and remains valid after it closes the descriptor used for handoff. A named byte string inside an anonymous segment is enough for the stored state.
- Coordinator
- Creates the anonymous backing, publishes
jobs/image-index/statuswith the valuequeued, and sends the descriptor over a connected Unix-domain socket. - Worker
- Receives and maps the descriptor after
exec, attaches the ready named extent, then replaces the value withworkingand finallycomplete. - Observer
- Reads the same byte string through its own mapping. Its native base may differ from both the coordinator and worker bases.
Queued : constant Ada.Streams.Stream_Element_Array :=
(1 => 16#71#, 2 => 16#75#, 3 => 16#65#,
4 => 16#75#, 5 => 16#65#);
Shared.Create_Anonymous (Backing, Segment_Length);
Shared.Map (Coordinator_Map, Backing);
Segments.Create_Or_Attach
(Coordinator_Segment, Coordinator_Map, Config, Segment_Open);
if Segment_Open /= Segments.Initialized_New then
raise Program_Error with "coordinator did not initialize the segment";
end if;
Segments.Attach_Region (Coordinator_Segment, Coordinator_Region);
Segments.Try_Find_Or_Create
(Coordinator_Segment, "jobs/image-index/status",
Byte_Strings.Required_Storage (32),
Status_Handle, Status_Claim, Named_Open, Failure);
if Named_Open = Segments.Created then
Segments.Claimed_Extent
(Coordinator_Segment, Status_Claim, Location, Extent);
Byte_Strings.Initialize
(Status, Coordinator_Region, Location, Maximum_Length => 32);
Byte_Strings.Assign (Status, Queued);
Segments.Publish (Coordinator_Segment, Status_Claim);
else
raise Program_Error with "new status extent was not created";
end if;
Unix_Sockets.Adopt
(Worker_Channel, Worker_Socket, Unix_Sockets.Trusted_Peer);
Unix_Sockets.Send
(Worker_Channel, Backing, Ownership => Unix_Sockets.Transfer);
-- Coordinator_Map and Status remain live after Transfer closes Backing.
Working : constant Ada.Streams.Stream_Element_Array :=
(1 => 16#77#, 2 => 16#6F#, 3 => 16#72#, 4 => 16#6B#,
5 => 16#69#, 6 => 16#6E#, 7 => 16#67#);
Complete : constant Ada.Streams.Stream_Element_Array :=
(1 => 16#63#, 2 => 16#6F#, 3 => 16#6D#, 4 => 16#70#,
5 => 16#6C#, 6 => 16#65#, 7 => 16#74#, 8 => 16#65#);
Unix_Sockets.Adopt
(Coordinator_Channel, Coordinator_Socket, Unix_Sockets.Trusted_Peer);
Unix_Sockets.Receive
(Coordinator_Channel, Segment_Length, Received,
Require_Immutable_Size => False);
Shared.Map (Worker_Map, Received);
Shared.Close (Received);
Segments.Create_Or_Attach
(Worker_Segment, Worker_Map, Config, Segment_Open);
if Segment_Open /= Segments.Attached_Existing then
raise Program_Error with "worker did not receive a ready segment";
end if;
Segments.Attach_Region (Worker_Segment, Worker_Region);
loop
Segments.Try_Find
(Worker_Segment, "jobs/image-index/status",
Status_Handle, Lookup, Failure);
exit when Lookup /= Segments.Registry_Busy;
delay 0.0;
end loop;
if Lookup = Segments.Found then
Segments.Resolve
(Worker_Segment, Status_Handle, Location, Extent);
Byte_Strings.Attach
(Status, Worker_Region, Location, Maximum_Length => 32);
Byte_Strings.Assign (Status, Working);
Index_Images;
Byte_Strings.Assign (Status, Complete);
else
raise Program_Error with "worker status is not ready";
end if;
The owned channel examples call Unix_Sockets.Adopt, then use the owned Unix_Sockets.Send and Unix_Sockets.Receive overloads. The worker calls Segments.Try_Find until the result is Found.
The coordinator can read the updated value through its retained Status view. Shared memory supplies no notification, so the application must choose how the coordinator learns that the value changed. It can poll at an application scheduling point or use a separate readiness channel. This example selects Trusted_Peer because the coordinator created and exec'd the worker; that declaration is application policy, not authentication performed by Flyology. A Linux deployment that receives from an untrusted peer selects Untrusted_Peer, which automatically requires memfd size seals. Process creation must also respect Flyology's post-tasking rule: after Ada tasking starts, a fork child performs only async-signal-safe work before exec or _exit.
Index real image files through one shared segment.
The maintained image-index showcase turns the ownership model into a loaded process workload. A producer continuously writes and queues a bounded set of 2,000 deterministic random P6 PPM files while eight native worker processes analyze earlier files. At the pipeline high-water mark, generation pauses and admits four more workers. After they help restore low pressure and the producer sustains a recovery interval, those workers drain and leave. The original segment remains mapped throughout.
# A terminal keeps the rolling pipeline live until q or Esc stops admission.
./showcases/run_shared_image_index.sh
# Fast regression shape: workers images width height passes index-rounds
NO_COLOR=1 ./showcases/run_shared_image_index.sh 4 64 64 64 2 8
# Optional seventh argument: run exactly three safety epochs.
NO_COLOR=1 ./showcases/run_shared_image_index.sh 8 2000 256 256 128 32 3
The two 64-slot MPMC rings are intentionally smaller than the corpus, so producers and consumers encounter full, empty, and CAS-contention outcomes. Generation and indexing share one pipeline fed by irregular bursts. At the 32-image high-water mark, the producer enters BACKOFF and admits four extra workers to drain work to the 16-image low-water mark. Generation then returns to FLOW, but the extra workers remain through a stable recovery interval before leaving.
Before storing each final image entry, workers also publish through one frequently used hash-map key. A joining worker attaches a distinct local map view, which requires one stable table snapshot under the persisted map guard. If publishers own that guard, the native worker retries Busy_Error after a bounded delay. The coordinator keeps the mapping, registry entries, rings, and hash map live in one segment for the complete session.
The shared gate stores the current safety epoch and the permitted worker-ID limit. While existing workers continue to transfer jobs and results, a joiner maps and validates the segment, resolves the registry, and attaches fresh local ring views. Ring attachment checks the published identity, geometry, and complete extent without scanning mutable claim positions, because producers and consumers advance positions before publishing slot sequences. Deep sequence validation occurs during destruction after quiescence.
The joiner publishes both acknowledgments while the old limit still excludes it. Only then does the coordinator raise the limit and permit that worker to enter the job ring. To remove workers, the coordinator lowers the limit, which each worker samples immediately before dequeue. A worker already analyzing an image finishes and publishes the result before acknowledging departure and detaching. Because a limit update can race with one dequeue, each departing worker can claim one additional image. Removal is not immediate cancellation, and it does not abandon claimed work.
Pressing q or Esc stops further image admission at the coordinator's next check. A generated image that has not entered the job ring is discarded. Temporary workers retire after publishing any image they already claimed, and the coordinator appends one epoch-end marker per remaining worker behind the admitted jobs. Shutdown therefore waits only for admitted work and worker quiescence, not for the rest of the configured batch. Mid-epoch membership changes do not advance the safety identity and do not abandon claimed work.
The interactive display uses the flyology_tui POSIX backend. The backend owns raw mode, the alternate screen, color fallback, typed key input, resize events, and terminal restoration. The coordinator remains the sole owner of pipeline state and renders a declarative view at most once every 75 ms. One input task publishes only resize and stop requests through protected control.
Wide terminals show separate pipeline, shared-segment, and worker panels. A medium layout combines the pipeline and segment data. Smaller terminals retain state, progress, and the stop-admission action while omitting secondary detail. Detached worker slots remain visible when they fit, and the worker panel reports any hidden slots.
When the run ends, the TUI backend restores the terminal. The runner then prints ordinary text with session totals, backoffs, join and leave counts, worker populations, contention, timing, and the unchanged stored segment layout.
The display reports producer backoffs and contention outcomes instead of hiding them behind blocking calls. Its segment panel uses the TUI table component. The table gives each registry entry a name, offset, reserved length, structure kind, and current activity column. Non-terminal and NO_COLOR=1 runs execute one deterministic epoch and produce stable line output. A seventh argument requests a specific finite epoch count. The relocatable data-structures guide explains the ring and map contracts used here.
The coordinator retains each handoff socket until its worker publishes an attached event through the result ring. A successful sendmsg means only that the local kernel queued the descriptor. It does not prove that the worker validated or mapped it. The coordinator closes the socket after the acknowledgment but retains the backing descriptor for later joiners.
At final shutdown, the coordinator closes the descriptor and then sets the worker limit to zero. Each process uses its existing mapping until it detaches. The runner removes the temporary corpus when it exits.
Because mapping, metadata calls, file reads, and pixel analysis are synchronous in this example, the runner prepares Flyology with the native project default. Running that work in a lightweight task could occupy its event-loop pthread. The runner fingerprints runtime inputs and compiler identity, so unchanged preparation and compilation report cache hits. Corpus generation still repeats because it is part of the demonstrated workload.
Use an owned channel for descriptor handoff.
A named-object peer calls Open_Named with the exact expected length, maps the returned descriptor, then calls segment Create_Or_Attach. Because that mapping is not an exclusive creator, it can attach a ready registry but cannot claim zero storage.
For capability handoff, Shared_Memory.Unix_Sockets uses a narrow protocol over a connected Unix-domain stream. Each record contains one nonzero carrier byte and one SCM_RIGHTS descriptor. Use the owned Handoff_Channel overloads by default. They enforce exclusive socket use and reject concurrent calls without waiting. Any operation failure, including transport, framing, ancillary, security, or backing validation failure, permanently poisons and closes the endpoint.
-- Sender_Socket is this process's end of one connected AF_UNIX
-- SOCK_STREAM pair. The application created the pair and authenticated
-- or launched the process at the other end. No other protocol, read,
-- write, or duplicate descriptor may use this endpoint.
Unix_Sockets.Adopt
(Sender, Sender_Socket, Trust => Unix_Sockets.Trusted_Peer);
-- Sender now owns the endpoint; Sender_Socket is invalid after success.
-- Send emits one carrier byte with exactly one SCM_RIGHTS descriptor.
-- Borrow keeps the coordinator's Backing descriptor open. The existing
-- Coordinator_Map is independent and would survive either ownership mode.
Unix_Sockets.Send
(Sender, Backing, Ownership => Unix_Sockets.Borrow);
-- Success means the local kernel accepted the record. It does not prove
-- that the worker received, validated, mapped, or attached the segment.
-- Keep the endpoint connected until an application acknowledgment proves
-- the worker has at least received and validated the record. The image-index
-- showcase keeps it through mapping and segment attachment. Closing earlier
-- can race receiver peer validation, particularly on Darwin.
Wait_For_Worker_Attached;
Unix_Sockets.Close (Sender);
-- Receiver_Socket is the worker's end of the dedicated stream. This
-- example trusts it because the coordinator created and exec'd this worker.
Unix_Sockets.Adopt
(Receiver, Receiver_Socket, Trust => Unix_Sockets.Trusted_Peer);
-- Receiver now owns the endpoint and rejects concurrent channel use.
-- Receive consumes one complete handoff record. Before returning, it checks
-- the carrier and ancillary layout, closes visible extra descriptors, sets
-- CLOEXEC, and requires writable regular backing of this exact local size.
Unix_Sockets.Receive
(Receiver,
Expected_Length => 1_048_576,
Item => Received,
Require_Immutable_Size => False);
-- Map creates a process-local mapping at any address. Closing the received
-- descriptor afterward is safe because Peer_Map owns an independent mapping
-- lifetime. The one-shot channel is independent too, so close it now.
Shared.Map (Peer_Map, Received);
Shared.Close (Received);
Unix_Sockets.Close (Receiver);
-- First attach the segment registry, then derive a relocatable region view
-- from it. Leaf structures borrow Peer_Region, never a native address.
Segments.Create_Or_Attach
(Peer_Segment, Peer_Map, Config, Segment_Open);
if Segment_Open /= Segments.Attached_Existing then
raise Program_Error with "peer did not receive a ready segment";
end if;
Segments.Attach_Region (Peer_Segment, Peer_Region);
-- Resolve the exact persisted name and attach the byte string at its offset.
loop
Segments.Try_Find
(Peer_Segment, "jobs/image-index/status",
Status_Handle, Lookup, Failure);
exit when Lookup /= Segments.Registry_Busy;
delay 0.0;
end loop;
if Lookup /= Segments.Found then
raise Program_Error with "worker status is not ready";
end if;
Segments.Resolve
(Peer_Segment, Status_Handle, Location, Extent);
Byte_Strings.Attach
(Status, Peer_Region, Location, Maximum_Length => 32);
Observe_Status (Byte_Strings.Length (Status));
-- Release borrows from the inside out. The mapping is the last live owner.
Byte_Strings.Detach (Status);
Regions.Detach (Peer_Region);
Segments.Detach (Peer_Segment);
Shared.Unmap (Peer_Map);
Trusted_Peer is an application declaration, not authentication. It permits Darwin and mutable regular files. The application promises that the peer will not exploit ancillary truncation, resize a retained duplicate, or mutate shared open-file-description state. Untrusted_Peer is available only on Linux and requires immutable grow, shrink, and seal seals. In practice, an untrusted handoff requires an exact-size writable sealed object such as Flyology's anonymous memfd. Darwin rejects this policy because XNU can hide installed excess descriptors after a truncated receive.
Choose the handoff pattern from the deployment.
- Coordinator launches a worker
- Create the socket pair before spawning or exec'ing the worker, pass one endpoint through the launch contract, and adopt both ends as
Trusted_Peer. Successful send is only local kernel acceptance. Retain the connected sender until an application acknowledgment proves the worker received and validated the record; the image-index showcase retains it through mapping and segment attachment.Transfermay still close the backing descriptor because existing mappings have independent lifetimes. - Linux service accepts a less-trusted peer
- Authenticate and authorize the connection outside Flyology, then adopt the receiving endpoint as
Untrusted_Peer. Receive automatically requires immutable Linux size seals, so use an exact-size sealed anonymousmemfd. Do not downgrade toTrusted_Peermerely to accept a regular file. Darwin deliberately has no equivalent untrusted mode. - Peers reopen persistent state
- Use
Open_Namedfor a POSIX shared-memory name orOpen_Filefor a stable file path when namespace discovery and persistence are intentional. Validate the exact size, map, and attach the segment registry.SCM_RIGHTSis useful when possession of an already-open descriptor is the capability; it is not required merely because two processes share the same persisted segment.
-- Authentication and authorization still happen before this call.
-- Untrusted_Peer is a receive policy, not an identity check.
Unix_Sockets.Adopt
(Receiver, Receiver_Socket, Trust => Unix_Sockets.Untrusted_Peer);
-- The trust policy automatically requires immutable grow, shrink, and seal
-- seals. The explicit flag may remain False; it cannot weaken the policy.
Unix_Sockets.Receive
(Receiver,
Expected_Length => Segment_Length,
Item => Received,
Require_Immutable_Size => False);
Shared.Map (Peer_Map, Received);
Shared.Close (Received);
The stream protocol requires an owned channel.
SCM_RIGHTS is attached to stream bytes, not to an abstract message object. A stateless Receive (Socket, ...) call cannot exclude another reader or writer. It also cannot prove that a failed stream is safe to reuse. The owned channel enforces those requirements. Raw socket overloads remain available for integrations that provide equivalent ownership and serialization. Retire the socket after any raw-operation exception.
- 1 · Byte association
- Ancillary rights must accompany at least one byte. Flyology sends one fixed nonzero byte and receives exactly one byte. The descriptor belongs to the byte range passed by that
sendmsg, not to the lifetime of the socket. - 2 · Stream boundaries
- One
sendmsgneed not correspond to one arbitraryrecvmsg. Linux and BSD also differ around ancillary boundaries. A one-byte record and a one-byte receive remove partial-record ambiguity for a compliant peer. - 3 · First-byte delivery
- The receive that consumes the range's first byte receives its control data. No
read,recv, or secondrecvmsgmay race the channel, and an endpoint duplicate is subject to the same ban. - 4 · Short reads
- Linux may stop a read at an ancillary boundary; ordinary stream conditions can also produce short reads. A short result is never treated as “the socket is drained,” and Flyology does not mix this channel with edge-triggered application draining.
- 5 · Crossing a range
- If a receiver only partly consumes a larger associated byte range, a later receive can pass its remainder and encounter later ancillary data. Flyology makes every valid range exactly one byte, so a successful receive consumes the complete record.
- 6 · Descriptor count
- Linux permits up to
SCM_MAX_FD(253) descriptors in one rights message. The native receiver reserves aligned control space for 512 descriptors, scans the count actually returned, accepts exactly one, and closes every visible extra. - 7 · Alignment
CMSG_SPACEincludes alignment padding, so a buffer sized “for one” can sometimes expose more than one descriptor. Flyology calculates payload size fromCMSG_LEN(0), counts complete descriptor integers, and never assumes the requested capacity is the returned count.- 8 · Every header
- One receive may contain several control headers. Flyology walks all bounded headers, accumulates every
SCM_RIGHTSdescriptor, closes extras, and rejects unrelated ancillary types instead of ignoring a hidden descriptor later in the buffer. - 9 · Truncation
MSG_CTRUNC,MSG_TRUNC, an overlong returned control length, a malformed header length, or an incomplete descriptor payload is a protocol error. Linux closes rights discarded by control truncation; Flyology also closes every descriptor it can observe.- 10 · Darwin kernel limits
- Darwin has no documented rights-count maximum and has had a truncation path that can leak installed descriptors. It can also report a control-header length beyond the returned control extent. Flyology limits parsing to the actual buffer and reserves space for 512 descriptors as an additional defense. These measures do not make receipt from an untrusted Darwin peer safe.
- 11 · Close on exec
- Linux requests
MSG_CMSG_CLOEXEC. Darwin setsFD_CLOEXECimmediately after receipt. This change cannot be atomic with a concurrentforkandexec. Do not perform work in a post-tasking fork child. - 12 · Hostile descriptors
- The receiver accepts only an
O_RDWR, exact-size regular or POSIX-shm backing descriptor. Sockets, pipes, devices, directories, and read-only files are rejected. Untrusted Linux receipt additionally requires immutable size seals so a retained peer duplicate cannot shrink the mapping and induceSIGBUS. - 13 · Shared open description
SCM_RIGHTSduplicates a reference to the same open file description; it does not create an independent underlying object. Trusted peers must not mutate shared status flags or backing state in ways outside the segment protocol. Shared payload mutation itself remains the intended capability.- 14 · Broken peers
- Linux sends with
MSG_NOSIGNAL; Darwin appliesSO_NOSIGPIPEwhen the channel is adopted. A closed peer therefore raises an Ada exception and poisons the channel instead of terminating the process withSIGPIPE. - 15 · Authentication
- The package does not use
SCM_CREDENTIALSas an identity oracle and does not create or authenticate the socket. Establish peer provenance through an application-owned Unix-socket namespace or inherited endpoint before selectingTrusted_Peer. - 16 · Acceptance is local
- A successful
Sendmeans the local kernel accepted the byte and capability. It does not mean the peer received, validated, mapped, attached, or published anything. Add a separate application acknowledgment when that distinction matters. - 17 · Portability choices
- Flyology uses
SOCK_STREAM, the mode documented for rights passing on both supported hosts. It does not substituteSOCK_SEQPACKET, and it does not use Linux-onlyrecvmmsg; neither removes the need for the portable framing and cleanup rules.
These constraints follow the failure cases collected in Kenton Varda's SCM_RIGHTS notes. The narrow C boundary performs one send or receive and exposes the host's control-message layout facts. Ada owns retries, exact-one framing, closure of every rejected descriptor, endpoint lifetime, nonblocking guard state, poisoning, backing validation, and the resulting limited owners. Stable platform constants and ordinary fixed-signature system calls stay in the platform Ada bodies.
Detach borrowed views before releasing owners.
- Detach every relocatable leaf view borrowing a named extent.
- Detach the region view.
- Detach the segment view.
- Unmap the process-local mapping.
- For a named object or file, call
Unlinkwhile its descriptor remains open if this owner is responsible for namespace removal. - Close the backing descriptor. Explicit
CloseandUnmapare idempotent; finalization is a non-raising fallback and never implicitly unlinks.
File-backed mappings expose two writeback operations. Flush (Map) requests mapping-page writeback through msync; Flush (Backing) requests descriptor-level persistence through fsync. Use both when the application requires those respective effects.
Resume an import from a file-backed checkpoint.
A single-owner importer can keep its last completed record in a regular-file segment. The first run creates the file and publishes a named byte string. Later runs open the same exact-size file, attach the registry, resolve the checkpoint, and continue from the stored value. The application creates the containing state directory before this startup sequence.
Checkpoint_Path : constant String := "state/orders-import.segment";
if Ada.Directories.Exists (Checkpoint_Path) then
Shared.Open_File (Backing, Checkpoint_Path, Segment_Length);
else
Shared.Create_File
(Backing, Checkpoint_Path, Segment_Length, Permissions => 8#600#);
end if;
Shared.Map (Map, Backing);
Segments.Create_Or_Attach (Segment, Map, Config, Segment_Open);
case Segment_Open is
when Segments.Initialized_New |
Segments.Attached_Existing =>
Segments.Attach_Region (Segment, Region);
when Segments.Initialization_In_Progress =>
raise Program_Error with "checkpoint initialization is incomplete";
end case;
Segments.Try_Find_Or_Create
(Segment, "imports/orders/last-record",
Byte_Strings.Required_Storage (32),
Checkpoint_Handle, Checkpoint_Claim, Named_Open, Failure);
case Named_Open is
when Segments.Created =>
Segments.Claimed_Extent
(Segment, Checkpoint_Claim, Location, Extent);
Byte_Strings.Initialize
(Checkpoint, Region, Location, Maximum_Length => 32);
Byte_Strings.Assign
(Checkpoint, Ada.Streams.Stream_Element_Array'(1 => 16#30#));
Segments.Publish (Segment, Checkpoint_Claim);
when Segments.Attached_Existing =>
Segments.Resolve
(Segment, Checkpoint_Handle, Location, Extent);
Byte_Strings.Attach
(Checkpoint, Region, Location, Maximum_Length => 32);
when others =>
raise Program_Error with "checkpoint is not available";
end case;
-- After importing record 25000, encode "25000" as bytes and assign it.
Byte_Strings.Assign (Checkpoint, Encoded_Record_Number);
Shared.Flush (Map, Synchronous => True);
Shared.Flush (Backing);
On restart, Byte_Strings.Read returns the last stored record number and the importer resumes after it. The file path is owned by one externally serialized startup authority in this example. Ada.Directories.Exists is not a multi-process create-or-open protocol.
Apply trust, blocking, and recovery limits.
- Namespace creation, open, unlink, mapping, unmapping, and flush are synchronous metadata or virtual-memory syscalls. They may occupy a lightweight task's event-loop pthread.
Unlinkrequires external exclusion of concurrent namespace replacement because identity comparison and unlink are separate operations. Darwin POSIX shm exposes no stable per-object identity for that comparison.- A dead creator can leave the registry guard or a named extent initializing. Flyology does not detect process death, steal ownership, or infer participant quiescence.
- Recovery requires independently authorized owner-death detection and quiescence. Replacing the complete backing object is often the simplest valid policy.
- The fixed registry does not grow. Capacity, name length, alignment, and schema remain fixed during replacement migration; only a newly created segment's total extent and zero-filled allocation tail may increase.
- Names are exact Ada
Stringbytes. The registry performs no Unicode normalization and treats different byte sequences as different names. - Shared memory does not provide peer discovery, authentication, higher-level permissions, wake channels, schema migration, or durable application transactions.
- The implementation is experimental and currently tested on Flyology's supported 64-bit Darwin and Linux targets.