Keep native addresses out of stored bytes.
A stored Flyology data structure contains fixed-width offsets, slot indices, generations, counters, hashes, state fields, and byte payloads. It contains no System.Address or Ada access value. A process-local leaf view, such as Vectors.View, may cache the native base and validated field addresses, but those values are not written to the region.
This separation allows the same physical bytes to be mapped at different virtual addresses. Each participant attaches a Regions.View to its own base and then attaches the leaf view at the same stored offset. Leaf operations resolve stored relationships through that local view.
- Stored metadata
- Every leaf records a 64-bit magic, layout version, schema, extent, configuration, lifecycle state, and initialization epoch.
- Attachment
- The expected identity and geometry must match. Null, truncated, misaligned, overflowing, incomplete, stale, or corrupt layouts fail before payload access.
- Payload types
- Persisted elements use explicit byte-array-backed representations. An arbitrary Ada private type may contain hidden access values or compiler metadata and is not accepted as a persistable element.
- Allocation
- Bounded leaves keep fixed capacity. Dynamic leaves can replace payload allocations inside a fixed managed arena, but no leaf grows or remaps the backing region. When the containing shared segment exhausts its byte extent, the application can quiesce every leaf and prepare a larger replacement segment while preserving stored offsets and generations.
Choose the leaf by ownership and access pattern.
Byte_Strings.View- A bounded variable-length byte sequence. Ordinary operations share one persisted guard.
Vectors.View- A bounded sequence generic over an immutable element adapter. Indexing is one-based at the public API.
Hash_Maps.View- Immutable typed keys and values in a bounded open-addressed table. Capacity is a power of two.
Arenas.View- A generic fixed managed extent using one compile-time allocation algorithm.
Allocation_Pools.Adaptive.View- A bounded table of arena-backed typed slab chunks. The table grows by chunks while the mapped arena stays fixed.
Dynamic.Byte_Strings.View- A byte sequence whose payload capacity grows through an arena allocation.
Dynamic.Vectors.View- An immutable typed vector whose payload capacity grows through an arena allocation.
Dynamic.Hash_Maps.View- An immutable typed map that allocates and rehashes larger tables inside an arena.
Slab_Pools.View- Immutable typed slots returned as generation-stamped handles. Per-slot state separates unrelated accesses and detects stale handles.
Rings.SPSC.View- A bounded immutable typed queue with exactly one producer and one consumer.
Rings.MPMC.View- A bounded immutable typed queue with multiple producers and consumers, per-slot sequences, and bounded claim campaigns.
Envelopes.View- An optional application signature and contract version around one nested leaf. The nested leaf still validates its own identity and geometry.
Storage_Types.Immutable defines the fixed byte-array representation. Its Value owns bytes, while Const_Ref reads published container bytes in place.
Storage_Types.Elements binds that representation once to an application-facing Source, Observed, creator, and observer. Containers invoke those bound operations; callers do not pass codecs or callbacks on each access. The observer must not leak the scoped reference or alter the container or mapping lifetime.
All generic vectors, maps, slabs, and rings, whether fixed or arena-backed, accept this same adapter contract. Storage_Types.Unsigned_64s.Element is the built-in eight-byte scalar adapter.
Regions.View supplies a checked process-local backing view, while Handles.Handle is the fixed-width slot-and-generation representation used by structures such as slabs. Neither package owns storage.
Select an allocator when instantiating an arena.
Arenas is a generic package over one Allocation_Algorithms.Contract instance. That package parameter fixes the stored allocator layout and operations at compile time. The arena facade consists of static renames: it stores no dispatch table, callback, Ada access value, or native address.
Flyology provides Allocation_Algorithms.Buddy, Allocation_Algorithms.Best_Fit, Allocation_Algorithms.TLSF, and Allocation_Algorithms.Slab_Span. Buddy rounds to power-of-two blocks and stores a complete tree outside the managed bytes. Best-fit stores boundary tags in the managed extent and indexes free blocks with an offset-based AVL tree. TLSF uses in-band boundary tags with fixed two-level bitmaps and offset-linked free lists. Slab/span keeps its descriptors and generation tables outside the managed bytes, serving small requests from bitmap slots in fixed runs and larger ones from contiguous runs. All four use one persisted metadata guard shared by all attached views.
- Buddy
- View-local order hints accelerate exact reuse. Release retains split paths; a miss coalesces the complete tree and retries, so worst-case search is linear. Metadata is predictable, but the full tree and power-of-two rounding consume more space.
- Best-fit
- An AVL tree selects the smallest fitting indexed block. Release retains adjacent free blocks. A miss scans for and coalesces adjacent free runs, then retries if the index changed.
- TLSF
- Successful class selection uses a fixed bitmap/list index. Release retains adjacent free blocks; a miss performs a linear physical coalescing pass and retries. Size-class rounding can leave small unusable fragments.
- Slab/span
- Requests up to half the configured run size claim a generation-stamped bitmap slot in a classed run; larger requests reserve whole contiguous runs. Release keeps an emptied slab indexed for reuse; a miss reclaims empty slabs to obtain the run or span it needs. Descriptors sit outside the managed bytes, and the run size is an allocator quantum rather than an operating-system page.
- Compile-time choice
- The package passed as
Algorithmdetermines allocator identity, configuration type, stored metadata, allocation policy, synchronization, and recovery behavior. Allocation_CapabilitiesArenas.Capabilitiesexposes search class, contention scope, metadata placement, split/coalesce behavior, timed operations, and release exclusion as compile-time data.- Runtime instance
- An
Arenas.Viewis attached to one allocator stored in one caller-owned region. Operations receive that view because a package instance is not a singleton allocator. - Stored relationship
- Each allocation returns an
Arenas.Allocation_Handlecontaining an opaque 64-bit token and a nonzero 64-bit generation. A native address is derived only inside an operation. - Dynamic consumer
Dynamic.Byte_Strings,Dynamic.Vectors, andDynamic.Hash_Mapsare generic over the arena package instance. This makes the arena algorithm and handle type part of each container's compile-time contract.
Bind a dynamic container to the arena package
The two generic instantiations below make separate choices. Buddy_Arenas selects the allocation algorithm. Dynamic_Vectors then selects that arena package and the immutable element adapter. No algorithm package or callback is supplied to later operations; the matching process-local Arena_View is still passed so Items can identify the particular stored arena.
package DS renames Flyology.Data_Structures;
package Buddy_Arenas is new DS.Arenas
(Algorithm => DS.Allocation_Algorithms.Buddy);
package Dynamic_Vectors is new DS.Dynamic.Vectors
(Arena_Provider => Buddy_Arenas,
Element => DS.Storage_Types.Unsigned_64s.Element);
Arena_Configuration : constant Buddy_Arenas.Configuration :=
(Usable_Capacity => 1_048_576,
Minimum_Block_Size => 64);
Buddy_Arenas.Create_Or_Attach
(Arena_View, Region, Location => 4_096,
Configuration => Arena_Configuration,
Instance_ID => 16#A8E4_7B19_2C63_D501#,
Result => Arena_Open);
if Arena_Open = DS.Initialization_In_Progress then
-- Another participant owns initialization; this view is detached.
return;
end if;
Dynamic_Vectors.Create_Or_Attach
(Items, Region, Location => 128,
Arena => Arena_View,
Initial_Capacity => 16,
Result => Vector_Open);
if Vector_Open = DS.Initialization_In_Progress then
return;
end if;
Dynamic_Vectors.Try_Append
(Items, Arena_View, 42, Growth);
if Growth = DS.Dynamic.Arena_Exhausted then
-- The fixed managed arena has no fitting free block.
null;
end if;
The dynamic leaves keep a fixed 128-byte header at their original region offset and move only the payload allocation. Growth allocates and initializes a replacement before publishing its handle. A deferred old handle remains recorded until reclamation succeeds, so arena contention cannot make a published allocation unreachable. Dynamic.Growth_Result separates completion, arena exhaustion, and arena contention.
Grow a fixed-size pool by slab chunks
Allocation_Pools.Adaptive is generic over an arena package, an immutable element adapter, Slots_Per_Chunk, and Maximum_Chunks. Its fixed outer extent stores only the bounded chunk table. Try_Allocate scans published slabs first, then uses one nonblocking outer guard to add a chunk from the arena. Once published, unrelated slots use the underlying slab's per-slot claims rather than the outer guard.
The returned Allocation_Pools.Adaptive.Handle records chunk, slot, generation, and pool epoch without a native address. Growth is bounded by both the configured chunk count and arena space. If a process dies while creating a chunk, recovery requires external owner-death and quiescence authority, then exclusive arena reinitialization followed by pool initialization. Reinitializing only the outer table would lose the handles for its earlier chunk allocations.
Open with the same allocator configuration
Arenas.Create_Or_Attach checks whether the stored arena is virgin or already ready. An existing arena must match the selected algorithm identity, every field of the algorithm-specific configuration, and the caller-selected Instance_ID. Dynamic.Vectors.Create_Or_Attach likewise checks initial capacity, adapter identity and geometry, arena algorithm, arena instance, and arena incarnation. A ready object with different creation parameters raises Layout_Error; neither call reinitializes or adapts it.
Use Arenas.Required_Storage with the same configuration when sizing the backing extent. If another participant is currently creating the arena, Create_Or_Attach reports Initialization_In_Progress and leaves the local view detached. This outcome is distinct from incompatibility and from recovery.
Provide another allocation algorithm only when its policy is needed
An allocation implementation instantiates Allocation_Algorithms.Contract. Applications pass that instance to Arenas in the same way as Buddy. The contract defines persisted identity, configuration, and a process-local view. It also defines checked create-or-attach and extent operations, immediate and timed allocation and release, generation-stamped handle validation, bounded payload access, destruction, poisoning, synchronization, and recovery. The generic contract does not make algorithms interchangeable over stored bytes. Attachment fails if the algorithm identity or configuration changes.
Payload read, write, copy, and Arenas.Attach_Allocation do not by themselves serialize release. The allocation-handle owner must exclude release while an access or nested allocation-region view is active. Other synchronization and owner-death rules belong to the selected algorithm; each of the four provided algorithms uses one process-shared metadata guard for allocation and release.
Create or attach through one operation.
The usual entry point is the leaf's Create_Or_Attach. The application first obtains contiguous bytes and attaches a region view. An outer allocation protocol must certify that an exact zero lifecycle means virgin storage. The leaf then either claims and initializes those bytes or validates an already-ready object against the same creation parameters. The example uses a local aligned array; the same calls apply to anonymous, file-backed, persistent, or shared mappings.
with Ada.Streams;
with Flyology.Data_Structures;
with Flyology.Data_Structures.Regions;
with Flyology.Data_Structures.Storage_Types.Unsigned_64s;
with Flyology.Data_Structures.Vectors;
with Interfaces;
procedure Local_Vector is
use type Interfaces.Unsigned_64;
package DS renames Flyology.Data_Structures;
package Regions renames DS.Regions;
package U64s renames DS.Storage_Types.Unsigned_64s;
package Vectors is new DS.Vectors
(Element => U64s.Element);
type Arena_Bytes is array
(Ada.Streams.Stream_Element_Offset range <>) of
Ada.Streams.Stream_Element;
for Arena_Bytes'Alignment use 8;
Arena : aliased Arena_Bytes (1 .. 8_192) := [others => 0];
Region : Regions.View;
Items : Vectors.View;
Open : DS.Open_Result;
Added : Boolean;
Value : Interfaces.Unsigned_64;
begin
Regions.Attach
(Region, Arena'Address, DS.Byte_Count (Arena'Length));
Vectors.Create_Or_Attach
(Items,
Region,
Location => 64,
Capacity => 128,
Result => Open);
case Open is
when DS.Initialized_New | DS.Attached_Existing =>
Vectors.Try_Append (Items, 42, Added);
if Added then
Value := Vectors.Read (Items, Vectors.Length (Items));
pragma Assert (Value = 42);
end if;
when DS.Initialization_In_Progress =>
-- Another participant owns initialization; Items is detached.
null;
end case;
-- Detach local views before Arena ceases to exist.
if Vectors.Is_Attached (Items) then
Vectors.Detach (Items);
end if;
Regions.Detach (Region);
end Local_Vector;
Vectors.Required_Storage computes the complete extent from capacity and the bound element adapter's size and alignment. Reserve that extent at a nonzero leaf-aligned Region_Offset; offset zero is the null sentinel.
Vectors.Create_Or_Attach, like the corresponding operation on every stored leaf, returns an Open_Result. Initialized_New identifies the caller that atomically claimed and completed initialization. Attached_Existing means the ready object passed full identity, adapter, extent, and creation-parameter validation. Initialization_In_Progress means another caller owns initialization; the returned view is detached and the operation does not wait.
Vectors.Try_Append accepts the adapter's ordinary source type. In this example, the source is an Unsigned_64 literal. Vectors.Read invokes the bound observer against published bytes and returns its ordinary observed type. The caller does not pass creation or observation procedures again.
Destroy invalidates a stored object when the leaf's quiescence preconditions hold. Detach only clears process-local state and leaves the stored bytes unchanged.
Attach independently at every mapping address.
Mapping creation remains outside the Data_Structures hierarchy. The first participant uses Create_Or_Attach after attaching its local region. After the application maps the same backing object again, a later participant attaches another region using that mapping's base and attaches the leaf with the same expected configuration. The next section shows the public Shared_Memory layer that can own those bytes.
Regions.Attach (Region_A, Mapping_A_Base, Mapping_Length);
Regions.Attach (Region_B, Mapping_B_Base, Mapping_Length);
Vectors.Create_Or_Attach
(Vector_A, Region_A, Location => 64,
Capacity => 128, Result => Open_A);
if Open_A = DS.Initialization_In_Progress then
return;
end if;
Vectors.Attach
(Vector_B, Region_B, Location => 64,
Capacity => 128);
-- Operations through Vector_A and Vector_B reach the same stored bytes.
-- The process-local native bases may differ.
Reinitialization advances a nonwrapping initialization epoch. Every view attached to the earlier epoch becomes stale and fails before using cached geometry or native addresses. Detach those views and attach again. After the final epoch value, retire the extent rather than wrapping it.
Before unmapping a range, detach every leaf view derived from it and then detach the region view. A leaf view does not retain or extend the mapping's lifetime.
Use named extents inside an owned shared mapping.
The shared-memory segments guide covers backing-mode selection, namespace races, mapping ownership, segment initialization, Unix descriptor handoff, persistence, and ordered teardown. This section focuses on the point where a named extent meets a relocatable leaf.
Flyology.Shared_Memory creates anonymous, named POSIX, or regular-file backing objects, maps them without execute permission or a fixed address, and attaches a Regions.View without exposing System.Address. Linux anonymous objects use memfd_create with immutable size seals and the runtime-supported no-execute seal; access is capability-based, so inode mode bits are not reported as an owner-only boundary. Darwin uses an unpredictable exclusive mode-0600 POSIX shm name and unlinks it before returning. Named objects and files retain explicit unlink ownership.
Shared_Memory.Segments persists a fixed-capacity registry before the payload area. The header records magic, version, application schema, complete extent, registry capacity, maximum name length, slot geometry, and alignment. A lookup checks the hash, exact length, and every name byte. Colliding hashes cannot alias different names.
Shared_Memory.Create_Anonymous (Backing, 1_048_576);
Shared_Memory.Map (Map, Backing);
Segments.Create_Or_Attach
(Segment, Map,
(Schema => 16#4D59_4150_5000_0001#,
Registry_Capacity => 64,
Maximum_Name_Length => 96,
Allocation_Alignment => 64),
Segment_Open);
Segments.Attach_Region (Segment, Region);
Segments.Try_Find_Or_Create
(Segment, "status", Byte_Strings.Required_Storage (256),
Handle, Claim, Named_Open, Failure);
case Named_Open is
when Segments.Created =>
Segments.Claimed_Extent (Segment, Claim, Location, Extent);
Byte_Strings.Initialize (Value, Region, Location, 256);
Segments.Publish (Segment, Claim);
when Segments.Attached_Existing =>
Segments.Resolve (Segment, Handle, Location, Extent);
Byte_Strings.Attach (Value, Region, Location, 256);
when others =>
-- Retry busy/in-progress outcomes at an application scheduling point;
-- handle failure, mismatch, and bounded exhaustion explicitly.
null;
end case;
Exactly one racing caller receives the limited creator claim. Its extent is unavailable to ordinary handles until success publication. A creator can instead publish a nonzero failure code; failed entries remain visible until explicit removal. Removal does not unlink the OS object or destroy application bytes. Later reuse requires a fitting stored reservation and advances a nonwrapping generation, making earlier handles stale.
- Backing lifecycle
- Descriptor close, mapping unmap, named-object unlink, file unlink, and registry removal are separate operations. Closing a descriptor leaves an established mapping live. Keep the descriptor open through
Unlinkand externally exclude namespace replacement; identity comparison and unlink are not atomic, and Darwin POSIX shm exposes no stable identity for comparison. - Initialization trust
- Only a mapping derived from exclusive creation may claim a zero segment lifecycle. Opened and received mappings report initialization in progress instead of repairing an abandoned zero object. Mutable frontier and slot geometry are bounds- and alignment-checked before allocation or reuse.
- Segment growth
Try_Prepare_Replacementsynchronously clones a quiescent segment into a larger, exclusively created virgin mapping. The clone preserves every named extent's offset and generation. Attach the published target in a separate call. Run the copy from a native task unless event-loop occupation is acceptable. All participants must stop before the copy and resume only after application-coordinated cutover.- Creator death
- A dead owner can leave the registry guard or a named extent initializing. Core never steals ownership or claims automatic process-death recovery. Replacement or recovery requires independently established death and quiescence.
- Persistence
msyncandfsyncexpose writeback control for file-backed mappings. They do not make a multi-object application update crash-transactional.- Descriptor handoff
Shared_Memory.Unix_Socketssends exactly one capability withSCM_RIGHTS, establishesFD_CLOEXEC, closes malformed ancillary descriptors, and validates type and exact size before mapping.
Add an application-level version when needed.
Every leaf already validates its own Layout_Identity. An application that assigns additional meaning to the payload can instantiate Envelopes with the leaf's exported Identity, a stable nonzero signature, and an application-controlled contract version.
package Contract_V1 is new DS.Envelopes
(Nested_Identity => Vectors.Identity,
Contract_Signature => 16#A17E_5B31_92C4_770D#,
Contract_Version => 1);
Vector_Extent : constant DS.Byte_Count :=
Vectors.Required_Storage (Capacity => 128);
Contract_V1.Create_Or_Attach
(Envelope, Region, Location => 64,
Content_Extent => Vector_Extent,
Content_Alignment => 8,
Result => Envelope_Open);
case Envelope_Open is
when DS.Initialized_New =>
-- A new envelope deliberately leaves its nested leaf incomplete.
Vectors.Initialize
(Items, Region,
Location => Contract_V1.Content_Location (Envelope),
Capacity => 128);
when DS.Attached_Existing =>
Vectors.Attach
(Items, Region,
Location => Contract_V1.Content_Location (Envelope),
Capacity => 128);
when DS.Initialization_In_Progress =>
null;
end case;
Envelope initialization first marks the nested lifecycle incomplete, publishes the envelope, and leaves nested initialization to the caller. If execution stops between those steps, leaf attachment rejects the incomplete nested object. The 64-bit signature reduces accidental contract confusion; it is not authentication or an integrity checksum.
Use the synchronization model stated by the leaf.
Relocation and concurrency are separate properties. Region attachment and backing lifetime are never internally synchronized. Leaf initialization, attachment, detachment, destruction, and reinitialization require the documented quiescence even when ordinary operations are internally coordinated.
- Guarded containers
- Byte strings, vectors, and hash maps serialize ordinary operations across separate attached views. Immediate operations make one claim and raise
Busy_Erroron contention. - Timed guarded access
- Timed overloads retry through one
Wait_Timeoutand yield between contended observations. A zero timeout permits one immediate attempt. - Slab pool
- Allocation reports an
Allocation_Resultafter a bounded capacity scan; payload access and reclamation claim one slot. Timed overloads retry transient claim contention, but true exhaustion still returnsExhausted. - SPSC ring
- Exactly one producer calls push operations and exactly one consumer calls pop operations. Producer and consumer may use different mappings.
- MPMC ring
- Multiple producers and consumers use process-capable atomics and per-slot sequences.
Tryoperations retain bounded failure outcomes. A new local view may attach while transfers are active: it validates immutable identity and geometry, while deep mutable-sequence validation is reserved for quiescent destruction.
A timed operation starts its monotonic deadline at the first failed claim. It uses delay 0.0 between attempts: a lightweight caller yields its fiber, while a native caller yields its pthread. It does not allocate a persisted wake object or issue a blocking wait syscall on an event-loop pthread.
Separate corruption detection from owner-death policy.
Layouts fail closed on corrupt identity, geometry, state, counters, and handles, but the core packages do not determine whether another task or process has terminated. Poison operations therefore require an independently authorized caller that has already established owner death and the leaf's required quiescence.
- Guard abandoned
- A dead owner can leave a byte string, vector, or hash map locked. Timed access eventually raises
Timeout_Error; it does not unlock the object. A supervisor may poison only after establishing whole-object quiescence. - Slab operation abandoned
- A transitional state affects one slot. Authorized recovery marks that slot poisoned and explicitly recycles it with a new generation. A published live allocation whose returned handle was lost requires an external ownership journal or whole-pool reinitialization.
- MPMC claim abandoned
- Termination after a position claim and before sequence publication can stop later progress. Recovery requires ring-wide quiescence, poisoning, and exclusive reinitialization.
- Mutation exception
- If a guarded mutation may have changed stored bytes before raising, the leaf publishes poison rather than returning the object to ready state.
Exclusive Initialize is the unconditional whole-object recovery path. It invalidates earlier views through the lifecycle epoch; every participant must attach again.
Review the complete lifetime sequence.
- Choose capacity and an element adapter; its identity and geometry contribute to the stored schema and
Required_Storage. - Allocate or map a contiguous region with the required extent and alignment.
- Attach one
Regions.Viewper local mapping. - Use
Create_Or_Attachwhen the allocation protocol certifies zero as virgin; handle all threeOpen_Resultoutcomes. - On later mappings, call
Attachwith the exact expected identity and creation parameters. - Use only the synchronization model documented by that leaf.
- Use explicit
Initializeonly for exclusively authorized destructive creation or recovery. Before reinitialization, destruction, mapping replacement, or recovery, establish the required quiescence. - Detach every leaf view, then detach the region view, before unmapping or releasing the backing bytes.
A failed leaf attachment leaves its output view detached. Is_Attached reports retained local mapping information; it does not promise that a later reinitialization has not made the cached epoch stale.
Know what the relocatable core does not provide.
- The relocatable leaf packages do not own backing storage.
Flyology.Shared_Memoryprovides focused open, map, flush, close, and descriptor-transfer operations without changing leaf layout contracts. - The leaf packages provide no peer discovery, descriptor transfer, access-control policy, wake channel, or automatic process-death recovery.
- Persisted elements are immutable fixed-layout bytes. Applications define their adapter's native layout, validation, stable signature, version changes, and schema migration; arbitrary Ada private types are not persisted directly.
- Timed operations cooperatively retry; they do not provide a pollable completion descriptor.
- Crash consistency of an application transaction spanning several structures requires an application protocol.
- The current implementation is experimental and has not been qualified as a general persistent database or crash-recovery system.
- Use the generated API reference for exact parameter, exception, and synchronization contracts.