The .laz Image Format

A directory, not a black box

The current MVP image is a directory ending in .laz. Its pieces can be inspected independently, copied with ordinary tools, and recovered even when an optional report or index is damaged.

INCOMPLETE and FINALIZED are explicit lifecycle markers. Metadata, the early partition-table snapshot, stored chunk stream, hash map, imaging log, verification result, source identity, bad-sector map, and reports remain separate artifacts.

Current layout

metadata.json: job identity, source facts, imaging statistics, and inspection summary
partition-table.bin: the first MiB of source context, capped by source size
disk.raw: concatenated data or compressed chunk payloads
hashes.dat: logical and stored offsets, lengths, hashes, and storage mode
source-identity.json: evidence used to bind a resume operation to its source
bad-sector-map.dat: ranges that Rescue Mode could not recover
verification.json and reports: factual outcomes produced after a cold reopen

Chunk records explain reconstruction

Format version 2 records an index, logical source offset and length, stored offset and length, SHA-256 of the logical source bytes, SHA-256 of the stored bytes, and whether the payload is data or a verified zero range.

Compressed and zero-elided chunks can be reconstructed independently. Older uncompressed records remain readable, allowing the format to evolve without making early images disposable.

Lifecycle rules in ArcoBASIC

ArcoBASIC states the conditions for promoting an image from incomplete work to a finalized artifact. Filesystem mechanisms remain behind the service boundary while the meaning of finalization stays readable to everyone.

ArcoBASICArcoBASIC communication example: image finalization gate

' Finalization is a claim that every required artifact is complete.
FUNCTION CanFinalize(image)
    IF image.ExpectedBytes <> image.LogicalBytes THEN
        RETURN {"Ok": FALSE, "Reason": "LOGICAL LENGTH MISMATCH"}
    END IF

    IF image.ChunkCount <> image.HashCount THEN
        RETURN {"Ok": FALSE, "Reason": "CHUNK MAP INCOMPLETE"}
    END IF

    IF image.MetadataClosed == FALSE THEN
        RETURN {"Ok": FALSE, "Reason": "METADATA STILL OPEN"}
    END IF

    IF image.DataFlushed == FALSE THEN
        RETURN {"Ok": FALSE, "Reason": "STORED DATA NOT FLUSHED"}
    END IF

    RETURN {"Ok": TRUE, "Reason": "READY TO WRITE FINALIZED"}
END FUNCTION

image = {
    "ExpectedBytes": 1000,
    "LogicalBytes": 1000,
    "ChunkCount": 4,
    "HashCount": 4,
    "MetadataClosed": TRUE,
    "DataFlushed": TRUE
}

decision = CanFinalize(image)
PRINT decision.Reason

Recoverability outranks compression

None and Zstandard are implemented compression modes. The format records enough information to validate stored bytes before decompression and validate logical bytes after decompression.

Maximum compression is not the goal. A recovery format should remain explainable, streamable, resumable, and partially useful under failure.

Designing a format that outlives its own decisions

SHA-256 chunk hashes, Zstandard compression, and a directory ending in .laz are choices specific to disk images. The reason those choices were made the way they were, favoring inspectability and survivability over any single optimization, is what a completely different file format could still learn from.

This is the same tradeoff behind log-structured storage formats, database write-ahead logs, and any serialization format that ships with a version field from day one. A format that has to survive being interrupted, partially corrupted, or read by code written years later needs a different set of priorities than one that only has to work once, today, when everything goes right.

Make the pieces of the format independently useful. metadata.json, hashes.dat, and source-identity.json are separate files on purpose, so a damaged or missing report does not take the recoverable payload down with it.
Version the record, not just the file. Format version 2 added per-chunk zero-range flags and stored and logical hash pairs while leaving older uncompressed records readable, so the format evolved without making earlier output disposable.
Gate the claim, not just the write. CanFinalize checks logical length, chunk-to-hash count, closed metadata, and flushed data before allowing INCOMPLETE to become FINALIZED, so the promotion to done is its own explicit decision, not a side effect of the last write returning successfully.
Optimize for the property that matters most, and say so plainly. The page states that maximum compression is not the goal; a format that stays streamable, resumable, and partially useful under failure was chosen over one that is merely smaller.

LEARNING LAYER

Key terms, in plain language

You do not need a systems background to follow the work. These are the specialized terms used on this page.

Disk image: A sector-by-sector copy of storage media saved into a file or another device. Recovery work usually analyzes the image so the fragile original is touched as little as possible.
Chunk hash: A compact fingerprint calculated from one region of data. Lazarus uses SHA-256 hashes to detect whether stored and reconstructed chunks changed.
Zero elision: Omitting the stored payload for a chunk proven to contain only zero bytes while retaining its logical length and hash so it can be reconstructed and verified.
Bad-sector map: A record of source ranges that could not be read successfully. It preserves where evidence is missing instead of silently presenting substituted bytes as recovered data.