LZ11 / LZS wrapper

DS game only γ¨γ‚“γŒγ‚Šγƒœγ‚¦γ‚·γ¨γŠγ—γ‚ƒγ‚Œγͺまほう぀かい The field guide documents the DS game's internals.

Outer LZS wrapper + inner Nintendo LZ11 stream. Used by ~70% of containers. Decoder + encoder in src/archive/lzs.py.

A .lzs file is an outer 16-byte LZS wrapper followed by a standard Nintendo LZ11 compressed stream. Decompressing it usually yields a RESO container.

Outer LZS wrapper (16 bytes)

OffsetSizeTypeFieldValue
0x004char[4]magic"1.00" = 31 2E 30 30
0x049u8[9]reservedZero-filled
0x0D3char[3]tag"sfo" = 73 66 6F
0x10β€”β€”LZ11 payload beginsβ€”

LZ11 stream header (4 bytes)

OffsetSizeTypeField
+0x001u8magic = 0x11
+0x013u24 LEdecompressed_size

Stream body

Repeating blocks of 1 flag byte + up to 8 chunks. The flag byte's MSB-first bits select chunk kind: 0 = literal byte, 1 = back-reference.

Literal chunk (1 byte)

Emit the byte directly to the output buffer.

Back-reference chunk

The first byte's top nibble (n = b1 >> 4) selects one of three encodings:

top nibble nbytes readlengthdisplength range
n > 1 2 (b1 b2) n + 1 ((b1 & 0xF) << 8) | b2 3..16
n == 0 3 (b1 b2 b3) (((b1 & 0xF) << 4) | (b2 >> 4)) + 0x11 ((b2 & 0xF) << 8) | b3 17..272
n == 1 4 (b1 b2 b3 b4) (((b1 & 0xF) << 12) | (b2 << 4) | (b3 >> 4)) + 0x111 ((b3 & 0xF) << 8) | b4 273..65808

After decoding, copy length bytes from output[len(output) βˆ’ disp βˆ’ 1 .. +length] into the output. The copy can read bytes it just wrote (RLE-like overlapping copies).

Decoder reference

# src/archive/lzs.py::decompress_lz11
def decompress_lz11(payload: bytes) -> bytes:
    decompressed_size = payload[1] | (payload[2] << 8) | (payload[3] << 16)
    out = bytearray()
    i = 4
    while len(out) < decompressed_size:
        flag = payload[i]; i += 1
        for bit in range(7, -1, -1):
            if len(out) >= decompressed_size: break
            if (flag & (1 << bit)) == 0:                       # literal
                out.append(payload[i]); i += 1
            else:                                              # back-ref
                b1 = payload[i]; b2 = payload[i+1]; i += 2
                top = b1 >> 4
                if top > 1:
                    length = top + 1
                    disp   = ((b1 & 0xF) << 8) | b2
                elif top == 0:
                    b3 = payload[i]; i += 1
                    length = (((b1 & 0xF) << 4) | (b2 >> 4)) + 0x11
                    disp   = ((b2 & 0xF) << 8) | b3
                else:                                          # top == 1
                    b3 = payload[i]; b4 = payload[i+1]; i += 2
                    length = (((b1 & 0xF) << 12) | (b2 << 4) | (b3 >> 4)) + 0x111
                    disp   = ((b3 & 0xF) << 8) | b4
                copy_from = len(out) - disp - 1
                for j in range(length):
                    out.append(out[copy_from + j])
    return bytes(out)

Encoder

src/archive/lzs.py::compress_lz11() implements a greedy encoder with one-byte lookahead. Limits (per spec):

FormLength rangeDisplacement range
2-byte ref3..0x101..0x1000
3-byte ref0x11..0x1101..0x1000
4-byte ref0x111..0x101101..0x1000

Output is padded to a multiple of 4 bytes (standard). The encoder is not byte-identical to the Nintendo SDK β€” different valid LZ11 streams can decompress to the same bytes. Round-trip decompress(compress(x)) == x is the validation test (passes on every blob tested, 2 KB NCGRs up to 110 KB inner RESOs).

Worked example

First 32 bytes of 2d/buyitem.lzs:

offs  bytes                                              meaning
----  -------------------------------------------------- ----------------------------------
0000  31 2E 30 30  00 00 00 00  00 00 00 00 00  73 66 6F   wrapper magic, zeros, "sfo" tag
0010  11 80 BF 01                                          LZ11: magic=0x11, size=0x01BF80
0014  FF                                                   flag byte (0b11111111 = 8 back-refs)
0015  20 00                                                back-ref #1: n=2, length=3,
                                                                       disp=((0x0<<8)|0x00)=0
                                                                       copy 3 from out[-1]
0017  ... 7 more 2-byte refs ...

Compression ratios (sampled)

FileCompressedDecompressedRatio
buyitem.lzs24,064114,5604.76Γ—
title_pack.ofs inner sub-blob 362,464~88,000~1.41Γ—
global.lzsvaries per entryvaries~3.5Γ—

Source