Skip to content

Bytes as Lists of Codes

in_ Clausal, a Python bytes value behaves as a list of byte codes — integers in the range [0, 255] — at the logic level. This is the classical Prolog codes representation, and it makes byte sequences participate in unification, term inspection, and DCGs the way character strings do under strings as lists.

Test("bytes is code list") <- (b"abc" is [97, 98, 99])  # nv
Test("with vars") <- (b"abc" is [X, Y, Z], X == 97, Y == 98, Z == 99)  # nv
Test("partial") <- (b"GET" is [71, E, 84], E == 69)  # nv

The motivating use case is DCGs over binary protocols: parsing and building byte streams with phrase//.

The codes model: a byte is an int

str and bytes look alike but are different domains, and Clausal keeps them distinct:

str (chars model) bytes (codes model)
literal "abc" b"abc"
decomposes to ['a', 'b', 'c'] (1-char strs) [97, 98, 99] (ints)
element fixed point? yes — "a"[0] is "a" no — b"a"[0] == 97
partial-term type SegString SegBytes

A str is a list of characters; a bytes is a list of integer codes. This mirrors Python itself: iterating a str yields 1-character strings, but iterating bytes yields ints (list(b"abc") == [97, 98, 99]).

A byte therefore has no fixed point — it decomposes to an int, never to a one-byte bytes:

# A byte has no fixed point: it decomposes to an int, not to a 1-byte bytes.
Test("byte decomposes to int") <- (b"a" is [97])  # nv
Test("byte is not a 1-byte bytes") <- (not (b"a" is [b"a"]))  # nv

The empty bytes unifies with the empty list:

Test("empty bytes is empty list") <- (b"" is [])  # nv

It is unification-equivalence, not conversion

A bytes value stays a bytes object — it keeps .decode(), .hex(), b"a"[0] == 97, every Python method. It merely unifies with its int-code list at the unification layer; nothing is converted. The int list is a logical view, exactly as the character list is for a str. A bytes threaded through unification and recursion stays bytes, so you can still call .decode() / .hex() on a bound result.

Comparison with Prolog

Prolog has always had two string representations: chars (a list of one-character atoms) and codes (a list of integer character codes). Clausal's strings as lists is the chars model; bytes-as-lists is the codes model.

The codes representation is what the Triska / DCG tradition uses for byte and binary work — SWI's get_byte/2, code-list phrase/2, and so on. Integer terminals are exactly what binary-protocol grammars want.

Feature Prolog codes Clausal bytes
Representation List of integer codes Python bytes
Element type int code int in [0, 255]
DCGs over byte streams Works (code lists) Works (bytes subject, code terminals)
Underlying storage Cons cells Compact bytes object
Python interop Requires conversion Native bytes

As with strings, Clausal takes the pragmatic middle path: a bytes behaves as a list of codes at the logic level but remains a compact, interoperable Python bytes object underneath.

Term Inspection

The ISO inspection predicates follow the codes-model cons cell — an int head and a bytes tail (symmetric with the char cons cell for str):

# Term inspection follows the codes-model cons cell: int head, bytes tail.
Test("functor") <- functor(b"abc", '.', 2)  # nv
Test("arg head is int") <- arg(1, b"abc", 97)  # nv
Test("arg tail is bytes") <- arg(2, b"abc", b"bc")  # nv
Test("univ") <- unpack(b"ab", ['.', 97, b"b"])  # nv

So b"abc" has functor '.', arity 2, first argument 97 (an int), and second argument b"bc" (the tail, still bytes).

List Predicates

The list predicates accept a bytes value as a sequence of codes. Element results are int codes; sequence results reconstruct as bytes (input-type-wins):

Test("append codes -> bytes") <- (append(b"GET", b" /", X), X == b"GET /")  # nv
Test("member code") <- in_(71, b"GET")  # nv
Test("length") <- length(b"GET", 3)  # nv
Test("reverse -> bytes") <- (reverse(b"abc", R), R == b"cba")  # nv
Test("take -> bytes") <- (take(2, b"hello", T), T == b"he")  # nv

Type Checks

Test("is_list bytes") <- is_list(b"abc")  # nv
Test("is_codes bytes") <- is_codes(b"abc")  # nv
Test("is_codes int-list") <- is_codes([97, 98, 99])  # nv
Test("bytes is not a str") <- (not is_str(b"abc"))  # nv
Test("bytes is not chars") <- (not is_chars(b"abc"))  # nv

is_list/1 accepts a bytes (it is list-shaped). is_codes/1 is the codes analog of is_chars/1: it succeeds for a bytes or a list of ints in [0, 255]. is_str/1 and is_chars/1 stay false for a bytes — a bytes is not a str, and a code sequence is not a character sequence.

Pattern Matching

A clause-head or body list pattern destructures a bytes argument: the head binds to an int code and the tail stays bytes.

# skip  (illustrative — exercised by tests/test_bytes_patterns.py)
head_tail([H, *T], H, T),

# head_tail(b"abc", H, T)  binds  H = 97 (int),  T = b"bc" (bytes)

The same works as a body goal: b"abc" is [First, *Rest] binds First = 97 and Rest = b"bc". Star variables bind to bytes substrings, preserving the type — exactly as they bind to str substrings under strings as lists. Multi-star patterns ([*A, *B], [*_, X, *_]) match bytes too, and enumerate splits on backtracking just as they do for lists and strings.

DCGs over Binary Protocols

This is the headline use case. A DCG parses a bytes subject with phrase//2,3; terminals are written as integer code lists, and the bytes remainder is preserved as bytes.

# skip  (illustrative — exercised by tests/test_bytes_dcg.py and
#         tests/test_bytes_patterns.py)
g >> ([71, 69, 84, 32])          # [71,69,84,32] is the code list for b"GET "

# phrase(g, b"GET /x", Rest)  succeeds with  Rest = b"/x"  (bytes)
# phrase(g, b"GET ")          succeeds (full consumption)

You can also wrap a bytes literal in the sequence//1 non-terminal to match it as a unit:

# skip  (illustrative — exercised by tests/test_bytes_dcg.py)
header >> (sequence(b"GET "))

# phrase(header, b"GET /index", Rest)  succeeds with  Rest = b"/index"

Under phrase/3, when the subject is a bytes, the residue is bound to a bytes slice — the input type is preserved end to end. (These grammar examples are marked # skip only because the documentation test harness compiles each block in isolation; the behaviour itself is covered by the test suite.)

Promiscuity: in-range int lists unify with bytes

Because byte codes are ordinary integers, any list of ints in [0, 255] unifies with the matching bytes:

# Because byte codes are ordinary ints, any in-range int list unifies with the
# matching bytes — this is intentional (the codes contract is promiscuous).
Test("int list unifies with bytes") <- (b"\x01\x02\x03" is [1, 2, 3])  # nv

This is intentional and symmetric with strings-as-lists (where any list of 1-char strings unifies with the matching str). The contract only fires when a bytes object is actually present on one side of the unification — two plain int lists unify as int lists, and nothing ever spuriously becomes bytes.

Out of Scope

# str and bytes are distinct domains and never cross-unify.
Test("str does not unify with bytes") <- (not (b"abc" is "abc"))  # nv
Test("char list does not unify with bytes") <- (not (b"abc" is ["a", "b", "c"]))  # nv
# An int out of [0, 255] simply fails to unify (it is not an error).
Test("out of range fails") <- (not (b"a" is [256]))  # nv
  • No str ↔ bytes cross-unification. "abc" does not unify with b"abc", and ['a', 'b', 'c'] does not unify with b"abc". They are distinct domains; cross between them explicitly with .encode() / .decode().
  • Out-of-range / non-int elements fail, they do not raise. b"a" simply does not unify with [256] — there is no byte equal to 256.
  • bytearray is not in scope. Only immutable bytes is a logic term; mutable bytearray is not addressed.

Current scope and limitations

The codes contract is wired through the same layers as strings-as-lists:

  • Unification — bytes ↔ int-code list, both directions, element binding, length-mismatch failure, with the bytes type preserved.
  • Pattern matching — single- and multi-star clause-head and body patterns ([H, *T], [*A, *B], [*_, X, *_]) bind int elements and bytes sub-slices (b"abc" is [F, *R] → F = 97, R = b"bc"), enumerating splits on backtracking.
  • List-library predicates — append/3, in_/2, length/2, reverse/2, take/3, drop/3, split_at/4, list_item/3, last/2, etc. accept a bytes value as a code sequence; element results are int codes and sequence results reconstruct as bytes (input-type-wins).
  • Term inspection — functor/3, arg/3, unpack/2 (=..).
  • SegBytes — the partial-byte-string term (the codes analog of SegString): concrete bytes segments alternating with variable-length holes that bind to bytes substrings. See clausal.terms.SegBytes.
  • DCGs / phrase// — bytes subjects with integer-code terminals and sequence//1, with the remainder preserved as bytes.
  • Type checks — is_list/1 accepts a bytes (it is list-shaped), and is_codes/1 is the codes analog of is_chars/1 (succeeds for a bytes or a list of ints in [0, 255]). is_str/1 and is_chars/1 stay false for a bytes — a bytes is not a str, and is a code sequence, not a char sequence.
  • Clause dispatch & first-argument indexing — a bytes-literal clause head matches an int-list caller and buckets with it.

Out-of-scope remains: no str/bytes cross-unification, and no bytearray support (only immutable bytes).

See also