A language and its compiler

Lobster is a small systems programming language, and a compiler for it, both written in Rust. Today the compiler can read a program, check it, and run it in a built-in interpreter. lobster run examples/hello.lobster prints 55. It does not yet produce a native executable.

The language specification is docs/spec.md. Current status is docs/status.md. Where those files and this manual disagree, the specification and the code win.

1. Introduction

Lobster is a systems language and compiler written in Rust. The lobster binary lexes, parses, resolves names, type-checks, lowers to HIR, MIR, and SSA, runs a small optimizer, and executes main on a reference interpreter. Native code generation is not in this tree.

The ISA for the Tomato32 target lives in the sibling Tomato repository. Lobster does not carry a private opcode table.

1.1. What is landed

Verified from repository sources on 2026-10-01 (docs/status.md):

  • Frontend: lexer (E100–E105), Pratt parser (E110–E112), span-annotated AST, and recovery tests.

  • Names and types: single-file resolver (E200–E203) and a bidirectional checker (E210–E218) with a compile-fail corpus.

  • Middle end: HIR, MIR control-flow graphs, SSA with a verifier, and O0/O1 passes that re-verify and execute identically to the unoptimized MIR.

  • Execution: lobster run examples/hello.lobster prints 55.

1.2. What is not claimed

Borrow checking, generics, const eval, and definite assignment are deferred. There is no native backend. Machine IR, register allocation, and the x86-64, RISC-V, WASM, and AArch64 targets are later milestones. Do not read the roadmap as implementation.

Normative detail stays in docs/language-spec.md, docs/spec.md, and docs/adr/. This manual records the structure those files already prove.

2. Architecture

One lobster binary is built from the workspace crates. Each crate owns one compiler phase. The interpreter is the only execution engine that runs today.

Compiler pipeline What lobster run executes today Compiler pipeline What lobster run executes today 01 Source A .lobster file, with byte spans 02 Lex and parse Pratt parser. Errors E100–E112 03 Names and types Resolve, then check types 04 HIR and MIR Typed IR and a control-flow graph 05 SSA and opt Verifier, then O0 or O1 06 Interpreter Runs main. hello prints 55
Figure 1. Compiler pipeline, from source to the reference interpreter

2.1. Crates

Crate Responsibility

lobster-source

FileId, half-open byte spans, 1-based line and column

lobster-diagnostics

error[E###] with primary and secondary spans, notes, suggestions

lobster-lexer / lobster-parser / lobster-ast

Tokens, Pratt parser, span-annotated AST

lobster-resolve / lobster-sema

Names, then bidirectional type checking

lobster-hir / lobster-mir

Typed IR and a control-flow graph

lobster-ssa / lobster-opt

SSA form, verifier, O0/O1 passes

lobster-interp

MIR execution, including traps

lobster-cli

lobster subcommands

2.2. Milestone tracker

Six of the twenty planned commits are landed. The rest are not started in this repository.

Commit Scope State

01

Workspace, diagnostics, CI, language specification

Landed

02

Lexer, parser, AST, recoverable syntax diagnostics

Landed

03

Name resolution, type checking, compile-fail tests

Landed

04

HIR, MIR, CFG, reference interpreter

Landed

05

SSA and verifier

Landed

06

Optimizer and levels

Landed

07–08

Machine IR and register allocation

Not started

09–13

x86-64, differential testing, RISC-V, WASM, AArch64

Not started

14–18

Tomato baseline and LUT synthesis

Not started

19–20

Toolchain, JIT, explorer, release benchmarks

Not started

3. Design

Decisions below are the ones the code implements. Open questions stay in docs/language-spec.md and are not treated as rules.

3.1. Names

Resolution is single-file. Items, locals, parameters, variants, and the println builtin resolve in that pass. Diagnostics are E200–E203. A dangling variant path resolves to the parent enum so the checker can emit E218 instead of a second name error. See docs/adr/003-name-resolution.md.

3.2. Types

The checker is bidirectional and has no inference variables. Unsuffixed literals adopt their context. There are no implicit conversions. Casts are a table (E212, E216). Match exhaustiveness is E213. The diagnostic range is E210–E218. See docs/adr/005-type-checking.md.

Definite assignment, borrow checking, const eval, and generics are named deferrals, not silent gaps in the checker.

3.3. SSA and optimization

Every run builds SSA and verifies it before execution. Each join has one phi per local. The verifier checks structure, single definition, phi/predecessor agreement, and operand types.

O0 does nothing. O1 folds trivial phis, removes dead phis, propagates copies, and deletes dead pure assignments, to a fixpoint. Traps, calls, prints, and field updates are never removed or reordered. The optimized SSA is verified again and must execute identically to the original on the interpreter.

3.4. Targets

No object code is emitted. ADR 002 sets the first native target as macOS when that work starts. Tomato32, when it exists, will follow Tomato’s ISA files rather than a table checked into this repo.

4. Interfaces

The public interface is the lobster command and the diagnostic codes it prints. Library crates are internal to the workspace.

4.1. Command line

lobster check lexes, parses, resolves, type-checks, and reports E1xx and E2xx diagnostics.

lobster run does the same lowering, then executes main. Useful flags:

Flag Effect

--dump-mir

Print MIR

--dump-ssa

Print SSA

--verify-ssa

Report ssa ok or fail closed

--opt-level 0|1

Select O0 or O1

Other section-4 subcommands exist and exit 2. They are declared stubs, not hidden implementations.

4.2. Diagnostics

Syntax errors are E100–E112. Name errors are E200–E203. Type errors are E210–E218. Runtime failures use TRAP-* or LOBSTER-00x and do not reuse E2xx numbers.

4.3. Files

Sources use the .lobster extension and UTF-8. Whitespace separates tokens and is otherwise insignificant. The reserved words are listed in docs/language-spec.md section 1; async, await, dyn, extern, macro, move, ref, try, and union are reserved so they can be adopted later without breaking identifiers.

5. Programming model

A program is one file with fn main. Integer +, -, and wrap. Division or remainder by zero traps. Shifts mask the count. Floats are strict IEEE. float as int saturates. u32 as char traps. & and trap because the memory model is deferred.

5.1. Example

examples/hello.lobster is the smoke program. lobster run examples/hello.lobster prints 55.

fn fib(n: u64) -> u64 {
    if n < 2 {
        return n;
    }
    fib(n - 1) + fib(n - 2)
}

fn main() {
    let result = fib(10);
    println(result);
}

Six examples under examples/ run: hello, arithmetic, control_flow, strings, structs_enums, and tomato_shapes.

5.2. Checking

lobster check stops after types. It does not execute. Compile-fail fixtures under crates/lobster-sema/tests/compile-fail/ lock the E2xx text.

6. Verification

Correctness is differential and snapshot-based. There is no native binary to compare against hardware.

  • Lexer, parser, and AST recovery have corpus tests and a no-panic bar.

  • The type checker has unit tests plus 14 compile-fail files.

  • SSA construction is checked by the verifier (SSA-verify, fail closed).

  • Optimization is checked by destructuring SSA back to MIR and running both versions on the interpreter. Trapping operations must survive O1.

  • lobster run examples/hello.lobster is the end-to-end smoke check: the process prints 55.

cargo build --workspace
cargo test --workspace
lobster run examples/hello.lobster
lobster run --verify-ssa --opt-level 1 examples/hello.lobster

A failing verifier or a divergent optimized run is a failed build, not a warning.

7. Implementation

The compiler is Rust. Cargo.toml builds one lobster binary from lobster-cli and the phase crates named in Architecture. Stage 0 is not self-hosted.

7.1. Build

cargo build --workspace
cargo test --workspace

The CLI tests live in crates/lobster-cli/tests/cli.rs. Stub subcommands are expected to exit 2.

7.2. Repository map

Path Role

crates/

One crate per phase

examples/

Programs the interpreter runs

docs/language-spec.md

Normative surface syntax and semantics

docs/status.md

Canonical facts and the milestone tracker

docs/adr/

ADRs 001–005

log/

Design journal, also in the sidebar

8. Performance

No throughput, compile-time, or code-size number is claimed. There is no native backend, so there is nothing to time against a machine target.

O1 is specified by which instructions it may delete, not by a speedup. The interpreter exists to compare optimized and unoptimized runs for equality, not to benchmark them.

When commits 19–20 land, this chapter is where those measurements go. Until then every performance cell is TBD (unmeasured).

9. Known limitations

  • No native code. macOS-first is the planned order (ADR 002), not a backend you can invoke.

  • No borrow checker, generics, const eval, or definite-assignment analysis.

  • Memory operators & and * trap. The memory model is deferred.

  • Resolution is single-file.

  • CLI subcommands other than check and run exit 2.

  • Tomato32 code generation is not in this repository. The opcode authority is Tomato’s docs/isa/*.csv and software/assembler.py.

  • Milestones 07 through 20 on the tracker in Architecture are not implemented.

The engineering log records why the language exists. It is not a status page. docs/status.md is.

10. Engineering log

Each note below is the original project log, in chronological order. Open the arrow in the sidebar to jump to an entry.

10.1. 2026-10-01 — Lobster - Tomato’s Programming Language and Beyond

About me

I’m Tyrone Marhguy, a Computer Engineering student at the University of Pennsylvania, class of 2028. Lobster is the systems language and compiler I am writing in Rust.

I grew up between Kumasi and Accra. In 2021, Achimota School denied me admission because I wear dreadlocks as an expression of my Rastafarian faith. Ghana’s High Court ordered the school to admit me; I finished secondary school there, then came to Penn on a full scholarship. BBC Pidgin and Deutsche Welle covered the case.

I design the architecture, build the boards, and write the software. Tomato is a 32-bit computer with a running FPGA implementation and discrete boards in progress. I also build systems software, including a search engine, a compiler, and a database, and I contribute fixes to open-source EDA tools such as LibreLane, OpenROAD, Verilator, and OpenFPGA. That work has included production software at Aragorn AI and battery-management hardware at Vero Electric.

Hardware

Software and systems

Open source

See open-source work for contributions to LibreLane, OpenROAD, Verilator, and OpenFPGA.