From AST to Bytecode: Making HHY's Default Engine 2.7× Faster

A few days ago, I published Optimizing HHY's AST Interpreter Without Switching to Bytecode. My position was deliberate: first remove repeated work with a resolver, static slots, lightweight call frames, and escape-safe reuse. I did not want to rewrite the runtime simply because a language implementation was expected to have a VM.

HHY v1.3.10 now executes Bytecode by default.

This is not a reversal. It is the next step in the same engineering rule: introduce a new execution layer only when profiles and fixed workloads show that AST dispatch has become a cost worth addressing—and keep the AST as the semantic authority afterward.

The final CI evidence says that Bytecode's wall-time ratio to AST is 0.3695 on the 1M Core Flow CPU workload, making it about 2.71× faster. The ratio is 1.0088 for a short task, effectively even, and 1.0207 for a sustained JSON workload, about 2.1% slower. Those differences matter more than a blanket claim that bytecode is faster: they show that the bottleneck changes with the workload.

I changed the AST's responsibility; I did not delete it

The HHY front end did not split when the VM arrived. Both execution paths share the same lexer, parser, checker, and resolver:

UTF-8 Source
  → Lexer / Parser
  → Checker / Resolver
  ├→ AST Evaluator
  └→ Bytecode Compiler → Verifier → Execution Plan → VM

The complete AST evaluator remains available explicitly:

hhy run --engine ast script.hhy
HHY_ENGINE=ast hhy run script.hhy

I treat it as a permanent semantic oracle, not legacy code waiting to be removed. Every complete fixture and practical project runs independently under AST and Bytecode, comparing stdout, stderr, exit status, diagnostics, resource limits, and cancellation behavior.

Maintaining two paths has a cost, but it gives optimization a precise boundary. If they disagree, I start by treating it as a VM bug rather than reinterpreting the language specification. The optimizer does not get to define semantics.

Bytecode is not the AST under a new name

HHY's Bytecode Compiler lowers every AST node into an explicit opcode while preserving its source location, child and subtree shape, constant reference, and the local-slot metadata already produced by the resolver.

hhy bytecode examples/00-hello.hhy

This command compiles, verifies, and disassembles without executing the result. Disassembly is not a showcase feature to me; it is a governance mechanism. When the runtime selects a path, I need to see exactly what the compiler emitted.

Bytecode converts tree-shaped dispatch into a denser, more sequential, pre-verifiable execution representation. The evaluator no longer needs to rediscover structure at each AST node; the VM follows prepared instructions and operand shapes.

It does not bypass the existing runtime. Closures, exceptions, GC, cancellation, streams, effects, and resource limits still have one owner in the shared runtime. The new layer changes how execution reaches an operation; it does not create a second Value, Stream, or Error model.

The Verifier was a prerequisite for making Bytecode the default

It is tempting to trust in-memory bytecode because the compiler that produced it lives in the same process. That assumption does not survive fuzzing, memory damage, or a possible future cache format.

An independent Verifier therefore follows the Compiler. It rejects:

• unknown opcodes and invalid constant references;
• malformed trees, excessive nesting, and non-canonical HALT;
• out-of-range slot metadata;
• invalid Stream Kernel type flow and stack shape;
• instructions that follow a terminal RETURN.

Execution planning also proves bounded instruction, frame, and operand shapes before evaluation. The Runtime cannot bypass Compile, Verify, and Prepare. This internal boundary is independently versioned and protected by static governance tests.

I was willing to make Bytecode the default not when it first ran a benchmark, but when malformed inputs failed closed, the AST rollback remained available, and both engines had continuous differential evidence.

From general Bytecode to Stream Kernels

HHY is a Flow-first systems scripting language. Optimizing general expression dispatch while ignoring common Stream composition would leave much of the opportunity untouched.

Before v1.3.7, the Runtime already contained a small Stream Int fast path, but its selection depended on magic kinds that were difficult to explain and verify. Subsequent releases tightened the design step by step:

1. v1.3.7 replaced magic kinds with named operations, centralized metadata, and stable fallback reasons;
2. v1.3.8 moved expression-shape recognition from the Runtime into the Compiler and emitted versioned Stream Kernel IR;
3. an independent Verifier checked opcodes, type flow, stack shape, constants, and RETURN;
4. the Runtime executed only verified kernels, while unknown or dynamic closures fell back losslessly to general Bytecode;
5. v1.3.9 made ordinary execution and profiling use the exact same optimization decision.

Fallback is not a performance failure in this design. It is a correctness mechanism. The Compiler specializes only what it can prove; general Bytecode preserves complete semantics everywhere else.

Performance: where it wins and where it does not

The final release-candidate gate used paired, interleaved fresh-process wall-time samples, executing the same source under AST and Bytecode.

Core Flow generates 0..999999, multiplies each value by two, retains values divisible by three, applies stable distinct, materializes, and counts. It magnifies expression dispatch and Stream processing, which is where Bytecode and verified kernels deliver a clear gain.

A short task is dominated by process startup and fixed initialization, leaving little for the execution engine to change. The JSON workload includes parsing, maps, and memory operations, where Bytecode currently shows no advantage. I kept both results visible because they tell me that the next phase should come from profiles rather than from adding opcodes indiscriminately.

Profiling itself is gated. Final overhead was 1.0269×, or about 2.786 ms, below the 1.35× and 12 ms limits. Profile JSON schema 2 records selection reasons, Kernel versions, and actual opcodes, then attributes CPU and Heap hotspots to the path that really executed.

A same-machine comparison with five other runtimes

To place the result in a more intuitive context, I compared Go, Lua, PHP, Java, HHY, and Python on the same 1M CPU/materialization workload on macOS 26.6.2 arm64. Every implementation returned 333334. Each of two rounds used two warmups followed by seven timed fresh processes per runtime in a deterministic shuffled order. Go and Java compilation was excluded; runtime startup was included.

On this workload, HHY is faster than Python and near PHP and Java, while remaining materially slower than Lua and Go. This is not a language ranking, and it says nothing general about file, HTTP, process, or crawler workloads. It shows only that, on a fixed CPU and materialization task, HHY's default Bytecode Runtime has entered a comparable range while retaining obvious room to improve.

I care more about publishing the method and all 84 samples than selecting the best single number for a headline.

Why I did not add a Bytecode cache

Once bytecode exists, .hhyc, an in-process cache, or externally precompiled artifacts look like the natural next feature. A cache, however, brings invalidation, compatibility, security, and atomic-write requirements. Completeness is not evidence of need.

v1.3.10 measured before deciding. Five fixed real workloads used 21 paired fresh processes each. Compile-and-verify medians were only 0.004–0.012 ms, or 0.0078%–0.1341% of cold Bytecode wall time. They reached neither side of the joint admission threshold: at least 1 ms and at least 20% of cold wall time.

HHY therefore currently has no:

• in-process Bytecode cache;
• disk .hhyc cache;
• external precompiled-Bytecode loader;
• public Bytecode ABI.

This is not a missing checkbox. It avoids trading almost invisible savings for a large compatibility and attack surface. If future data crosses the threshold, a cache must bind source and recursive dependency hashes, HHY / Bytecode / Kernel versions, compiler features, the target, and security policy. Even a hit must pass checksum validation, bounded decoding, the full Verifier, and execution-plan verification.

From “not yet” to default Bytecode

Looking back, the meaningful question was never whether AST or Bytecode was more advanced. It was whether each step answered the most real problem at the time.

In v1.1.1, the major costs were repeated name lookup, general environment allocation, and frame lifetime. The resolver and slots produced about a 3.39× improvement on the same Fibonacci(30) benchmark. That work also prepared stable slot and scope metadata for later lowering.

Once the repeated semantic work was removed, profiling exposed the remaining dispatch cost more clearly. Bytecode then became a measured engineering choice instead of an architectural wish.

My conclusion has not changed:

Turn semantic proof into a shorter path first, then let data decide whether a new execution layer is justified.

HHY v1.3.10 defaults to Compiler- and Verifier-checked Bytecode, with the AST permanently retained as the semantic oracle and emergency fallback. CPU-heavy flows already gain substantially; short tasks and JSON paths still tell me to stay disciplined.

A VM is not finished when it can execute instructions. It is finished when it can run faster while keeping every language promise explainable, reversible, and verifiable.

References

• HHY Language Health Report
• HHY Language CLI: AST and Bytecode
• HHY Language source
• Optimizing HHY's AST Interpreter Without Switching to Bytecode