Optimizing HHY's AST Interpreter Without Switching to Bytecode

HHY Language v1.1.1 is still an AST interpreter.

After lexing, parsing, and checking, the runtime walks the AST directly. There is no bytecode lowering stage and no JIT. For a systems scripting language, this path has useful properties: source locations remain close to execution, errors are easier to explain, and a semantic change does not require a second instruction system to evolve in lockstep.

Direct execution, however, should not mean rediscovering the same facts on every call. Under recursion, closures, and call-heavy workloads, the expensive questions become repetitive: Which lexical scope owns this name? Where should an argument live? Does every call require a general-purpose environment object? Can the frame be reused safely after return?

The goal of this work was intentionally narrow:

Keep the AST interpreter, but remove work that can be resolved ahead of time, addressed directly, or reused safely.

Profile first, rewrite later

Interpreter work can easily begin with an attractive solution: define opcodes, write a compiler, implement a VM, and expect speed to follow. Bytecode is not a free abstraction. It adds an intermediate representation, control-flow rules, debug mapping, stack traces, closure capture, and another compatibility surface.

I prefer to start with a smaller question: where is the current cost?

HHY v1.1.1 includes a profiler:

hhy profile examples/09-profile-algorithms.hhy -- fibonacci 20
hhy profile --cpu examples/09-profile-algorithms.hhy fibonacci 20
hhy profile --heap --format json --output profile.json \
  examples/09-profile-algorithms.hhy fibonacci 20

profile executes the real script. CPU reports include samples and call counts. Heap reports include managed allocations, object counts, peak usage, and live memory after GC. Reports go to stderr by default, leaving script output and exit status intact.

The baseline also quantified the problem. Naive Fibonacci(30) enters an HHY function 2,692,537 times. An early runtime path produced 32,310,598 allocations and about 1.98 GB of cumulative allocation traffic. That was not 1.98 GB live at once; it was millions of calls creating and quickly discarding short-lived objects.

The data separated call overhead into two parts:

1. name resolution and environment lookup repeated inside hot calls;
2. general-purpose environments allocated for calls that needed much less state.

Neither problem required bytecode first. Both required the runtime to stop forgetting facts the language front end already knew.

With the optimized path, the same Fibonacci(30) benchmark—three warmups and twenty fixed, randomly interleaved runs—moved from a 681.27 ms median to 200.79 ms, a cumulative 3.39× improvement. More importantly, the algorithm and all 2,692,537 function calls remained unchanged:

Fibonacci is not HHY's target workload, but it magnifies function-call cost. The result shows that the general runtime path does less work; it does not imply the same end-to-end speedup for file, HTTP, or process-heavy flows.

Resolve pass: move stable decisions before execution

In the original path, an Identifier node carried a name. When the runtime reached total, it searched from the current environment outward until it found a binding, then continued to globals and builtins if needed.

That is semantically correct and mechanically wasteful. The lexical owner of one AST node usually does not change on its ten-thousandth execution.

The v1.1.1 resolver walks the AST before function execution and classifies names where possible:

• parameters receive slots in the current function;
• deterministic locals receive slots;
• cross-function closure captures, globals, and builtins retain the Env path;
• scopes containing dynamic import do not receive a forced static layout;
• identifiers that cannot use a direct slot receive reusable lookup metadata.

This is not merely a string cache. It turns stable lexical relationships into metadata the runtime can address directly.

Identifier("total")
  before: lookup(env, "total")
  after:  slots[3]

The parser still owns structure. The checker still owns semantic constraints. The resolver translates proven scope facts into a faster runtime representation.

Slots: locals do not need repeated name lookup

Once parameters and deterministic locals live in slots[], reads change from a name-based environment walk to an indexed array access.

Local / Param       → slots[index]
Closure / Global    → Env lookup
Builtin             → Env lookup

I deliberately did not force every name into a slot. Captured variables outlive individual calls, globals belong to a shared namespace, and builtins participate in runtime registration. Keeping the compatible Env path for those cases preserves a clear model and avoids complex invalidation rules.

The principle is simple: the fast path serves the common, provable case; the slow path preserves complete semantics. They are two responsibilities, not an unfinished compromise.

Lightweight call frames: a call is not a full environment

Slots answer where values live. The next question is what a call must create.

A function call needs a relatively small core of state: its function, parameter and local slots, a parent or captured environment when necessary, plus runtime state for return and error propagation. Building a general-purpose map-backed environment on every call increases lookup cost and managed allocation.

The hot path now uses a lightweight call frame. The implementation does not introduce a second object model beside Env. Instead, Env carries contiguous Binding capacity, an escaped flag, and a free-list link:

CallFrame
├── function
├── slots[]
├── parent / captured env
└── runtime state

Each call acquires capacity from the resolver's frame_slot_count. Arguments are written in order and deterministic locals then occupy stable slots. A function-body block uses the call frame directly instead of allocating another parent-only block environment. Lookup walks the compatible Env chain only for closures, globals, or dynamic paths.

This does not turn a dynamic language into a static one. HHY values remain dynamic. What becomes stable is the binding location: a value's type may be determined at runtime while a proven local name no longer needs its address rediscovered on every read.

Identifier caches: the slow path can still remember

Not every identifier can use a slot, but that does not mean every lookup must begin from zero.

For Env-based identifiers, the AST node caches cached_env_depth and cached_binding_slot. A later execution first follows that depth and slot, then compares the binding name's length and bytes again. Only a validated name is a hit. If the environment shape no longer matches, the runtime invalidates the cache, performs a full lookup, and rebinds it.

The cache must belong to a semantically stable object and have explicit invalidation. A process-wide map from name to address would incorrectly share results across recursion, modules, and closures. HHY's cache follows the AST node and its scope relationship, not the string alone.

The most dangerous optimization is not one that fails to improve performance. It is one that is faster on most inputs and occasionally reads the wrong binding. A cache hit must never bypass lexical semantics.

Frame pools: decide escape before reuse

Lightweight frames still need initialization. In a call-heavy function, discarding each frame and waiting for GC continues to create allocation pressure.

A pool is the obvious next step, but a frame cannot be reused merely because the function returned. A closure or stream may retain the current environment. The stack call is over; the object lifetime is not. Resetting that frame would make an older closure observe values from a newer call.

HHY therefore gives the frame pool a hard boundary:

non-escaping frame  → reset → pool → reuse
captured by closure → mark escaped → no reuse → GC managed
captured by stream  → mark escaped → no reuse → GC managed

That is why I think of the design as escape-safe reuse rather than a generic object pool. Pooling is the mechanism. Escape classification provides correctness.

What an optimization must not change

The acceptance criterion is not only a faster benchmark. Language promises must remain intact:

• errors still point to original source locations;
• closure capture and lifetime remain unchanged;
• shadowing across locals, modules, globals, and builtins remains unchanged;
• stream laziness, single consumption, and cancellation remain unchanged;
• hhy check, hhy ast, --dry-run, and real execution share one meaning;
• resource limits, exit codes, and unwind paths cannot be bypassed by a fast path.

That calls for regression tests around recursion, deep scopes, shadowing, escaping closures, stream capture, stack traces, and objects that remain alive after GC—not just a timing fixture. The new frame-slots-escape.hhy creates two independent counter closures and mutates their captured slots after the factories return. It also returns a lazy stream that captures a local and is only collected after its function has returned. Its output—11, 12, 101, and [8, 9, 10]—shows that frames were not incorrectly reused and delayed stream evaluation did not observe cleared state.

The full release suite, ASan/UBSan suite, and Multi-API Data Collector self-test also have to pass. An optimizer should be removable without changing program results.

Why HHY does not use a bytecode VM yet

A bytecode VM remains a possible direction, but data should trigger it—not the aesthetic of a roadmap.

After the resolver, slots, lightweight frames, and safe reuse are in place, profiling may still show AST dispatch as the dominant remaining cost. If that cost also matters in real flows, modules, and system-call-heavy workloads, bytecode will have a clear case. The resolver work will remain useful because it already organizes names, slots, and closure boundaries into lowering-friendly facts.

If a script spends most of its time in files, HTTP, processes, or stream operators, cutting AST dispatch in half may barely affect end-to-end latency. A systems scripting language cannot optimize only for Fibonacci.

My current sequence is:

Profile
  → eliminate repeated semantic work
  → direct-address stable bindings
  → reduce call-frame allocation
  → reuse only non-escaping state
  → profile again
  → consider bytecode when AST dispatch becomes the bottleneck

The lesson I am keeping

This work reinforced a useful idea: performance problems are often not caused by too much abstraction. They happen because the system fails to consume facts it has already proved.

If the front end knows the scope of a local, the runtime should not guess again. If a frame does not escape, it does not need a full GC lifetime. If an identifier's environment location is stable, even the compatible path can avoid starting from the root.

The best optimizations do not skip semantics. They turn semantic proof into a shorter execution path.

HHY v1.1.1 still executes the AST directly. That is not a statement that bytecode is unimportant. It means the more valuable step today is to make the AST interpreter repeat less work while keeping source diagnostics, closures, streams, and resource boundaries dependable.

References

• HHY Language CLI reference: AST, profiling, and interpreter evolution
• HHY Language source
• Building HHY Language: From Flow Philosophy to a C Runtime