From v1.5 to v1.7: Building a Verifiable Optimization Pipeline for HHY

After building HHY Database 1.0, I returned to the Runtime and Compiler with a more concrete definition of optimization.

Executing fewer instructions is valuable. But once a language supports persistent web services, database transactions, lazy Streams, and closures, speed is one requirement among several. Errors must still occur at the original operation. Cancellation must remain reachable. Borrowed resources must be released. Existing programs must preserve their behavior after an upgrade.

From v1.5.0 to v1.7.0, I followed that problem: integrate application resource lifetimes into the Runtime, identify execution costs, then move provable transformations into the compiler.

The release relationship matters. v1.5.0 was released independently. The v1.6 work comprised VM profiling, experiments, and architecture decisions delivered together in v1.7.0. Version 1.7.0 is released, but the new IR compiler, feedback specialization, and scalar replacement remain opt-in. This is not a story of three successive default speedups.

This article follows release commit 8fb0b8f, the current compiler documentation, and the acceptance policy. Performance figures come from existing final-release measurements; I did not rerun benchmarks while writing it.

v1.5.0: Resource lifetimes become an optimization constraint

The Database work in v1.5.0 involved much more than SQL. Pools, remote TLS, transactions, cursors, exact types, cancellation, and Worker isolation all required explicit Runtime responsibilities.

A failed transaction callback must not return an uncertain connection to the pool. An incompletely consumed Stream still needs its cursor closed. Extension resources must not outlive a request scope that has ended. Database retains its independent 1.0.0 version; HHY 1.5.0 provides the Runtime integration it needs.

These constraints carry directly into optimization. Changing call frames touches resource unwinding. Changing allocation touches quotas and GC. Deleting code can remove a cancellation check. Reordering expressions can move the first error or an external effect.

The foundation for the next stage was therefore straightforward: Runtime optimizations must preserve the lifetimes that applications already depend on. Once scripts become services, equal printed output is no longer sufficient evidence of equal semantics.

v1.6: Make the VM's costs visible first

The roadmap calls v1.6 VM Architecture & Runtime. I divided that work into four questions without assuming that more instructions or a different collector would be the answer.

The first challenge is interpreting profiles correctly. HHY_PROFILE_DISPATCH=1 records recursive switch-entry sequences, including opcode and pair/triple counts. These are not necessarily adjacent Bytecode instructions that can be fused. They do not measure pure dispatch CPU time either.

A frequent combination is a candidate for investigation. Turning it into a superinstruction requires proof about evaluation order, error locations, resource checks, and actual execution benefit. Not admitting a superinstruction was a legitimate result of the evaluation.

For me, richer instruction selection means making execution choices explainable through evidence. Increasing the number of opcodes has no value by itself.

Calls and unwind: Fast paths must finish failure paths

Call optimization naturally draws attention to normal returns: one fewer allocation, direct access to a function body, or a reused frame. The difficult cases happen when execution leaves a call early.

The v1.6.1 experiments connect compiler-generated, independently verified call layouts with compact logical call records, exception-region tables, and a versioned unwind action table. Logical records preserve environment, argument, and target roots, together with restoration points for source, contract, effect, trace, and profiler state.

Normal returns, language Errors, cancellation, and host memory-quota jumps all need to restore registered state. Exited records must be cleared so stale references cannot keep completed environments alive.

This does not turn the interpreter into a trampoline. Machine-level calls and returns still use C and the existing setjmp/longjmp machinery. The addition is a logical record that the Runtime can verify, observe, and unwind.

Closures continue to capture a shared Env. Two closures reading and writing the same variable still refer to the same lexical environment. Environments captured by returned closures, closures passed through errors, or lazy Streams cannot be reclaimed into the frame pool. The optional bounded pool reuses only non-escaping environments, with limits of 64 Envs and 64 KiB of cached GC allocation capacity.

I did not simultaneously introduce flat upvalues or tail-call elimination. The former needs additional capture and aliasing proofs; the latter would change current recursion-depth and stack behavior. Each requires its own semantic and performance evidence.

Inline caches, GC, and scheduling need an admission case

The Map inline-cache experiment remembers a previously found slot without retaining a Map or Value. Every entry checks the current Map's count, the key length at that slot, and the complete key bytes before reading the current value.

It caches a checked location hint. Replacing the object, rearranging fields, or updating a value under the same key must never return an old result. Frequently changing sites fall back to generic lookup instead of accumulating unlimited cache shapes.

Observation needs equal care. guard_hit can describe a simulated candidate hit while caching is disabled. cached_reads records actual fast-path reads. Confusing them produces an impressive hit-rate report without demonstrating execution savings.

GC and scheduling decisions follow the same reasoning. HHY retains Boehm conservative GC and bounded process-based parallel with results delivered in input order. A slow first item creates head-of-line waiting; switching to completion order would change semantics.

This evaluation did not establish sufficient production bottleneck and requirement evidence for a generational collector, incremental collector, or async scheduler rewrite. Not admitting those changes today does not establish that they can never help. It means the architecture cost still needs a convincing case.

v1.7.0: Move provable transformations into compilation

The v1.6 work clarified the Runtime boundaries. In v1.7, I began moving repeated proof work into compilation.

The default remains Source → AST → direct Bytecode Compiler → Verifier → VM. The optional path introduces structured HIR and six optimization passes before Bytecode emission. Eligible functions can also carry independently verified integer register MIR plans. The AST engine remains a permanent semantic oracle and explicit fallback.

HIR is a versioned, non-SSA structured representation. It retains node relationships, source locations, local slots, types, constants, and effects. Its CFG describes statement control flow, loop phases, callable ownership, and conservative exception edges. Expressions preserve the language's evaluation order.

This is not a general SSA compiler, LLVM integration, or JIT. Integer MIR currently handles bounded straight-line expressions; other syntax continues through structured Bytecode. Plans are generated at compilation boundaries. The Runtime does not compile machine code on demand.

That scope lets me verify transformations within the existing language without introducing a second model of functions, errors, or memory.

The verifier must be able to reject the compiler

An optimizer labeling a value constant does not make it constant. Labeling an expression effect-free does not prove it can be deleted.

HIR tracks throw, cancel, allocation, and external effect separately. Unknown operations conservatively carry all effect bits. This means there is insufficient proof to delete or reorder them, not that every unknown operation necessarily performs every effect.

The verifier independently reconstructs types, constants, effects, CFG, region ownership, and local-slot provenance. Copy propagation must resolve to the nearest preceding immutable definition without crossing shadowing or accepting self-reference. Every enabled pass is followed by verification, followed by another check before emission and the existing independent Bytecode verifier afterward.

My requirement is that verification must not depend on trusting a pass to have run correctly. The boundary matters only if it can reject an incorrect compiler output.

Six passes, each with a defined proof boundary

The execution order is copy propagation, constant propagation, fold, peephole, unreachable, then DCE. Each pass can be disabled independently.

An expression such as (10 + 20) * (4 + 6) offers a clear scalar proof. An unknown x + 0 cannot be simplified using mathematical intuition alone. Removing a block's final expression can change its implicit return value.

Executed allocations, closure captures, calls, and loop cancellation points remain. Fewer dispatches for successfully eliminated operations are expected, but error locations, exit codes, output, effects, quotas, and cancellation still require differential agreement.

MIR: Feedback proposes; guards authorize execution

Parameter type feedback enables integer MIR for supported functions. The current limits include eight parameters and 64 register instructions per target, with a bounded feedback target table.

A target must receive Int arguments on eight consecutive calls before specialization activates. Every subsequent entry still checks argument types. A type change immediately selects generic execution; eight accumulated type mismatches disable that slot.

Past stability does not prove future stability. Function rebinding must not reuse the old target's plan, so feedback identity is tied to the current chunk and function owner without retaining parameter or closure object references.

The fast path unboxes at entry, uses C-stack int64 registers for intermediate values, then boxes an Int or Bool result. Supported operations include checked integer addition, subtraction, multiplication, remainder, negation, and relational comparison.

When arithmetic fails, deoptimization preserves the failing instruction and operands, then resumes generic error handling at the original source operation. It must not restart the function and repeat allocations that already occurred.

Scalar replacement still preserves allocation reservations

The escape-analysis boundary is intentionally narrow: one local integer List, immediately accessed through a known valid constant index, with no other users. Nested aggregates, Maps, aliases, captures, returned aggregates, and values passed to calls conservatively fall back.

For an eligible shape, the selected element can return directly through registers, removing List element Value writes and reads. Every element is still evaluated in its original order, including elements that are not selected.

The implementation also preserves managed-storage reservations with the original size, scan kind, allocation order, and lifetime through the expression. Physically removing allocation could make a program succeed where it previously hit a memory quota, or change the lifetime observed by GC.

I therefore describe this as scalar replacement and reduced Value traffic, without claiming physical heap-allocation elimination or memory savings. That distinction matters to the credibility of a release note.

Performance admission: Synthetic gains cannot replace real gains

The compiler has a joint budget: compilation at most twice direct compilation plus 500 μs; no growth in candidate instruction count or logical Bytecode storage; Runtime at most 95% of baseline; managed allocation at most 101%. A candidate exceeding the code-size budget falls back entirely to direct Bytecode.

Runtime measurements include prepare time rather than only precompiled hot loops. Logical Bytecode bytes, reserved IR arena bytes, and peak process memory are separate metrics.

The following figures come from the final-release local macOS arm64 budget records, with two warmups per mode and 15 paired samples. Static IR and typed MIR were evaluated separately. They are not points on a version-to-version speedup curve. Ratios mean optimized mode divided by its comparison mode; lower is better.

The static constant loop takes about 25.1% less time; the typed local-List example takes about 20.0% less. Yet the core, JSON, and closure workloads representing more general execution paths do not meet the 5% improvement requirement. Some regress.

The implementation has a measurable effect on specific shapes. That is not enough to make every user pay the additional compilation cost by default. These are also not whole-release comparisons between v1.5.0 and v1.7.0, and they do not establish overall business-application speedups.

Passing four platforms is separate from enabling by default

The final release records include CI and release acceptance for macOS arm64, Linux arm64, Linux x86_64, and Windows x86_64. Correctness, packaging, and compatibility passing do not establish admission for general performance benefits across platforms.

Validation covers AST, direct Bytecode, and IR Bytecode differential execution; all 64 combinations of six passes; independent IR/MIR mutations; sanitizers; fuzzing; quotas; cancellation; and real projects. Recorded evidence includes 10,000 IR field mutations per build and 285 additional typed-specialization execution paths.

During release work, I also corrected measurement and test methods. Profiler timing moved to interleaved paired sampling while retaining every sample and the existing thresholds. Repeated full executions of a large workload used a smaller equivalent case, while the original large cancellation case and real SIGINT validation remained.

These changes make a measurement answer the question it is meant to answer. The final default remains direct Bytecode, with AST available and the new optimizations explicitly enabled.

I want the next optimization to remain switchable

The controls expose the experiment boundaries directly. Here, program.hhy stands for the application script being evaluated:

# Enable the structured IR compiler
HHY_COMPILER=ir hhy run program.hhy

# Keep the IR pipeline but disable every static pass
HHY_COMPILER=ir HHY_COMPILER_DISABLE=all hhy run program.hhy

# Enable typed integer specialization and eligible local List scalar replacement
HHY_COMPILER=ir HHY_FEEDBACK_SPECIALIZATION=1 HHY_SCALAR_REPLACEMENT=1 hhy run program.hhy

# Inspect compilation metrics and reports
HHY_COMPILER=ir HHY_COMPILER_REPORT=1 hhy bytecode --metrics program.hhy

From v1.5 resource lifetimes through v1.6 profiling and Runtime experiments to v1.7 HIR/MIR, I have been building a method that can support further work: identify a cost, define supported shapes, verify independently, retain observations, then decide admission per platform and workload.

An optimization can be implemented, verified, and released while remaining disabled by default. That can be a complete and responsible engineering outcome. I want future HHY improvements to explain where they help, what they cost, and how execution returns to a reliable path when their assumptions no longer hold.

References and related articles

• HHY v1.7.0 release
• v1.7.0 compiler implementation and supported scope
• Compiler admission policy and recorded results
• Call and unwind design, inline-cache evaluation, and GC and scheduling decisions
• From AST to Bytecode: Making HHY's Default Engine 2.7× Faster
• HHY Database 1.0: Building Database Access for Long-Running Applications