Over the past few months, I built a systems scripting language from scratch in C: HHY Language.
It is not a new spelling for a handful of shell commands, and it is not a natural-language wrapper around an executor. HHY has its own tokens, grammar, AST, scopes, semantic checker, runtime value model, lazy streams, structured errors, and resource limits. From the moment source enters the lexer to the point where a file is opened, a process starts, or an HTTP request is sent, every step has defined behavior.
The positioning is deliberately simple:
Pipe Everything. Bring files, processes, networks, and structured data into one Flow.
path("./logs")
|> files("**/*.log")
|> flat_map { file -> read_lines(file.path) }
|> where { line -> contains(line, "ERROR") }
|> take(100)
|> save_lines(path("errors.txt"))
This article is not only a syntax tour. It is a retrospective on what actually matters when a language moves from an idea to a frozen v1.0 contract.
Why build another language?
Systems automation does not lack tools. Shell is excellent at connecting processes. Python is excellent for complete programs. jq, awk, and curl each solve a focused class of problems. Friction appears when one task crosses directory traversal, text streams, JSON, HTTP, concurrency, and error recovery: the programmer keeps switching mental models.
I wanted to test whether those objects could be expressed as sources, transformations, and actions—so that a program mostly describes how data moves rather than manually coordinating every step.
source |> transform |> filter |> action
That makes |> more than visual punctuation. It is a composition rule shared by the language, standard library, and runtime:
x |> f => f(x)
x |> f(a, b) => f(x, a, b)
x |> obj.f(a) => obj.f(x, a)
Once that rule is stable, files, processes, HTTP requests, and ordinary collections can share one expression model. A language philosophy becomes meaningful only when it becomes executable semantics.
Freeze semantics before expanding features
The easiest trap in language design is to keep adding keywords and library functions. The harder questions look smaller: When does a newline terminate a statement? Is / division or the start of a regular expression? How do maps and blocks share {}? Does a pipe flatten a nested stream? Is parallel output ordered? Who closes upstream resources after an error?
I made docs/HHY_V1.md the single source of truth. Before expanding the implementation, v1.0 froze a set of decisions: dynamic typing, immutable bindings by default, explicit closures, one call syntax, lazy single-consumption streams, fail-fast errors, request construction separated from send, and bounded ordered parallelism.
The order matters. The specification is not documentation written after the code; it is a constraint applied before changing the code. When an experiment conflicts with the specification, the language decision comes first, then the spec and tests, then the implementation.
From source text to effects
HHY currently executes the AST directly, but it still has a complete language front end:
UTF-8 Source
→ Lexer / Token
→ Parser / AST
→ Checker / Scope & Contract
→ Runtime / Value & Environment
→ Stream / System Effect
Each layer answers a different question. The lexer does not resolve names. The parser does not open files. The checker does not send HTTP. The runtime does not rediscover operator precedence. Clear boundaries make diagnostics more precise and evolution safer.
The lexer: turning characters into located facts
The lexer turns UTF-8 source into tokens. Every HHY token retains its kind, raw source slice, line, and column. Location is not a debugging accessory; it is the evidence used by parser, checker, and runtime diagnostics.
Lexing still requires context. /ERROR/i and total / count both contain /; HHY decides between regex and division by asking whether the previous token can end an expression. Native values such as 10mib, 500ms, and 80% become Bytes, Duration, and Percent tokens instead of strings interpreted later by library calls.
The lexer validates UTF-8, rejects unsupported string escapes, and preserves newlines as tokens. Because a pipeline may continue across lines, blindly discarding whitespace would lose grammar-relevant information.
The practical rule is: tokens should retain enough source truth without taking over semantic work. Once the lexer starts understanding scopes or function contracts, the architecture has already blurred.
The parser: encoding precedence and context
HHY uses a recursive-descent parser. Expression rules build progressively through postfix calls and member access, unary operators, multiplication, addition, comparison, logic, null coalescing, pipes, and assignment.
The advantage is not merely implementation simplicity. Each grammar rule remains visible in code. Maps and blocks both use braces, but appear in different grammatical positions; closures are recognized when a call stage provides the relevant context. It is cleaner to decide there than to invent multiple brace tokens in the lexer.
Error recovery matters as much as successful parsing. Stopping after one missing expression forces users into a slow one-error-at-a-time loop. Continuing without synchronization produces cascades of false errors. HHY synchronizes around declarations, control flow, newlines, and semicolons so one hhy check can report multiple real problems without consuming the rest of the file as part of a broken expression.
Why the AST matters
The abstract syntax tree is the central representation between surface syntax and execution. It discards punctuation that no longer matters while preserving program structure.
processes
|> where { process -> process.memory > 1gb }
|> take(10)
|> print
Conceptually becomes:
Pipe
├── Pipe
│ ├── Pipe
│ │ ├── Identifier "processes"
│ │ └── Call "where"
│ │ └── Closure
│ │ └── Binary ">"
│ └── Call "take"
│ └── Literal "10"
└── Identifier "print"
An HHY AST node contains a kind, its source token, and child nodes. Node kinds cover declarations, control flow, calls, member access, closures, pipes, lists, maps, and literals. The source token remains attached through runtime execution, which allows runtime errors to point back to the expression the user wrote.
The AST creates four important boundaries:
- The parser decides what structure the source represents.
- The checker can inspect the program without executing effects.
- The runtime executes stable nodes instead of parsing text again.
hhy ast, the formatter, snapshot tests, and future optimization passes can share one representation.
A useful AST should not mechanically preserve every punctuation mark, and it should not lower too early into instructions tied to one runtime. It must be abstract enough for semantics while retaining source links for diagnostics and tooling.
The checker: useful certainty in a dynamic language
HHY is dynamically typed, but dynamic does not mean every error should wait until runtime. The checker catches undefined names, duplicate bindings, assignment to immutable bindings, arity mismatches, return outside a function, break outside a loop, circular module imports, and missing exports.
It also protects concurrency boundaries. A parallel closure cannot capture mutable bindings or a single-consumption stream because those values are unsafe to share across workers. Rejecting that before workers start is more dependable than discovering it as a race.
Standard functions register more than a C function pointer. Each HHY callable has a contract describing arity, effect category, laziness, cancellation support, sendability, input/output constraints, and threading. The checker, runtime, dry-run planner, and future extension protocol all consume the same contract.
The principle is straightforward: if a problem can be proven before an effect begins, do not defer it to runtime.
The runtime: where the language becomes responsible for consequences
The runtime owns values, environments, calls, error propagation, and system resources. HHY values include Null, Bool, Int, Float, String, List, and Map, but also Regex, Bytes, Duration, Percent, Path, File, Process, HttpRequest, DateTime, Function, and Stream.
When a Pipe node executes, the runtime evaluates the left side and injects the resulting value into the callable on the right. One rule implements the language semantics instead of adding grammar for every library operation.
The interesting part of a systems language is not computing 1 + 1; it is handling consequences. Are files closed? Is a timed-out process reaped? Is TLS verified? Does cancellation propagate through a stream? Does a failed write leave a damaged output file?
HHY uses a constrained conservative tracing GC for language heap values, while files, processes, and network handles close explicitly. GC answers when language objects are unreachable. It cannot answer when an operating-system resource must be released. Conflating those lifetimes may survive short scripts and fail under long flows or error paths.
Streams: laziness as an execution protocol
HHY streams are pull-based, lazy, and single-consumption. where, map, and take do not materialize the whole input. Each downstream request pulls one value through the chain.
sink.next()
→ take.next()
→ where.next()
→ files.next()
→ one value returns downstream
Large files and directories stay bounded in memory. take(10) can stop upstream early. Errors and cancellation can close the chain.
Laziness still needs explicit barriers. sort_by, group_by, and some reductions must buffer input, so the runtime enforces memory and record limits. parallel is bounded too: maximum concurrency, finite buffering, ordered output, fail-fast behavior, and cleanup on cancellation.
I came away with a stronger belief that the difficult part of a concurrency API is not how work starts, but how it stops. Early termination, error, Ctrl-C, and timeout paths need one unwind model; fast happy-path execution is not enough.
Explicit effects and safe defaults
HHY separates describing an action from performing it where practical. http.get constructs an HttpRequest; send performs network I/O. run receives an argument array and does not invoke a shell; shell semantics require explicit shell. File saves prefer atomic replacement.
The runtime imposes limits on memory, open files, processes, parallelism, HTTP bodies, regex steps, recursion, and total runtime. hhy run --dry-run produces a redacted execution plan without external effects. That feature is credible because it uses the AST and callable contracts, not string matching.
Testing a language means testing failure
The dangerous behavior is often outside valid examples. HHY therefore tests:
- exact token and AST snapshots;
- parser recovery and invalid escapes;
- checker failures for scopes, arity, modules, and parallel captures;
- integer overflow, resource unwind, cancellation, timeout, and structured runtime errors;
- acceptance flows for files, processes, HTTP, JSON, CSV, file watching, and parallel work;
- ASan, UBSan, and coverage-guided fuzzing;
- native CI on macOS arm64, Linux arm64, and Linux x86_64;
- parsing and checking the HHY code blocks in the README and specification;
- release archives, notices, dependencies, and SHA-256 checksums.
Executable documentation is especially valuable. If examples never enter CI, language evolution will eventually turn them into plausible-looking programs that no longer run.
What I deliberately left out
HHY v1.0 has no JIT, no static type system, and no public native ABI. Extension boundaries exist, but the first stable runtime does not load unknown third-party extensions.
These are scope decisions, not forgotten features. A language earns trust by fully implementing a smaller set of promises. Direct AST interpretation let me establish semantics, errors, resource behavior, and portability first. Dynamic typing matches the feedback loop of systems scripting. Delaying a native ABI avoids freezing internal structures before they are ready to carry long-term compatibility.
The lasting lesson
Building HHY changed how I think about abstraction. An abstraction is not a better name around code; it is a constraint that remains consistent across layers.
|> becomes Flow-first only when the parser associates it correctly, the AST represents it precisely, the checker validates the callable, the runtime propagates cancellation, and the library obeys the lazy protocol. Otherwise, it is only an operator.
The AST matters, but it is not the destination. Parsing matters, but successful parsing does not imply dependable execution. The runtime matters, but a runtime without a specification and evidence is only the accidental behavior of the current build.
My working definition is now:
language = syntax + semantics + execution model + effect boundaries
+ diagnostics + tooling + compatibility promises
HHY v1.0 is still young, but it can already apply one Flow model to files, processes, networks, and structured data—and explain what it will do, where it failed, how it stops, and how it releases resources. That is much closer to the problem I wanted to solve than simply inventing another syntax.