Add jetc: a prototype JET-to-C++ compiler, plus the Tag API it needed

jetc (JetCore/jetc/) compiles a subset of JET template syntax - literal
text, $[...]/#[...] variable output, <set>, <if>/<else>, <for>, <while> -
into a standalone C++ program that links against CoreUtils and the same
JetCore tag-class library jet-2.0 itself uses. The static tag/text
structure becomes real C++ control flow (if/for/while), resolved once at
build time instead of being re-scanned character-by-character from raw
source on every request and, for loop bodies, on every single iteration
(confirmed by reading Tag::parseContainer - it re-tokenizes its container
text from scratch every time it runs, since a tag's parse and its
execution are the same pass in this codebase). Anything genuinely
dynamic - variable values, expr/value comparisons, loop bounds - still
calls into the existing runtime (Tag::getVariable/resolveKeyword,
jet::Operand) exactly as the interpreter does; compiling doesn't try to
eliminate that; only the repeated re-parsing of the template's own shape.

A <if type="..."> or <while type="..."> comparison type is resolved to
its C++ operator once at compile time (type is a literal attribute value
in practice, not runtime data) instead of being re-checked against six
string constants on every evaluation.

Any tag jetc doesn't have a codegen rule for (mysql, sql, call, include,
a <set> with a container body, etc.) is captured as its exact raw source
span and handed to the existing interpreter at compiled-program runtime
via the new Tag::interpretSpan(), so an unsupported construct degrades
gracefully instead of failing to compile - a real, if narrower, program
rather than requiring full tag coverage up front.

Tag.h/.cpp: made assignVariable public (parallel to the already-public
getVariable - generated code needs a write path without being a Tag
subclass) and added interpretSpan() as the fallback's entry point into
parseContainer.

Verified live: two representative templates (jetc/demo.jet, an if/for/set
walkthrough; jetc/demo2.jet, a nested <if> inside a <for>, a <while> that
mutates and rereads its own condition variable across iterations, and a
fallback-interpreted <comment> tag) both compile, link, run standalone,
and produce byte-identical output to jet-2.0 running the same source.
Building and running jetc/demo2.jet's <while> loop surfaced a real,
previously-unknown MString bug (fixed separately, see CoreUtils commit
6e4e463) along with a separate, already-existing bug in __while.cpp's
value1/value2/type form (unrelated to this compiler, not fixed here -
see project memory).

jetc/build-compiled.sh builds the compiler and a given .jet file's
compiled binary in one step; jetc/demo.jet and demo2.jet are kept as
worked examples. Known limitations: <set> with a container body isn't
modeled (falls back to interpretation); a fallback-interpreted tag
nested inside compiled control flow uses the template's single top-level
scope for parent/local context rather than a faithful per-level chain,
which only matters for legacy tags that rely on nested "local" scoping.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Brad ArantandClaude Sonnet 5 committed 2026-08-23 19:18:41 -07:00
1 parent fc849c44a4
commit d52ca61250
11 files changed
+822 -6

No files matched your search

+69
View File
@@ -0,0 +1,69 @@
#ifndef __JetParser_h__
#define __JetParser_h__
#include "JetAST.h"
#include <string>
#include <map>
#include <utility>
namespace jetc {
// Standalone parser for a subset of the JET template grammar. Mirrors the
// character-level grammar implemented by JetCore's Tag::parseContainer /
// Tag::Tag (attribute scanning, same-tag-name nesting depth, the <if>
// <else> split) closely enough for well-formed templates, but is a fresh
// implementation - it does not reuse JetCore's Tag classes, since those
// fuse parsing and execution together (a real parse/execute split there
// is a separate, larger refactor). Anything outside the modeled subset
// (set/if/for/while/$[...]/literal text) is captured as an exact raw
// source span and left for the existing interpreter (Tag::interpretSpan)
// to run at compiled-program runtime, so unsupported constructs degrade
// gracefully instead of failing to compile.
class Parser {
public:
explicit Parser(const std::string &src) : s(src), pos(0) {}
std::vector<NodePtr> parseNodes();
private:
std::string s;
size_t pos;
size_t nextId = 0;
bool eof() const { return pos >= s.size(); }
bool starts(const std::string &lit) const;
static bool isNameChar(char c);
void skipWs();
std::string readName();
std::string peekTagName(size_t at) const;
bool matchTagOpen(const std::string &name, size_t at) const;
bool matchTagClose(const std::string &name, size_t at) const;
// p points at '<' of some tag's opening; advances p past its '>' or
// "/>", skipping quoted attribute values. Returns true if self-closing.
bool skipOpenTag(size_t &p) const;
std::map<std::string, std::string> parseAttrs(bool &hasContainer);
// Scans forward from bodyStart (just after the opening tag's '>') for
// tagName's matching close, tracking same-tag-name nesting depth and
// (if lookForElse) a depth==1 "<else>" split point.
// Returns {closePos, elsePos} (elsePos == npos if not found/not sought).
std::pair<size_t, size_t> scanBody(size_t bodyStart, const std::string &tagName, bool lookForElse);
std::vector<NodePtr> parseSub(size_t start, size_t end);
NodePtr parseVar();
NodePtr parseSet();
NodePtr parseIf();
NodePtr parseFor();
NodePtr parseWhile();
NodePtr parseFallback();
std::string newId(const std::string &prefix) { return prefix + std::to_string(nextId++) + "_"; }
};
}
#endif