Add jetc: a prototype JET-to-C++ compiler, plus the Tag API it needed

jetc (JetCore/jetc/) compiles a subset of JET template syntax - literal
text, $[...]/#[...] variable output, <set>, <if>/<else>, <for>, <while> -
into a standalone C++ program that links against CoreUtils and the same
JetCore tag-class library jet-2.0 itself uses. The static tag/text
structure becomes real C++ control flow (if/for/while), resolved once at
build time instead of being re-scanned character-by-character from raw
source on every request and, for loop bodies, on every single iteration
(confirmed by reading Tag::parseContainer - it re-tokenizes its container
text from scratch every time it runs, since a tag's parse and its
execution are the same pass in this codebase). Anything genuinely
dynamic - variable values, expr/value comparisons, loop bounds - still
calls into the existing runtime (Tag::getVariable/resolveKeyword,
jet::Operand) exactly as the interpreter does; compiling doesn't try to
eliminate that; only the repeated re-parsing of the template's own shape.

A <if type="..."> or <while type="..."> comparison type is resolved to
its C++ operator once at compile time (type is a literal attribute value
in practice, not runtime data) instead of being re-checked against six
string constants on every evaluation.

Any tag jetc doesn't have a codegen rule for (mysql, sql, call, include,
a <set> with a container body, etc.) is captured as its exact raw source
span and handed to the existing interpreter at compiled-program runtime
via the new Tag::interpretSpan(), so an unsupported construct degrades
gracefully instead of failing to compile - a real, if narrower, program
rather than requiring full tag coverage up front.

Tag.h/.cpp: made assignVariable public (parallel to the already-public
getVariable - generated code needs a write path without being a Tag
subclass) and added interpretSpan() as the fallback's entry point into
parseContainer.

Verified live: two representative templates (jetc/demo.jet, an if/for/set
walkthrough; jetc/demo2.jet, a nested <if> inside a <for>, a <while> that
mutates and rereads its own condition variable across iterations, and a
fallback-interpreted <comment> tag) both compile, link, run standalone,
and produce byte-identical output to jet-2.0 running the same source.
Building and running jetc/demo2.jet's <while> loop surfaced a real,
previously-unknown MString bug (fixed separately, see CoreUtils commit
6e4e463) along with a separate, already-existing bug in __while.cpp's
value1/value2/type form (unrelated to this compiler, not fixed here -
see project memory).

jetc/build-compiled.sh builds the compiler and a given .jet file's
compiled binary in one step; jetc/demo.jet and demo2.jet are kept as
worked examples. Known limitations: <set> with a container body isn't
modeled (falls back to interpretation); a fallback-interpreted tag
nested inside compiled control flow uses the template's single top-level
scope for parent/local context rather than a faithful per-level chain,
which only matters for legacy tags that rely on nested "local" scoping.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Brad ArantandClaude Sonnet 5 committed 2026-08-23 19:18:41 -07:00
1 parent fc849c44a4
commit d52ca61250
11 files changed
+821 -5

No files matched your search

+4
View File
@@ -139,6 +139,10 @@ namespace jet {
parseContainer(container, out, container2, topLevel); parseContainer(container, out, container2, topLevel);
} }
void Tag::interpretSpan(coreutils::ZString span, coreutils::MString &out) {
parseContainer(span, out);
}
void Tag::parseContainer(coreutils::ZString &in, coreutils::MString &out, coreutils::ZString container2, bool topLevel) { void Tag::parseContainer(coreutils::ZString &in, coreutils::MString &out, coreutils::ZString container2, bool topLevel) {
coreutils::ZString tag; coreutils::ZString tag;
char *start = in.getCursor(); char *start = in.getCursor();
+12 -5
View File
@@ -34,6 +34,18 @@ namespace jet {
Tag *parent; Tag *parent;
Tag *local; Tag *local;
// Public write counterpart to the public getVariable() read path -
// needed by anything (e.g. compiled/codegen'd output) that assigns
// variables without being a Tag subclass itself. See Tag.cpp for the
// "@name" session-file routing this also handles.
void assignVariable(coreutils::ZString variable, coreutils::MString value, coreutils::ZString scope);
// Falls back to the normal text-scanning interpreter (parseContainer)
// for a raw source span - used by generated code to run any tag a
// compiler doesn't have a codegen rule for yet, without duplicating
// the 22 tag classes' logic.
void interpretSpan(coreutils::ZString span, coreutils::MString &out);
protected: protected:
bool hasContainer = false; bool hasContainer = false;
bool hasContainer2 = false; bool hasContainer2 = false;
@@ -54,11 +66,6 @@ namespace jet {
void renderVariableName(coreutils::ZString &variable, coreutils::MString &name, coreutils::MString &modifier); void renderVariableName(coreutils::ZString &variable, coreutils::MString &name, coreutils::MString &modifier);
void storeVariable(coreutils::ZString variable, coreutils::MString value, coreutils::ZString scope); void storeVariable(coreutils::ZString variable, coreutils::MString value, coreutils::ZString scope);
void storeVariable(coreutils::ZString variable); void storeVariable(coreutils::ZString variable);
// Single write path shared by both storeVariable overloads. An
// "@name"-prefixed variable is routed to the active JSON session
// file (global.sessionFile) instead of the normal global/local/parent
// variables maps, mirroring getVariable's "@" read branch.
void assignVariable(coreutils::ZString variable, coreutils::MString value, coreutils::ZString scope);
private: private:
bool containerOnly = false; bool containerOnly = false;
Executable
+64
View File
@@ -0,0 +1,64 @@
#ifndef __JetAST_h__
#define __JetAST_h__
#include <string>
#include <vector>
#include <memory>
namespace jetc {
enum class NodeKind {
TEXT, VAR, SET, IF, FOR, WHILE, FALLBACK
};
enum class CondMode { VALUES, EXPR };
struct Node;
using NodePtr = std::shared_ptr<Node>;
struct Cond {
CondMode mode = CondMode::EXPR;
std::string value1, value2, type; // VALUES mode
std::string expr; // EXPR mode
};
struct Node {
NodeKind kind;
// TEXT
std::string text;
// VAR - raw source slice, e.g. "$[@greeting]" or "#[foo]"
std::string varText;
// SET
std::string setName;
bool setHasValue = false;
std::string setValue;
bool setHasExpr = false;
std::string setExpr;
std::string setScope; // "" = default/global
// IF
Cond cond;
std::vector<NodePtr> thenBody;
std::vector<NodePtr> elseBody;
bool hasElse = false;
// FOR
bool forHasStart = false;
std::string forStart;
std::string forEnd;
std::string forStep;
bool forHasName = false;
std::string forName;
std::string forScope;
std::vector<NodePtr> body; // also used by WHILE
// FALLBACK - exact original source text of one unsupported tag
std::string rawSpan;
};
}
#endif
+210
View File
@@ -0,0 +1,210 @@
#include "JetCodegen.h"
#include <stdexcept>
namespace jetc {
std::string Codegen::ind(int indent) {
return std::string(indent * 3, ' ');
}
std::string Codegen::cstr(const std::string &raw) {
std::string r;
r.reserve(raw.size() + 8);
r += '"';
for (unsigned char c : raw) {
switch (c) {
case '\\': r += "\\\\"; break;
case '"': r += "\\\""; break;
case '\n': r += "\\n"; break;
case '\r': r += "\\r"; break;
case '\t': r += "\\t"; break;
default:
if (c < 0x20) {
char buf[8];
snprintf(buf, sizeof(buf), "\\x%02x\" \"", c);
r += buf;
} else {
r += (char)c;
}
}
}
r += '"';
return r;
}
const char *Codegen::staticTypeOp(const std::string &type, const char *rc) {
// type is known at compile time (a literal attribute value, not a
// runtime expression) so the comparison operator is picked once here
// instead of being re-checked against six string constants on every
// request the way Tag::__if/__while do it at runtime.
static thread_local std::string expr;
if (type == "eq") expr = std::string(rc) + " == 0";
else if (type == "ne") expr = std::string(rc) + " != 0";
else if (type == "lt") expr = std::string(rc) + " == -1";
else if (type == "le") expr = std::string(rc) + " != 1";
else if (type == "gt") expr = std::string(rc) + " == 1";
else if (type == "ge") expr = std::string(rc) + " != -1";
else throw std::runtime_error("jetc: type=\"" + type + "\" is not one of eq/ne/lt/le/gt/ge "
"(and jetc only supports a literal type= value - use expr= for a dynamic comparison).");
return expr.c_str();
}
void Codegen::emitBody(const std::vector<NodePtr> &nodes, int indent) {
for (auto &n : nodes) emitNode(n, indent);
}
void Codegen::emitNode(const NodePtr &n, int indent) {
std::string I = ind(indent);
switch (n->kind) {
case NodeKind::TEXT: {
if (n->text.empty()) break;
out << I << "out << " << cstr(n->text) << ";\n";
break;
}
case NodeKind::VAR: {
out << I << "{ coreutils::ZString v(" << cstr(n->varText) << "); out << scope.getVariable(v, true); }\n";
break;
}
case NodeKind::SET: {
std::string k = id("set_");
out << I << "{\n";
out << I << " scope.keywords[\"" << k << "name\"] = " << cstr(n->setName) << ";\n";
out << I << " scope.keywords[\"" << k << "scope\"] = " << cstr(n->setScope) << ";\n";
out << I << " coreutils::MString val;\n";
if (n->setHasExpr) {
out << I << " scope.keywords[\"" << k << "expr\"] = " << cstr(n->setExpr) << ";\n";
out << I << " val = jet::Operand(scope.keywords[\"" << k << "expr\"], scope).string;\n";
} else {
out << I << " scope.keywords[\"" << k << "value\"] = " << cstr(n->setHasValue ? n->setValue : std::string()) << ";\n";
out << I << " val = scope.resolveKeyword(\"" << k << "value\");\n";
}
out << I << " scope.assignVariable(scope.resolveKeyword(\"" << k << "name\"), val, scope.resolveKeyword(\"" << k << "scope\"));\n";
out << I << "}\n";
break;
}
case NodeKind::IF: {
std::string k = id("if_");
out << I << "{\n";
out << I << " bool cond;\n";
if (n->cond.mode == CondMode::EXPR) {
out << I << " scope.keywords[\"" << k << "expr\"] = " << cstr(n->cond.expr) << ";\n";
out << I << " cond = jet::Operand(scope.keywords[\"" << k << "expr\"], scope).boolean;\n";
} else {
out << I << " scope.keywords[\"" << k << "value1\"] = " << cstr(n->cond.value1) << ";\n";
out << I << " scope.keywords[\"" << k << "value2\"] = " << cstr(n->cond.value2) << ";\n";
out << I << " int rc = scope.resolveKeyword(\"" << k << "value1\").compare(scope.resolveKeyword(\"" << k << "value2\"));\n";
out << I << " cond = (" << staticTypeOp(n->cond.type, "rc") << ");\n";
}
out << I << " if (cond) {\n";
emitBody(n->thenBody, indent + 2);
if (n->hasElse) {
out << I << " } else {\n";
emitBody(n->elseBody, indent + 2);
}
out << I << " }\n";
out << I << "}\n";
break;
}
case NodeKind::FOR: {
std::string k = id("for_");
out << I << "{\n";
out << I << " scope.keywords[\"" << k << "start\"] = " << cstr(n->forHasStart ? n->forStart : std::string("0")) << ";\n";
out << I << " scope.keywords[\"" << k << "end\"] = " << cstr(n->forEnd) << ";\n";
out << I << " scope.keywords[\"" << k << "step\"] = " << cstr(n->forStep) << ";\n";
out << I << " double ix = scope.resolveKeyword(\"" << k << "start\").asDouble();\n";
if (n->forHasName) {
out << I << " scope.keywords[\"" << k << "name\"] = " << cstr(n->forName) << ";\n";
out << I << " scope.keywords[\"" << k << "scope\"] = " << cstr(n->forScope) << ";\n";
}
out << I << " for (; ix <= scope.resolveKeyword(\"" << k << "end\").asDouble(); ix += scope.resolveKeyword(\"" << k << "step\").asDouble()) {\n";
if (n->forHasName) {
out << I << " coreutils::MString nameSel = scope.resolveKeyword(\"" << k << "name\");\n";
out << I << " coreutils::MString scopeSel = scope.resolveKeyword(\"" << k << "scope\");\n";
out << I << " if ((scopeSel == \"\") || (scopeSel == \"global\")) global.variables[nameSel] = ix;\n";
out << I << " else if (scopeSel == \"local\") scope.local->variables[nameSel] = ix;\n";
out << I << " else if (scopeSel == \"parent\") scope.parent->local->variables[nameSel] = ix;\n";
out << I << " else throw coreutils::Exception(\"scope value is not valid.\");\n";
}
emitBody(n->body, indent + 2);
out << I << " }\n";
out << I << "}\n";
break;
}
case NodeKind::WHILE: {
std::string k = id("while_");
out << I << "{\n";
out << I << " bool cond;\n";
if (n->cond.mode == CondMode::EXPR) {
out << I << " scope.keywords[\"" << k << "expr\"] = " << cstr(n->cond.expr) << ";\n";
out << I << " coreutils::MString exprSaved = scope.keywords[\"" << k << "expr\"];\n";
out << I << " cond = jet::Operand(scope.keywords[\"" << k << "expr\"], scope).boolean;\n";
out << I << " while (cond) {\n";
emitBody(n->body, indent + 2);
out << I << " scope.keywords[\"" << k << "expr\"].reset();\n";
out << I << " scope.keywords[\"" << k << "expr\"] = exprSaved;\n";
out << I << " cond = jet::Operand(scope.keywords[\"" << k << "expr\"], scope).boolean;\n";
out << I << " }\n";
} else {
out << I << " scope.keywords[\"" << k << "value1\"] = " << cstr(n->cond.value1) << ";\n";
out << I << " scope.keywords[\"" << k << "value2\"] = " << cstr(n->cond.value2) << ";\n";
out << I << " int rc = scope.resolveKeyword(\"" << k << "value1\").compare(scope.resolveKeyword(\"" << k << "value2\"));\n";
out << I << " cond = (" << staticTypeOp(n->cond.type, "rc") << ");\n";
out << I << " while (cond) {\n";
emitBody(n->body, indent + 2);
out << I << " int rc2 = scope.resolveKeyword(\"" << k << "value1\").compare(scope.resolveKeyword(\"" << k << "value2\"));\n";
out << I << " cond = (" << staticTypeOp(n->cond.type, "rc2") << ");\n";
out << I << " }\n";
}
out << I << "}\n";
break;
}
case NodeKind::FALLBACK: {
out << I << "{ coreutils::ZString span(" << cstr(n->rawSpan) << "); scope.interpretSpan(span, out); }\n";
break;
}
}
}
std::string Codegen::generate(const std::vector<NodePtr> &top, const std::string &sourceFileComment) {
out.str("");
out.clear();
out << "// " << sourceFileComment << "\n";
out << "// Generated by jetc - DO NOT EDIT BY HAND.\n";
out << "#include \"Global.h\"\n";
out << "#include \"Tag.h\"\n";
out << "#include \"Operand.h\"\n";
out << "#include \"ZString.h\"\n";
out << "#include \"MString.h\"\n";
out << "#include \"Exception.h\"\n";
out << "#include <iostream>\n\n";
out << "static void jetc_render(jet::Global &global, coreutils::MString &out) {\n";
out << " coreutils::ZString jetc_empty(\"\");\n";
out << " coreutils::MString jetc_scopeOut;\n";
out << " jet::Tag scope(jetc_empty, jetc_scopeOut, global, NULL, NULL);\n";
emitBody(top, 1);
out << "}\n\n";
out << "int main(int argc, char **argv, char **envp) {\n";
out << " jet::Global global(envp);\n";
out << " coreutils::MString out;\n";
out << " try {\n";
out << " jetc_render(global, out);\n";
out << " global.outputHeaders();\n";
out << " std::cout << out;\n";
out << " } catch(coreutils::Exception e) {\n";
out << " std::cout << out;\n";
out << " std::cout << \"\\n----- Error: \" << e.text << std::endl;\n";
out << " return 1;\n";
out << " }\n";
out << " return 0;\n";
out << "}\n";
return out.str();
}
}
+35
View File
@@ -0,0 +1,35 @@
#ifndef __JetCodegen_h__
#define __JetCodegen_h__
#include "JetAST.h"
#include <string>
#include <sstream>
namespace jetc {
// Emits a standalone .cpp translation unit that renders a compiled JET
// template. The static tag/text structure becomes real C++ control flow;
// anything genuinely dynamic (variable values, expr/value comparisons,
// loop bounds) still calls into the existing JetCore/CoreUtils runtime
// (Tag::getVariable/resolveKeyword/assignVariable, jet::Operand) exactly
// as the interpreter does - compiling doesn't try to eliminate that, only
// the repeated re-parsing of the template's own structure.
class Codegen {
public:
std::string generate(const std::vector<NodePtr> &top, const std::string &sourceFileComment);
private:
std::ostringstream out;
size_t nextId = 0;
std::string id(const std::string &prefix) { return prefix + std::to_string(nextId++) + "_"; }
void emitBody(const std::vector<NodePtr> &nodes, int indent);
void emitNode(const NodePtr &n, int indent);
static std::string ind(int indent);
static std::string cstr(const std::string &raw); // C++ string-literal escape
static const char *staticTypeOp(const std::string &type, const char *rcVar);
};
}
#endif
+305
View File
@@ -0,0 +1,305 @@
#include "JetParser.h"
#include <cctype>
#include <stdexcept>
namespace jetc {
bool Parser::starts(const std::string &lit) const {
return s.compare(pos, lit.size(), lit) == 0;
}
bool Parser::isNameChar(char c) {
return std::isalnum((unsigned char)c) || c == '-' || c == '_';
}
void Parser::skipWs() {
while (!eof() && std::isspace((unsigned char)s[pos])) pos++;
}
std::string Parser::readName() {
size_t start = pos;
while (!eof() && isNameChar(s[pos])) pos++;
return s.substr(start, pos - start);
}
std::string Parser::peekTagName(size_t at) const {
size_t p = at + 1;
size_t start = p;
while (p < s.size() && isNameChar(s[p])) p++;
return s.substr(start, p - start);
}
bool Parser::matchTagOpen(const std::string &name, size_t at) const {
if (s.compare(at, 1, "<") != 0) return false;
size_t p = at + 1;
size_t nstart = p;
while (p < s.size() && isNameChar(s[p])) p++;
if (s.compare(nstart, p - nstart, name) != 0) return false;
if (p >= s.size()) return false;
char c = s[p];
return c == ' ' || c == '>' || c == '/';
}
bool Parser::matchTagClose(const std::string &name, size_t at) const {
std::string closer = "</" + name + ">";
return s.compare(at, closer.size(), closer) == 0;
}
bool Parser::skipOpenTag(size_t &p) const {
p++; // '<'
while (p < s.size() && isNameChar(s[p])) p++;
while (p < s.size()) {
if (s[p] == '"') {
p++;
while (p < s.size() && s[p] != '"') {
if (s[p] == '\\' && p + 1 < s.size()) p++;
p++;
}
if (p < s.size()) p++;
continue;
}
if (s.compare(p, 2, "/>") == 0) { p += 2; return true; }
if (s[p] == '>') { p += 1; return false; }
p++;
}
throw std::runtime_error("jetc: unterminated tag near end of input");
}
std::map<std::string, std::string> Parser::parseAttrs(bool &hasContainer) {
std::map<std::string, std::string> attrs;
while (true) {
skipWs();
if (eof()) throw std::runtime_error("jetc: unterminated tag near end of input");
if (starts("/>")) { pos += 2; hasContainer = false; break; }
if (starts(">")) { pos += 1; hasContainer = true; break; }
std::string aname = readName();
if (aname.empty()) throw std::runtime_error("jetc: malformed attribute near offset " + std::to_string(pos));
if (starts("=\"")) {
pos += 2;
std::string val;
while (!eof() && s[pos] != '"') {
if (s[pos] == '\\' && pos + 1 < s.size()) { val += s[pos]; val += s[pos + 1]; pos += 2; continue; }
val += s[pos++];
}
if (!eof()) pos++; // closing quote
attrs[aname] = val;
}
}
return attrs;
}
std::pair<size_t, size_t> Parser::scanBody(size_t bodyStart, const std::string &tagName, bool lookForElse) {
size_t p = bodyStart;
int depth = 1;
size_t elsePos = std::string::npos;
while (p < s.size()) {
if (matchTagClose(tagName, p)) {
depth--;
if (depth == 0) return {p, elsePos};
p += 3 + tagName.size();
continue;
}
if (lookForElse && depth == 1 && elsePos == std::string::npos && s.compare(p, 6, "<else>") == 0) {
elsePos = p;
p += 6;
continue;
}
if (matchTagOpen(tagName, p)) {
bool selfClosing = skipOpenTag(p);
if (!selfClosing) depth++;
continue;
}
if (s[p] == '<') {
if (s.compare(p, 2, "</") == 0) {
size_t q = p + 2;
while (q < s.size() && isNameChar(s[q])) q++;
if (q < s.size() && s[q] == '>') q++;
p = q;
} else {
skipOpenTag(p);
}
continue;
}
p++;
}
throw std::runtime_error("jetc: unterminated <" + tagName + "> - no matching close tag found");
}
std::vector<NodePtr> Parser::parseSub(size_t start, size_t end) {
Parser sub(s.substr(start, end - start));
return sub.parseNodes();
}
NodePtr Parser::parseVar() {
size_t start = pos;
pos += 2; // "$[" or "#["
int depth = 1;
while (!eof() && depth > 0) {
if (starts("$[") || starts("#[")) { depth++; pos += 2; continue; }
if (s[pos] == ']') { depth--; pos++; continue; }
pos++;
}
auto n = std::make_shared<Node>();
n->kind = NodeKind::VAR;
n->varText = s.substr(start, pos - start);
return n;
}
NodePtr Parser::parseSet() {
size_t origStart = pos;
pos += 4; // "<set"
bool hasContainer;
auto attrs = parseAttrs(hasContainer);
if (hasContainer) {
// Container-form <set> (value comes from rendering nested content)
// isn't modeled by this prototype - fall back to full interpretation
// of the whole tag instead of failing to compile.
auto span = scanBody(pos, "set", false);
pos = span.first + 3 + 3; // "</set>"
auto n = std::make_shared<Node>();
n->kind = NodeKind::FALLBACK;
n->rawSpan = s.substr(origStart, pos - origStart);
return n;
}
auto n = std::make_shared<Node>();
n->kind = NodeKind::SET;
n->setName = attrs.count("name") ? attrs["name"] : "";
if (attrs.count("value")) { n->setHasValue = true; n->setValue = attrs["value"]; }
if (attrs.count("expr")) { n->setHasExpr = true; n->setExpr = attrs["expr"]; }
n->setScope = attrs.count("scope") ? attrs["scope"] : "";
return n;
}
NodePtr Parser::parseIf() {
pos += 3; // "<if"
bool hasContainer;
auto attrs = parseAttrs(hasContainer);
auto n = std::make_shared<Node>();
n->kind = NodeKind::IF;
if (attrs.count("expr")) {
n->cond.mode = CondMode::EXPR;
n->cond.expr = attrs["expr"];
} else {
n->cond.mode = CondMode::VALUES;
n->cond.value1 = attrs.count("value1") ? attrs["value1"] : "";
n->cond.value2 = attrs.count("value2") ? attrs["value2"] : "";
n->cond.type = attrs.count("type") ? attrs["type"] : "";
}
if (hasContainer) {
auto span = scanBody(pos, "if", true);
size_t closePos = span.first, elsePos = span.second;
size_t thenEnd = (elsePos != std::string::npos) ? elsePos : closePos;
n->thenBody = parseSub(pos, thenEnd);
if (elsePos != std::string::npos) {
n->hasElse = true;
n->elseBody = parseSub(elsePos + 6, closePos);
}
pos = closePos + 3 + 2; // "</if>"
}
return n;
}
NodePtr Parser::parseFor() {
pos += 4; // "<for"
bool hasContainer;
auto attrs = parseAttrs(hasContainer);
auto n = std::make_shared<Node>();
n->kind = NodeKind::FOR;
if (attrs.count("start")) { n->forHasStart = true; n->forStart = attrs["start"]; }
n->forEnd = attrs.count("end") ? attrs["end"] : "0";
n->forStep = attrs.count("step") ? attrs["step"] : "1";
if (attrs.count("name")) { n->forHasName = true; n->forName = attrs["name"]; }
n->forScope = attrs.count("scope") ? attrs["scope"] : "";
if (hasContainer) {
auto span = scanBody(pos, "for", false);
n->body = parseSub(pos, span.first);
pos = span.first + 3 + 3; // "</for>"
}
return n;
}
NodePtr Parser::parseWhile() {
pos += 6; // "<while"
bool hasContainer;
auto attrs = parseAttrs(hasContainer);
auto n = std::make_shared<Node>();
n->kind = NodeKind::WHILE;
if (attrs.count("expr")) {
n->cond.mode = CondMode::EXPR;
n->cond.expr = attrs["expr"];
} else {
n->cond.mode = CondMode::VALUES;
n->cond.value1 = attrs.count("value1") ? attrs["value1"] : "";
n->cond.value2 = attrs.count("value2") ? attrs["value2"] : "";
n->cond.type = attrs.count("type") ? attrs["type"] : "";
}
if (hasContainer) {
auto span = scanBody(pos, "while", false);
n->body = parseSub(pos, span.first);
pos = span.first + 3 + 5; // "</while>"
}
return n;
}
NodePtr Parser::parseFallback() {
size_t origStart = pos;
std::string tagName = peekTagName(pos);
size_t p = pos;
bool selfClosing = skipOpenTag(p);
if (!selfClosing) {
auto span = scanBody(p, tagName, false);
p = span.first + 3 + tagName.size();
}
pos = p;
auto n = std::make_shared<Node>();
n->kind = NodeKind::FALLBACK;
n->rawSpan = s.substr(origStart, pos - origStart);
return n;
}
std::vector<NodePtr> Parser::parseNodes() {
std::vector<NodePtr> nodes;
std::string textBuf;
auto flush = [&]() {
if (!textBuf.empty()) {
auto n = std::make_shared<Node>();
n->kind = NodeKind::TEXT;
n->text = textBuf;
nodes.push_back(n);
textBuf.clear();
}
};
while (!eof()) {
if (starts("|>")) {
flush();
pos += 2;
std::string buf;
while (!eof() && !starts("<|")) buf += s[pos++];
if (!eof()) pos += 2;
auto n = std::make_shared<Node>();
n->kind = NodeKind::TEXT;
n->text = buf;
nodes.push_back(n);
continue;
}
if (s[pos] == '<') {
if (matchTagOpen("set", pos)) { flush(); nodes.push_back(parseSet()); continue; }
if (matchTagOpen("if", pos)) { flush(); nodes.push_back(parseIf()); continue; }
if (matchTagOpen("for", pos)) { flush(); nodes.push_back(parseFor()); continue; }
if (matchTagOpen("while", pos)) { flush(); nodes.push_back(parseWhile()); continue; }
flush();
nodes.push_back(parseFallback());
continue;
}
if (starts("$[") || starts("#[")) {
flush();
nodes.push_back(parseVar());
continue;
}
textBuf += s[pos++];
}
flush();
return nodes;
}
}
+69
View File
@@ -0,0 +1,69 @@
#ifndef __JetParser_h__
#define __JetParser_h__
#include "JetAST.h"
#include <string>
#include <map>
#include <utility>
namespace jetc {
// Standalone parser for a subset of the JET template grammar. Mirrors the
// character-level grammar implemented by JetCore's Tag::parseContainer /
// Tag::Tag (attribute scanning, same-tag-name nesting depth, the <if>
// <else> split) closely enough for well-formed templates, but is a fresh
// implementation - it does not reuse JetCore's Tag classes, since those
// fuse parsing and execution together (a real parse/execute split there
// is a separate, larger refactor). Anything outside the modeled subset
// (set/if/for/while/$[...]/literal text) is captured as an exact raw
// source span and left for the existing interpreter (Tag::interpretSpan)
// to run at compiled-program runtime, so unsupported constructs degrade
// gracefully instead of failing to compile.
class Parser {
public:
explicit Parser(const std::string &src) : s(src), pos(0) {}
std::vector<NodePtr> parseNodes();
private:
std::string s;
size_t pos;
size_t nextId = 0;
bool eof() const { return pos >= s.size(); }
bool starts(const std::string &lit) const;
static bool isNameChar(char c);
void skipWs();
std::string readName();
std::string peekTagName(size_t at) const;
bool matchTagOpen(const std::string &name, size_t at) const;
bool matchTagClose(const std::string &name, size_t at) const;
// p points at '<' of some tag's opening; advances p past its '>' or
// "/>", skipping quoted attribute values. Returns true if self-closing.
bool skipOpenTag(size_t &p) const;
std::map<std::string, std::string> parseAttrs(bool &hasContainer);
// Scans forward from bodyStart (just after the opening tag's '>') for
// tagName's matching close, tracking same-tag-name nesting depth and
// (if lookForElse) a depth==1 "<else>" split point.
// Returns {closePos, elsePos} (elsePos == npos if not found/not sought).
std::pair<size_t, size_t> scanBody(size_t bodyStart, const std::string &tagName, bool lookForElse);
std::vector<NodePtr> parseSub(size_t start, size_t end);
NodePtr parseVar();
NodePtr parseSet();
NodePtr parseIf();
NodePtr parseFor();
NodePtr parseWhile();
NodePtr parseFallback();
std::string newId(const std::string &prefix) { return prefix + std::to_string(nextId++) + "_"; }
};
}
#endif
+46
View File
@@ -0,0 +1,46 @@
#!/bin/bash
# Builds jetc itself (if needed), compiles a .jet file to C++, and links the
# result into a standalone executable against CoreUtils + the JetCore
# runtime (everything jet-2.0 itself links against, minus jet-2.0.cpp's own
# main() - the compiled program brings its own).
#
# Usage: ./build-compiled.sh <input.jet> <output-binary-name>
set -e
JC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CU="$JC/../CoreUtils"
HERE="$JC/jetc"
if [ $# -ne 2 ]; then
echo "usage: $0 <input.jet> <output-binary-name>"
exit 1
fi
INPUT="$1"
OUTNAME="$2"
if [ ! -x "$HERE/jetc" ]; then
echo "Building jetc..."
g++ -g -std=c++23 -o "$HERE/jetc" "$HERE/jetc.cpp" "$HERE/JetParser.cpp" "$HERE/JetCodegen.cpp"
fi
echo "Compiling $INPUT -> $OUTNAME.compiled.cpp"
"$HERE/jetc" "$INPUT" "$HERE/$OUTNAME.compiled.cpp"
echo "Building JetCore runtime objects (everything Tag::interpretSpan's fallback path may need)..."
list=""
for f in "$JC"/*.cpp; do
base=$(basename "$f")
if [ "$base" = "jet-2.0.cpp" ]; then continue; fi
obj="$HERE/${base%.cpp}.o"
if [ ! -f "$obj" ]; then
g++ -g -c -std=c++23 -I"$CU" -I"$JC" "$f" -o "$obj" &
fi
list="$list $obj"
done
wait
g++ -g -c -std=c++23 -I"$CU" -I"$JC" "$HERE/$OUTNAME.compiled.cpp" -o "$HERE/$OUTNAME.compiled.o"
g++ -g -o "$HERE/$OUTNAME" "$HERE/$OUTNAME.compiled.o" $list -std=c++23 -L"$CU" -lCoreUtils -lmysqlclient -lcrypto
echo "Built $HERE/$OUTNAME"
Executable
+16
View File
@@ -0,0 +1,16 @@
#!../jet-2.0
Hello, this is a jetc compile demo.
<set name="mood" value="curious" />
Mood is: $[mood]
<if value1="$[mood]" value2="curious" type="eq">
The mood matches!
<else>
The mood does not match.
</if>
Counting:
<for start="1" end="5" step="1" name="i">
- number $[i]
</for>
<set expr="2 + 3 * 4" name="calc" />
Calc result: $[calc]
Done.
Executable
+17
View File
@@ -0,0 +1,17 @@
#!../jet-2.0
Nested and fallback test.
<for start="1" end="6" step="1" name="n">
<if value1="$[n]" value2="3" type="gt">
big:$[n]
<else>
small:$[n]
</if>
</for>
<set name="counter" value="0" />
<while expr="$[counter] < 3">
loop:$[counter]
<set expr="$[counter] + 1" name="counter" />
</while>
<comment>this text is suppressed by the fallback-interpreted comment tag $[counter]</comment>
after comment.
The end.
Executable
+43
View File
@@ -0,0 +1,43 @@
#include "JetParser.h"
#include "JetCodegen.h"
#include <fstream>
#include <sstream>
#include <iostream>
int main(int argc, char **argv) {
if (argc < 3) {
std::cerr << "usage: jetc <input.jet> <output.cpp>" << std::endl;
return 1;
}
std::ifstream in(argv[1]);
if (!in) {
std::cerr << "jetc: cannot open " << argv[1] << std::endl;
return 1;
}
std::ostringstream ss;
ss << in.rdbuf();
std::string src = ss.str();
// Skip a leading shebang line, mirroring jet-2.0.cpp's own data.goeol()
// call (run before it constructs the top-level __jet tag) so a
// "#!../jet-2.0" first line doesn't get compiled in as literal output.
if (src.rfind("#!", 0) == 0) {
size_t nl = src.find('\n');
src = (nl == std::string::npos) ? std::string() : src.substr(nl + 1);
}
try {
jetc::Parser parser(src);
auto nodes = parser.parseNodes();
jetc::Codegen codegen;
std::string cpp = codegen.generate(nodes, std::string("compiled from ") + argv[1]);
std::ofstream outFile(argv[2]);
outFile << cpp;
outFile.close();
std::cout << "jetc: wrote " << argv[2] << std::endl;
} catch (std::exception &e) {
std::cerr << "jetc: " << e.what() << std::endl;
return 1;
}
return 0;
}