Multi-day accumulated fixes: tag dispatch, session/pooling correctness, SPA rollout, jetc compiler

JetCore engine (Tag.cpp/.h, Global.cpp/.h, __*.cpp tag classes):
- Numerous __mysql.cpp connection-pooling correctness fixes (stale
  result sets, double-indirection sessionid bugs, field caching,
  pool keying) and a session-registration-before-connection-confirmed
  ordering fix (dangling pointer on a failed connect)
- Tag.cpp: large body of fixes to the tag-dispatch/container-scanning
  logic (bare-name fallback, database aliasing, single-quote handling,
  @-prefix session variable resolution, TAGDEF caching, nested split-tag
  scanning for if/while/until/ifrow so an inner one doesn't corrupt an
  outer one's own boundary scan) plus the process-first rendering
  order fix (a <process> now runs before <checksession>/page
  resolution, fixing a real "blank page after successful login" bug)
- Global.h/.cpp: added imapSessions (parallel to the existing __mysql*
  session map) for the new <imap> tag family
- __whiledir.cpp/__include.cpp: caching and directory-name-aliasing
  fixes
- __call.cpp/.h: fixed a dangling-pointer bug
- __write.cpp: fixed an O_WRONLY-only open that couldn't create a new
  file
- __ifrow.cpp/__sql.cpp: fixed the same sessionid double-indirection
  bug __mysql.cpp had (resolveKeyword's result was being used as a map
  key lookup instead of the resolved value itself)

jetc compiler (jetc/JetParser.cpp/.h, JetCodegen.cpp/.h, JetAST.h,
jetc.cpp, build-shared.sh, build-compiled.sh):
- Parser fixes: inline <include>, nested-quote handling, void-tag
  recognition
- Codegen fixes and expanded AST support
- build-shared.sh: atomic rename into place for a freshly-built .jtc
  (a live server dlopen()ing the same path mid-rebuild was corrupting
  its own already-mapped view of the old file) and a staleness check
  so a cached JetCore .o doesn't silently keep serving yesterday's
  fixes forever
- Added -lssl to both build scripts' link lines (needed once the new
  <imap> tag's client-side TLS code existed - previously nothing in
  this codebase used OpenSSL's SSL/TLS client API, only libcrypto)

Site content (jetc/barant-site/): SPA (fetch-and-swap navigation)
rollout across essentially every .page/.jet/.section/.tag/.proc/.method
file - dynamic-content fixes, fragment-mode stripping, native
$[@name] session variables replacing older mechanisms, and this
session's own checkout.page redesign, login start-param redirect fix,
and per-page side-menu (menu.bar/$[PAGE].menu) refresh support.
Completed the earlier main.css/control.css move from assets/ to
assets/images/ (git still showed the old paths as tracked-but-deleted
until now).

Deliberately left out of this commit: dozens of .pre-<description>-
<date> manual backup files (this session's and earlier ones' own
before/after snapshots, not meant for permanent history), compiled
.jtc build outputs, ad hoc test binaries and scripts under jetc/
(login_asan, wcwc_test, optest2_bin, etc.), and jetc/__pycache__/ -
none of these are meant to be tracked; a .gitignore covering them
would be worth adding separately.
This commit is contained in:
Brad Arant committed 2026-09-03 05:20:48 +00:00
1 parent 54b6e522f9
commit b0cadc3f08
162 files changed
+3567 -2887

No files matched your search

+299 -30
View File
@@ -1,6 +1,11 @@
#include "JetParser.h"
#include <cctype>
#include <stdexcept>
#include <set>
#include <cstdio>
#include <fstream>
#include <sstream>
#include <cstdio>
namespace jetc {
@@ -29,37 +34,148 @@ namespace jetc {
return s.substr(start, p - start);
}
namespace {
// jet-1.4 (the language this parser also has to accept, alongside
// real 2.0 source) was fully case-insensitive on tag names - the
// production source mixes case freely, even within one tag's own
// body (e.g. <SET ...> and <set ...> side by side). Tag names are
// matched case-insensitively throughout this parser for that
// reason; nothing else (attribute values, text content) is
// affected by this helper.
bool ciEquals(const char *a, size_t alen, const std::string &b) {
if (alen != b.size()) return false;
for (size_t i = 0; i < alen; i++)
if (std::tolower((unsigned char)a[i]) != std::tolower((unsigned char)b[i]))
return false;
return true;
}
}
bool Parser::matchTagOpen(const std::string &name, size_t at) const {
if (s.compare(at, 1, "<") != 0) return false;
size_t p = at + 1;
size_t nstart = p;
while (p < s.size() && isNameChar(s[p])) p++;
if (s.compare(nstart, p - nstart, name) != 0) return false;
if (!ciEquals(s.data() + nstart, p - nstart, name)) return false;
if (p >= s.size()) return false;
char c = s[p];
return c == ' ' || c == '>' || c == '/';
}
bool Parser::matchTagClose(const std::string &name, size_t at) const {
std::string closer = "</" + name + ">";
return s.compare(at, closer.size(), closer) == 0;
if (s.compare(at, 2, "</") != 0) return false;
size_t p = at + 2;
size_t nstart = p;
while (p < s.size() && isNameChar(s[p])) p++;
if (!ciEquals(s.data() + nstart, p - nstart, name)) return false;
return p < s.size() && s[p] == '>';
}
bool Parser::skipOpenTag(size_t &p) const {
namespace {
// jet-1.4's real parser knew which tags never take a body (its own
// "void element" list) - kept here, ahead of skipOpenTag/
// skipEmbeddedTag/parseFallback, since all of them need it.
bool isVoidTag(const std::string &name) {
static const std::set<std::string> voidTags = {
// jet's own built-ins that never take a body. exclude is
// NOT here - __exclude.cpp's own constructor requires a
// real body (throws "exclude must have a container"
// otherwise), it was previously miscategorized as void.
"set", "include", "cookie", "header", "read", "write",
"call", "system", "dump",
// <container> (see parseContainerSlot/Tag::renderContainer) -
// real content always writes it as a bare marker with no
// separate closing tag, even when it carries its own
// attributes (e.g. SESSIONCTL.tag's own
// <CONTAINER EVAL="NO">) - without this it was previously only
// reachable via the generic FALLBACK path (scanBody would
// incorrectly scan forward for a "</container>" that never
// exists, consuming unrelated following content).
"container",
// standard HTML5 void elements - real 1.4-era content (e.g.
// SESSIONCTL.tag) writes these the classic HTML way too,
// bare '>' and never a closing tag.
"br", "img", "input", "hr", "meta", "link", "area", "base",
"col", "embed", "source", "track", "wbr",
// <else> is a marker inside an <if>/<while>/<until> body
// (scanBody's own lookForElse handles it at the top level) -
// it never has its own close tag, so when it shows up
// embedded (inside a quoted attribute or another tag's
// attribute region, same as <if> does in <linkbutton>) it
// needs the same void treatment.
"else"
};
std::string lower = name;
for (char &c : lower) c = (char)std::tolower((unsigned char)c);
return voidTags.count(lower) != 0;
}
}
// Real 1.4 content nests real jet tags in two places a plain
// quote/angle-bracket scanner can't handle on its own:
// 1. Inside a quoted attribute value - SESSIONCTL.tag's <linkbutton>
// has HREF="javascript: <IF VALUE1="..." VALUE2="" TYPE=NE> ...
// </IF> ... " - the outer string and the embedded tag's own
// attributes reuse the same quote character with no escaping.
// 2. Inside another tag's OWN opening-tag attribute region, before
// its closing '>' - linkbutton's <A ...> has a conditional
// <if value1="#[CLASS]" value2="" type="ne">CLASS="#[CLASS]"</if>
// sitting between its other attributes and its own '>' - the
// literal '>' that ends the embedded <if ...>'s OWN opening tag
// would otherwise be mistaken for <A>'s closing '>'.
// skipEmbeddedTag() skips one such nested tag as a whole unit -
// its own attributes (recursing through skipOpenTag, so an embedded
// tag's embedded tag works too) and, if it isn't a void tag and wasn't
// self-closed, its real body up through its own matching close tag
// (via the same matchTagOpen/matchTagClose depth-tracking scanBody
// uses at the top level). skipOpenTag's own attribute scan and
// skipQuotedValue both call this for anything that looks like a
// nested tag, instead of treating it as ordinary content.
void Parser::skipEmbeddedTag(size_t &p) {
size_t nameStart = p + 1;
size_t nameEnd = nameStart;
while (nameEnd < s.size() && isNameChar(s[nameEnd])) nameEnd++;
std::string tagName = s.substr(nameStart, nameEnd - nameStart);
bool selfClosing = skipOpenTag(p);
if (!selfClosing && !isVoidTag(tagName)) {
auto span = scanBody(p, tagName, false);
p = span.first + 3 + tagName.size();
}
}
void Parser::skipQuotedValue(size_t &p) {
p++; // opening '"'
while (p < s.size()) {
if (s[p] == '"') { p++; return; }
if (s[p] == '\\' && p + 1 < s.size()) { p += 2; continue; }
if (s[p] == '<' && p + 1 < s.size() && isNameChar(s[p + 1])) {
skipEmbeddedTag(p);
continue;
}
if (s[p] == '<' && s.compare(p, 2, "</") == 0) {
p += 2;
while (p < s.size() && isNameChar(s[p])) p++;
if (p < s.size() && s[p] == '>') p++;
continue;
}
p++;
}
}
bool Parser::skipOpenTag(size_t &p) {
p++; // '<'
while (p < s.size() && isNameChar(s[p])) p++;
while (p < s.size()) {
if (s[p] == '"') {
p++;
while (p < s.size() && s[p] != '"') {
if (s[p] == '\\' && p + 1 < s.size()) p++;
p++;
}
if (p < s.size()) p++;
skipQuotedValue(p);
continue;
}
if (s.compare(p, 2, "/>") == 0) { p += 2; return true; }
if (s[p] == '>') { p += 1; return false; }
if (s[p] == '<' && p + 1 < s.size() && isNameChar(s[p + 1])) {
skipEmbeddedTag(p);
continue;
}
p++;
}
throw std::runtime_error("jetc: unterminated tag near end of input");
@@ -74,14 +190,40 @@ namespace jetc {
if (starts(">")) { pos += 1; hasContainer = true; break; }
std::string aname = readName();
if (aname.empty()) throw std::runtime_error("jetc: malformed attribute near offset " + std::to_string(pos));
if (starts("=\"")) {
pos += 2;
if (starts("=")) {
pos += 1;
std::string val;
while (!eof() && s[pos] != '"') {
if (s[pos] == '\\' && pos + 1 < s.size()) { val += s[pos]; val += s[pos + 1]; pos += 2; continue; }
val += s[pos++];
if (!eof() && s[pos] == '"') {
pos++;
while (!eof() && s[pos] != '"') {
if (s[pos] == '\\' && pos + 1 < s.size()) { val += s[pos]; val += s[pos + 1]; pos += 2; continue; }
val += s[pos++];
}
if (!eof()) pos++; // closing quote
} else if (!eof() && s[pos] == '\'') {
// Single-quoted value - real 1.4 content writes both
// (SESSIONCTL.tag's own name='tmp_#[name]' alongside
// ordinary name="..." elsewhere in the same file).
pos++;
while (!eof() && s[pos] != '\'') {
if (s[pos] == '\\' && pos + 1 < s.size()) { val += s[pos]; val += s[pos + 1]; pos += 2; continue; }
val += s[pos++];
}
if (!eof()) pos++; // closing quote
} else {
// Unquoted bareword value - real 1.4 content writes this
// too (type=eq, format=tobrowser, TYPE=NE throughout
// SESSIONCTL.tag) - the runtime interpreter's own
// attribute scanning already tolerates it; jetc's own
// parser previously didn't, since nothing had ever
// recursively parsed a tag definition's own body deeply
// enough to reach one before (see parseTagDef). Runs to
// the next whitespace or the tag's own closing '>'/'/>' .
while (!eof() && !std::isspace((unsigned char)s[pos]) && s[pos] != '>' &&
!(s[pos] == '/' && pos + 1 < s.size() && s[pos + 1] == '>')) {
val += s[pos++];
}
}
if (!eof()) pos++; // closing quote
attrs[aname] = val;
}
}
@@ -126,7 +268,40 @@ namespace jetc {
}
std::vector<NodePtr> Parser::parseSub(size_t start, size_t end) {
Parser sub(s.substr(start, end - start));
// Always topLevel=false - an <if>/<for>/<while> body is never the
// file's true top level, regardless of what topLevel this Parser
// itself was constructed with (see parseTagDef/isVoidTag's doc
// comments - a <tag name="X"> found here stays FALLBACK, unchanged).
Parser sub(s.substr(start, end - start), baseDir, includeDepth, false);
return sub.parseNodes();
}
std::vector<NodePtr> Parser::parseInclude() {
pos += 8; // "<include"
bool hasContainer;
// <include> is always void, the same way <set> is (see parseSet) -
// real content writes it bare (<include file="X">, no explicit
// '/>'), which parseAttrs reports as hasContainer=true since it
// can't tell that apart from a real container tag on syntax alone.
auto attrs = parseAttrs(hasContainer);
(void)hasContainer;
auto it = attrs.find("file");
if (it == attrs.end())
throw std::runtime_error("jetc: <include> must have a file attribute.");
if (includeDepth > 32)
throw std::runtime_error("jetc: <include> depth exceeded (possible include cycle) resolving " + it->second);
std::string fullPath = baseDir.empty() ? it->second : baseDir + "/" + it->second;
std::ifstream in(fullPath);
if (!in)
throw std::runtime_error("jetc: <include> cannot open " + fullPath);
std::ostringstream ss;
ss << in.rdbuf();
// Inherits the includer's own topLevel - an <include> spliced in at
// the file's true top level reads as if its content were written
// inline there (see this method's own doc comment), so a <tag
// name="X"> defined in the included file is just as eligible for
// compilation as one written directly in the includer.
Parser sub(ss.str(), baseDir, includeDepth + 1, topLevel);
return sub.parseNodes();
}
@@ -146,21 +321,19 @@ namespace jetc {
}
NodePtr Parser::parseSet() {
size_t origStart = pos;
pos += 4; // "<set"
bool hasContainer;
auto attrs = parseAttrs(hasContainer);
if (hasContainer) {
// Container-form <set> (value comes from rendering nested content)
// isn't modeled by this prototype - fall back to full interpretation
// of the whole tag instead of failing to compile.
auto span = scanBody(pos, "set", false);
pos = span.first + 3 + 3; // "</set>"
auto n = std::make_shared<Node>();
n->kind = NodeKind::FALLBACK;
n->rawSpan = s.substr(origStart, pos - origStart);
return n;
}
// jet-1.4's <set name=... value=... expr=...> is always a plain
// attribute-based assignment - it never had a container/body form.
// Real 1.4 content (config.cfg, SESSIONCTL.tag, throughout) writes
// it with a bare closing '>' and no explicit </set>, which is
// syntactically indistinguishable from a (speculative, and per the
// comment this replaced, never actually implemented) 2.0
// container-form <set>...</set>. Since the container form was never
// real to begin with, <set> is always treated as self-closing here,
// matching 1.4's actual, exclusive behavior rather than guessing.
(void)hasContainer;
auto n = std::make_shared<Node>();
n->kind = NodeKind::SET;
n->setName = attrs.count("name") ? attrs["name"] : "";
@@ -246,7 +419,7 @@ namespace jetc {
std::string tagName = peekTagName(pos);
size_t p = pos;
bool selfClosing = skipOpenTag(p);
if (!selfClosing) {
if (!selfClosing && !isVoidTag(tagName)) {
auto span = scanBody(p, tagName, false);
p = span.first + 3 + tagName.size();
}
@@ -257,6 +430,94 @@ namespace jetc {
return n;
}
NodePtr Parser::parseTagDef() {
size_t origStart = pos;
pos += 4; // "<tag"
bool hasContainer;
auto attrs = parseAttrs(hasContainer);
std::string name = attrs.count("name") ? attrs["name"] : "";
// A dynamic name ($[...]/#[...]) or a body-less definition can't be
// (and, per __tag.cpp, isn't even valid as) a compiled definition -
// fall back to the exact same FALLBACK behavior as before rather
// than failing to compile. Real content always uses a literal name.
bool dynamicName = name.find("$[") != std::string::npos || name.find("#[") != std::string::npos;
if (!hasContainer || name.empty() || dynamicName) {
if (hasContainer) {
auto span = scanBody(pos, "tag", false);
pos = span.first + 3 + 3; // "</tag>"
}
auto n = std::make_shared<Node>();
n->kind = NodeKind::FALLBACK;
n->rawSpan = s.substr(origStart, pos - origStart);
return n;
}
std::string lowerName = name;
for (char &c : lowerName) c = (char)std::tolower((unsigned char)c);
auto span = scanBody(pos, "tag", false);
size_t closePos = span.first;
size_t afterClose = closePos + 3 + 3; // "</tag>"
// Best-effort: this compiler's own modeled grammar is a real subset
// of what the runtime interpreter tolerates (jet-1.4 content is
// lenient in ways this parser doesn't fully cover yet - e.g. a
// completely unrecognized bare tag like SESSIONCTL.tag's own
// <setcookie ...>, which the interpreter just echoes as literal
// text rather than treating as a real tag needing a close - see
// Tag.cpp's own final dispatch fallback). Rather than teach every
// such case to this parser up front, fall all the way back to the
// exact same FALLBACK behavior a pre-TAGDEF compiler would have
// produced for this whole span whenever the body can't be parsed -
// this tag definition simply isn't compiled (stays fully
// interpreted, unchanged), with zero risk to anything else in the
// file. pos is never mutated by the failed parseSub call itself
// (it operates on its own copied substring), so no explicit
// rewind is needed before falling through.
std::vector<NodePtr> body;
bool bodyOk = true;
try {
body = parseSub(pos, closePos);
} catch (const std::exception &) {
bodyOk = false;
}
if (!bodyOk) {
pos = afterClose;
auto n = std::make_shared<Node>();
n->kind = NodeKind::FALLBACK;
n->rawSpan = s.substr(origStart, pos - origStart);
return n;
}
pos = afterClose;
auto n = std::make_shared<Node>();
n->kind = NodeKind::TAGDEF;
n->tagDefName = lowerName;
n->tagDefBody = body;
// Exact original source text - codegen still emits the same
// interpretSpan-based raw registration into global.tags alongside
// the new compiled fast path (see JetCodegen's TAGDEF case), so
// anything that reaches this tag some other way (a nested/locally-
// scoped definition shadowing it, or interpreted content elsewhere)
// keeps working exactly as it does today.
n->rawSpan = s.substr(origStart, pos - origStart);
return n;
}
NodePtr Parser::parseContainerSlot() {
pos += 10; // "<container"
bool hasContainer;
auto attrs = parseAttrs(hasContainer);
// Attributes on <container> (eval=/name=/scope=, as real content
// sometimes writes) are not consulted by the interpreter's own
// <container> dispatch branch either (see Tag::renderContainer) -
// matching that exactly rather than inventing new behavior here.
(void)attrs;
(void)hasContainer;
auto n = std::make_shared<Node>();
n->kind = NodeKind::CONTAINER_SLOT;
return n;
}
std::vector<NodePtr> Parser::parseNodes() {
std::vector<NodePtr> nodes;
std::string textBuf;
@@ -287,6 +548,14 @@ namespace jetc {
if (matchTagOpen("if", pos)) { flush(); nodes.push_back(parseIf()); continue; }
if (matchTagOpen("for", pos)) { flush(); nodes.push_back(parseFor()); continue; }
if (matchTagOpen("while", pos)) { flush(); nodes.push_back(parseWhile()); continue; }
if (matchTagOpen("include", pos)) {
flush();
auto included = parseInclude();
nodes.insert(nodes.end(), included.begin(), included.end());
continue;
}
if (matchTagOpen("container", pos)) { flush(); nodes.push_back(parseContainerSlot()); continue; }
if (topLevel && matchTagOpen("tag", pos)) { flush(); nodes.push_back(parseTagDef()); continue; }
flush();
nodes.push_back(parseFallback());
continue;