jetc: add --shared mode to compile templates to a dlopen()-able .so

For the server you're about to build: `jetc --shared template.jet out.cpp`
now emits no main() at all - instead the render logic is exported as
`extern "C" void jetc_render_<template>(jet::Global &global,
coreutils::MString &out)`, one call per request, meant to be dlsym()'d
by a long-running host process instead of spawning a jet-2.0 process (or
even a separate compiled executable) per request. extern "C" here only
fixes the symbol's name mangling for a stable dlsym() target - the
parameter types are still real C++ classes, safe to pass across the
dlopen() boundary since the host and the .so share the same
CoreUtils/JetCore headers and toolchain (a same-ABI plugin boundary, not
a foreign C one). coreutils::Exception propagates out of the call
uncaught, so the host decides how to handle a render failure rather than
the module unilaterally deciding (the executable mode's try/catch prints
to stdout and exits, which is CGI-appropriate but wrong for a persistent
server). The exported symbol name is derived from the input filename
(jetc.cpp's baseNameNoExt+sanitizeIdent) so multiple compiled templates'
.so's stay collision-free even under dlopen(..., RTLD_GLOBAL).

jetc/build-shared.sh builds the .so: JetCore's 22 tag classes are still
force-linked directly (Tag::interpretSpan's fallback path can dispatch
to any of them at runtime, regardless of what a given template
statically references), but CoreUtils objects are compiled -fPIC and
archived into a throwaway libCoreUtils_pic.a instead of being linked
directly - libCoreUtils.a itself isn't position-independent, so linking
it into a .so fails outright ("relocation ... can not be used when
making a shared object"), and force-linking every CoreUtils .cpp
directly (rather than archiving, so the linker only pulls in what's
actually referenced) surfaced an unrelated, pre-existing incomplete
class - IMFMessage's vtable is never emitted anywhere, presumably a
partially-migrated class (see project memory on IMFMultipart/IMFText
still being fd-based) - as an unresolved dlopen()-time symbol, even
though nothing here uses it. A normal archive-based link (like the
existing libCoreUtils.a already does for jet-2.0) never pulls IMFMessage
in at all, so building libCoreUtils_pic.a the same way sidesteps it
without touching the unrelated bug itself.

jetc/host_test.cpp is a minimal stand-in for the real server: dlopen()s
a compiled .so, dlsym()s its entry point, and calls it twice with fresh
jet::Global objects (simulating two separate requests against one loaded
module) to prove the .so is genuinely reusable, not single-use. Verified
both demo templates (demo.jet, demo2.jet - the latter covering nested
if/for, a stateful while loop, and the interpretSpan fallback path) build
as .so's and produce the same byte-identical output as the executable
and interpreter paths, across two calls each. Also re-verified the
existing (non-shared) executable build path still matches jet-2.0's
output after the Codegen::generate() signature change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

jetc/JetCodegen.cpp

@@ -172,7 +172,8 @@ namespace jetc {
       }
    }
 
-   std::string Codegen::generate(const std::vector<NodePtr> &top, const std::string &sourceFileComment) {
+   std::string Codegen::generate(const std::vector<NodePtr> &top, const std::string &sourceFileComment,
+                                  bool sharedMode, const std::string &entrySymbol) {
       out.str("");
       out.clear();
       out << "// " << sourceFileComment << "\n";
@@ -184,6 +185,17 @@ namespace jetc {
       out << "#include \"MString.h\"\n";
       out << "#include \"Exception.h\"\n";
       out << "#include <iostream>\n\n";
+
+      if (sharedMode) {
+         out << "extern \"C\" void " << entrySymbol << "(jet::Global &global, coreutils::MString &out) {\n";
+         out << "   coreutils::ZString jetc_empty(\"\");\n";
+         out << "   coreutils::MString jetc_scopeOut;\n";
+         out << "   jet::Tag scope(jetc_empty, jetc_scopeOut, global, NULL, NULL);\n";
+         emitBody(top, 1);
+         out << "}\n";
+         return out.str();
+      }
+
       out << "static void jetc_render(jet::Global &global, coreutils::MString &out) {\n";
       out << "   coreutils::ZString jetc_empty(\"\");\n";
       out << "   coreutils::MString jetc_scopeOut;\n";

jetc/JetCodegen.h

@@ -16,7 +16,28 @@ namespace jetc {
    // the repeated re-parsing of the template's own structure.
    class Codegen {
     public:
-      std::string generate(const std::vector<NodePtr> &top, const std::string &sourceFileComment);
+      // sharedMode == false (default): emits a static jetc_render() plus a
+      // main() that runs it once, catches coreutils::Exception, and writes
+      // the result to stdout - the existing CGI-style standalone program.
+      //
+      // sharedMode == true: emits no main() at all. Instead the render
+      // logic is exported as `extern "C" void <entrySymbol>(jet::Global
+      // &global, coreutils::MString &out)`, meant to be built into a .so
+      // (see build-shared.sh) and dlsym()'d by a long-running host server -
+      // one call renders one request into the caller-owned `out`, using
+      // whatever Global the host passes in (its lifecycle, cgi/session
+      // setup etc. is entirely the host's business, not this module's).
+      // extern "C" only affects name mangling here, not the parameter
+      // types - both sides are compiled from the same CoreUtils/JetCore
+      // headers with the same toolchain, so passing C++ class types (and
+      // letting coreutils::Exception propagate to the caller) across the
+      // dlopen() boundary is safe; this is a same-ABI plugin boundary, not
+      // a foreign C ABI one. entrySymbol should be unique per compiled
+      // template (see jetc.cpp) so multiple .so's can coexist even if a
+      // host ever dlopen()s them RTLD_GLOBAL instead of keeping separate
+      // handles.
+      std::string generate(const std::vector<NodePtr> &top, const std::string &sourceFileComment,
+                            bool sharedMode = false, const std::string &entrySymbol = "jetc_render");
 
     private:
       std::ostringstream out;

jetc/build-shared.sh

@@ -0,0 +1,80 @@
+#!/bin/bash
+# Compiles a .jet file into a shared object exporting a stable, dlsym()-able
+# render entry point - `jetc_render_<basename>(jet::Global &global,
+# coreutils::MString &out)` - for a long-running server to load and call
+# directly instead of spawning a jet-2.0 process per request.
+#
+# Usage: ./build-shared.sh <input.jet> <output.so>
+#
+# The host server links/dlopen()s the .so and calls the exported symbol
+# (see the "entry symbol:" line jetc prints) with its own jet::Global -
+# this script does not decide that lifecycle, only builds the module.
+# coreutils::Exception propagates out of the call uncaught; the host
+# decides how to handle a rendering failure (log it, 500, etc.).
+
+set -e
+JC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+CU="$JC/../CoreUtils"
+HERE="$JC/jetc"
+
+if [ $# -ne 2 ]; then
+   echo "usage: $0 <input.jet> <output.so>"
+   exit 1
+fi
+
+INPUT="$1"
+OUTPUT="$2"
+STEM="$(basename "$OUTPUT" .so)"
+
+if [ ! -x "$HERE/jetc" ]; then
+   echo "Building jetc..."
+   g++ -g -std=c++23 -o "$HERE/jetc" "$HERE/jetc.cpp" "$HERE/JetParser.cpp" "$HERE/JetCodegen.cpp"
+fi
+
+echo "Compiling $INPUT -> $STEM.compiled.cpp (shared mode)"
+"$HERE/jetc" --shared "$INPUT" "$HERE/$STEM.compiled.cpp"
+
+echo "Building JetCore runtime objects as position-independent code..."
+list=""
+for f in "$JC"/*.cpp; do
+   base=$(basename "$f")
+   if [ "$base" = "jet-2.0.cpp" ]; then continue; fi
+   obj="$HERE/${base%.cpp}.pic.o"
+   if [ ! -f "$obj" ]; then
+      g++ -g -fPIC -c -std=c++23 -I"$CU" -I"$JC" "$f" -o "$obj" &
+   fi
+   list="$list $obj"
+done
+
+# libCoreUtils.a (built by CoreUtils/compile) is not position-independent -
+# ld refuses to link non-PIC static-library code into a shared object
+# ("relocation ... can not be used when making a shared object"). Rather
+# than touching CoreUtils's own build (its .a is fine and used as-is by
+# jet-2.0 and everything else), build a PIC copy of its objects here and
+# archive them - archived the same way libCoreUtils.a itself is, so the
+# linker only pulls in objects something actually references, same as a
+# normal -lCoreUtils link. This matters: force-linking every CoreUtils
+# .cpp directly (as JetCore's own objects are, below) surfaces an
+# unrelated, pre-existing incomplete class (IMFMessage - a known
+# partially-migrated class, see project memory) as an unresolved vtable
+# at dlopen() time, even though nothing here actually uses it - a normal
+# archive link never pulls IMFMessage.o in at all, so it never hits this.
+cu_pic_list=""
+for f in "$CU"/*.cpp; do
+   base=$(basename "$f")
+   obj="$HERE/cu_${base%.cpp}.pic.o"
+   if [ ! -f "$obj" ]; then
+      g++ -g -fPIC -c -std=c++23 -I"$CU" "$f" -o "$obj" &
+   fi
+   cu_pic_list="$cu_pic_list $obj"
+done
+wait
+ar rcs "$HERE/libCoreUtils_pic.a" $cu_pic_list
+
+g++ -g -fPIC -c -std=c++23 -I"$CU" -I"$JC" "$HERE/$STEM.compiled.cpp" -o "$HERE/$STEM.compiled.pic.o"
+
+echo "Linking $OUTPUT..."
+g++ -shared -fPIC -o "$OUTPUT" "$HERE/$STEM.compiled.pic.o" $list -std=c++23 \
+   -L"$HERE" -lCoreUtils_pic -lmysqlclient -lcrypto
+
+echo "Built $OUTPUT"

jetc/host_test.cpp

@@ -0,0 +1,51 @@
+// Minimal stand-in for the not-yet-built server: dlopen()s a compiled
+// template .so and calls its exported render entry point directly, the
+// way a persistent content-provider process would (no process spawn per
+// request, no re-parsing the .jet source - both the compile step AND
+// the shared-object load happen once, ahead of time).
+#include "Global.h"
+#include "MString.h"
+#include "Exception.h"
+#include <dlfcn.h>
+#include <iostream>
+#include <cstdlib>
+
+typedef void (*RenderFn)(jet::Global &, coreutils::MString &);
+
+int main(int argc, char **argv, char **envp) {
+   if (argc < 3) {
+      std::cerr << "usage: host_test <module.so> <symbol>" << std::endl;
+      return 1;
+   }
+   void *handle = dlopen(argv[1], RTLD_NOW);
+   if (!handle) {
+      std::cerr << "dlopen failed: " << dlerror() << std::endl;
+      return 1;
+   }
+   dlerror();
+   RenderFn render = (RenderFn)dlsym(handle, argv[2]);
+   const char *err = dlerror();
+   if (err) {
+      std::cerr << "dlsym failed: " << err << std::endl;
+      return 1;
+   }
+
+   // Simulate the host calling the SAME module twice, for two different
+   // "requests", to prove the .so is reusable across calls without being
+   // reloaded - the actual point of a persistent content provider.
+   for (int i = 0; i < 2; ++i) {
+      jet::Global global(envp);
+      coreutils::MString out;
+      try {
+         render(global, out);
+         global.outputHeaders();
+         std::cout << "--- render call " << i << " ---" << std::endl;
+         std::cout << out;
+      } catch (coreutils::Exception &e) {
+         std::cout << "render call " << i << " threw: " << e.text << std::endl;
+      }
+   }
+
+   dlclose(handle);
+   return 0;
+}

jetc/jetc.cpp

@@ -3,15 +3,47 @@
 #include <fstream>
 #include <sstream>
 #include <iostream>
+#include <cctype>
+#include <vector>
+
+namespace {
+
+   std::string baseNameNoExt(const std::string &path) {
+      size_t slash = path.find_last_of("/\\");
+      std::string base = (slash == std::string::npos) ? path : path.substr(slash + 1);
+      size_t dot = base.find_last_of('.');
+      if (dot != std::string::npos) base = base.substr(0, dot);
+      return base;
+   }
+
+   std::string sanitizeIdent(const std::string &s) {
+      std::string r;
+      for (char c : s) r += (std::isalnum((unsigned char)c) || c == '_') ? c : '_';
+      return r;
+   }
+
+}
 
 int main(int argc, char **argv) {
-   if (argc < 3) {
-      std::cerr << "usage: jetc <input.jet> <output.cpp>" << std::endl;
+   bool shared = false;
+   std::vector<std::string> pos;
+   for (int i = 1; i < argc; ++i) {
+      std::string a = argv[i];
+      if (a == "--shared") shared = true;
+      else pos.push_back(a);
+   }
+   if (pos.size() < 2) {
+      std::cerr << "usage: jetc [--shared] <input.jet> <output.cpp>" << std::endl;
+      std::cerr << "  --shared  emit a dlsym()-able entry point (see build-shared.sh)" << std::endl;
+      std::cerr << "            instead of a standalone main()." << std::endl;
       return 1;
    }
-   std::ifstream in(argv[1]);
+   const std::string &inputPath = pos[0];
+   const std::string &outputPath = pos[1];
+
+   std::ifstream in(inputPath);
    if (!in) {
-      std::cerr << "jetc: cannot open " << argv[1] << std::endl;
+      std::cerr << "jetc: cannot open " << inputPath << std::endl;
       return 1;
    }
    std::ostringstream ss;
@@ -30,11 +62,20 @@ int main(int argc, char **argv) {
       jetc::Parser parser(src);
       auto nodes = parser.parseNodes();
       jetc::Codegen codegen;
-      std::string cpp = codegen.generate(nodes, std::string("compiled from ") + argv[1]);
-      std::ofstream outFile(argv[2]);
+      // Symbol is derived from the input filename (not the output path) so
+      // it stays stable and predictable for whatever builds and dlsym()'s
+      // the resulting .so - unique per template, so multiple compiled
+      // templates' .so's can coexist in one process even under
+      // dlopen(..., RTLD_GLOBAL) instead of requiring isolated handles.
+      std::string symbol = "jetc_render_" + sanitizeIdent(baseNameNoExt(inputPath));
+      std::string cpp = codegen.generate(nodes, std::string("compiled from ") + inputPath, shared, symbol);
+      std::ofstream outFile(outputPath);
       outFile << cpp;
       outFile.close();
-      std::cout << "jetc: wrote " << argv[2] << std::endl;
+      if (shared)
+         std::cout << "jetc: wrote " << outputPath << " (entry symbol: " << symbol << ")" << std::endl;
+      else
+         std::cout << "jetc: wrote " << outputPath << std::endl;
    } catch (std::exception &e) {
       std::cerr << "jetc: " << e.what() << std::endl;
       return 1;