6b188fbee5d0a2d8b1e587d3e6b26d57d3a9d0d9
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brad ArantandClaude Sonnet 5
|
6b188fbee5 |
Restore execute bit on testing/compile (lost via UNC-path edit)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Brad ArantandClaude Sonnet 5
|
cb845efe69 |
Add HTTPRequest/HTTPResponse: real HTTP parsing on the Parser/StreamReader stack
Stage 2 of the APIServer plan. This codebase has only ever sat behind a CGI gateway - IMFRequest/IMFResponse/IMFMessage predate the StreamReader/Parser rebuild and are stubs (IMFResponse's header-emission calls addHeader()/output(), neither of which exist on the class; IMFRequest is a fragile non-resumable 3-token line split). Nothing in this repo has ever actually parsed a raw HTTP request off a socket. HTTPRequest : public Parser fills that gap the same way every other Parser here is built: REQUEST_LINE -> HEADERS -> BODY -> DONE, chaining LineParser (request line) and IMFHeader (headers, reused as-is - HTTP header syntax is IMF-derived, nothing email-specific in IMFHeader itself) via checkParse() over one shared StreamReader, exactly the pattern CGIFormData already established for chaining IMFHeader/ IMFBoundary. The BODY state dispatches on Content-Type as CGIFormData itself does for CGI bodies - application/x-www-form-urlencoded and multipart/form-data delegate directly to CGIFormData (reused, not reimplemented), anything else is captured as a raw body (e.g. JSON). Query string is split off the path before anything else sees it. getHeader() does a case-insensitive lookup (HTTP header names are case-insensitive per RFC 7230, unlike IMFHeader's own std::map, which compares byte-for-byte) since a route handler constantly needs this and getting it wrong would be a silent framing bug, not a cosmetic one. The boundary-extraction code mirrors __jet.cpp's own fix exactly (same cursor-mutation trap: ZString::find() leaves the searched string's cursor at the match, and CGIFormData's constructor copy-constructs its own contentType member from whatever's passed in, inheriting that same moved cursor unless reset() first - would otherwise silently miss its own startsWith() dispatch checks). HTTPResponse is a small, new, output-only builder (status line + a real headers container + body, Content-Length auto-computed unless set explicitly) - deliberately not patched from IMFResponse, which has nothing safe to extend. v1 limitations, flagged not fixed (matching the standard already set by other Parser subclasses here for known-but-out-of-scope gaps): chunked Transfer-Encoding isn't supported, and repeated header names last-wins (inherited from IMFHeader's own std::map). Verified live via testing/http_test.cpp (34 assertions, new - now wired into testing/compile): byte-at-a-time GET with a query string and case-insensitive header lookup; a GET with no query string; a POST with an urlencoded body deliberately split mid-body (not just mid-header) across two feed() calls, confirming the CGIFormData delegation resumes correctly; a raw JSON POST body; testing/http_request.data (a real fixture, previously unused - normalized from its stored LF line endings to real CRLF, since HTTP requires CRLF and IMFHeader enforces that strictly) fed byte-at-a-time; and HTTPResponse::serialize() both with an auto-computed and an explicitly-set Content-Length (no duplicate header). Also reran the full existing testing/compile suite (zstring/mstring/jstring/mfile/jfile) against the rebuilt libCoreUtils.a to confirm no regression from the two new files. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Brad ArantandClaude Sonnet 5
|
6e4e463a96 |
Fix MString assignment operators leaving cursor at its old (stale) offset
operator=(MString)/operator=(ZString)/operator=(std::string)/operator=(const char*)/operator=(char) all replaced the buffer's content via setSize()+ memcpy() but never reset cursor back to the start - setSize() preserves the *offset* the cursor was already at (correct for its own use, growing a buffer while keeping a read position stable), so a full content replacement left the cursor sitting at whatever offset it had into the *previous* content, not the new content. operator=(double)/operator=(int) already did `cursor = data;` after the same setSize()+memcpy() pattern, confirming this was the intended behavior everywhere - just missing from the other five overloads. This is invisible to JetCore's interpreter, which never reassigns an already-read-from MString (each tag is a freshly parsed C++ object every time its container is scanned, so a keyword's MString is never both read and then reassigned on the same instance). It surfaced in a prototype JET compiler (JetCore/jetc) built this session, which intentionally reuses one persistent keyword slot across loop iterations instead of reconstructing everything from text every pass - exactly the case this bug needed. A <while expr="..."> loop reusing a slot for its condition worked for two iterations (matching a decoy pattern: the second assignment happened to land the stale cursor at the exact end of the new, same-length content) and threw "operand is not valid" on the third, since a keyword reassigned inside the loop body left the cursor already at eod() from the read. Verified via jstring_test/mstring_test/zstring_test/parser_test (no regressions), JetCore's tests/posttest.sh and the two-process JSON session persistence test (no regressions), and the jetc prototype's two demo templates - both now produce byte-identical output to jet-2.0's interpreter, including a <while> loop that mutates and rereads the same keyword slot every iteration. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Brad ArantandClaude Sonnet 5
|
3e6e410fe8 |
Fix JFile empty-document seed and MString's ZString-source pointer/length mismatch
JFile.cpp: JFile and MFile both "virtual public MString" a single shared
base, so a virtual base is initialized only by the *most derived* class's
constructor. JFile's own constructor never mentions MString, so
JString's own "{}" default initializer never ran - a JFile backed by a
new or nonexistent file started as a raw empty string instead of a valid
empty JSON object. Every subsequent locate()/set() call then silently
misbehaved (no leading "{" for ifNext("{") to match, so writes landed at
offset 0 with no braces or separators - fields ended up prepended in
reverse order, concatenated with no delimiters at all). Fixed by seeding
"{}" via append (matching MFile's own load pattern) whenever the loaded
content is empty.
MString.cpp: several ZString-source copy paths (the ZString/MString copy
constructors, both assignment operators, operator<<(ZString), insert(),
write()) called zstring.getData() (the object's absolute start) paired
with zstring.getLength() (which is documented/implemented as "remaining
bytes from cursor to end", i.e. length - (cursor - data)). Pairing an
absolute start with a cursor-relative length silently corrupts any
cursor-advanced source: the copy starts from the wrong (too-early)
position but is truncated by exactly the amount the cursor had already
advanced, keeping unwanted leading bytes while dropping trailing ones.
Found via the CGIFormData-style resumability testing pattern used
elsewhere this session, triggered by JetCore's new "@name" session
variable feature stripping a sigil prefix via ifNext() before streaming
the remainder into an MString. Fixed by pairing zstring.getCursor()
(cursor-relative start) with zstring.getLength() consistently everywhere
this pattern appeared.
Verified via the existing jstring_test/mstring_test/zstring_test/
parser_test suites (all pass, no output changes) plus a new isolated
JFile round-trip test proving persistence across two separate process
instances sharing one file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Brad ArantandClaude Sonnet 5
|
5b6d4ea8ab |
Fix ambiguous MString << JString::Proxy overload resolution
JString::Proxy offers several implicit conversion operators (ZString, MString, std::string, double, bool) so a single $[...]-style access can be used wherever any of those types is expected. That flexibility made "MString << Proxy" genuinely ambiguous: operator ZString() and operator MString() (via its own implicit upcast to ZString) both satisfy MString::operator<<(ZString), and operator double()/operator bool() both satisfy MString::operator<<(int) via a standard numeric conversion, so the compiler had no basis to prefer one conversion path over another. Fixed the same way Proxy already solves this exact problem for std::cout << proxy - a dedicated friend operator<<(MString&, const Proxy&) that needs no conversion at all, so it's always the preferred candidate. Verified by actually running testing/jstring_test.cpp (not just compiling it) - the line that used to fail to compile now correctly prints "this is a test: this is a comment". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Brad ArantandClaude Sonnet 5
|
bf6810b781 |
Rebuild CGIFormData on Parser/StreamReader, fix a 2-byte-lookahead bug
CGIFormData now inherits Parser instead of MString and parses a CGI
request body (urlencoded or multipart/form-data) resumably against the
shared StreamReader, exposing the result via operator[] - matching every
other Parser in this redesign. multipart/form-data reuses IMFBoundary and
IMFHeader via checkParse() (chained through the same StreamReader) for
each part's boundary line and header block; urlencoded bodies wait for
the full Content-Length before delegating to the existing
CGIFormattedData parser. File-upload parts (Content-Disposition with a
filename= parameter) are recognized and skipped, matching this
codebase's convention that uploads get a separate mechanism.
Values are stored under an indexed "name:N" key (N = 0-based occurrence
count for that name), matching the :N-suffix convention JetCore's
Tag::getVariable already applies to its $[:name] CGI variable syntax
(defaulting to :0) - this also naturally supports repeated field names
(checkboxes, multi-select). operator[] applies the same :0 default so a
bare name still works when called directly, not just through that
convention layer.
Two real resumability bugs found and fixed via a byte-at-a-time test
(feeding one byte per StreamReader::feed() call, the harshest possible
fragmentation):
- CGIFormData's opening boundary marker ("--boundary", no leading CRLF
per RFC 2046 for the first boundary) crammed "find+consume the marker"
and "disambiguate -- vs CRLF" into one state; when the disambiguation
needed more data, INCOMPLETE was returned without recording that the
marker was already consumed, so retrying re-searched for it past where
it had already been consumed, silently skipping content forever. Split
into two states (INIT / OPENING_TAIL), the same fix already applied
once to IMFBoundary's PREFIX/TAIL split.
- IMFHeader's FIELD state checked eod() (a 1-byte availability check)
immediately before an ifNext("\r\n") call that needs 2 bytes - a lone
trailing \r with no \n yet available was wrongly treated as malformed
instead of "wait for more data". Fixed to check getLength() < 2.
Verified via a dedicated test (64 assertions total across this and the
earlier Parser/StreamReader test) covering urlencoded, multipart with a
skipped file upload, and the byte-at-a-time stress case - plus a real
end-to-end run through jet-2.0 with a simulated CGI multipart POST
request (see the matching JetCore commit).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Brad ArantandClaude Sonnet 5
|
a033cc6718 |
Add StreamReader and rebuild Parser/LineParser/IMFHeader/IMFBoundary on it
Introduces StreamReader to accumulate socket data delivered in arbitrary chunks into one buffer, so a Parser can build an object (a line, a header block, a MIME boundary) that spans any number of packets without losing its place. Parser is redesigned around this: no longer inherits MString (was redundant with its own buffer member), takes a StreamReader&, and drives a resumable COMPLETE/INCOMPLETE/DATA_AVAIL state-machine protocol via doParse(). LineParser, IMFHeader, and IMFBoundary are rewritten as real, working resumable parsers against the shared buffer instead of the previous non-compiling stubs (mismatched constructors/return types, a syntax error, checkParse() never implemented). IMFFormData is left as the stub it already was, just restored to compiling against the new IMFHeader signature - fully wiring it up needs IMFMultipart/IMFText migrated to StreamReader too, out of scope here. Verified with real runtime tests (not just compilation) covering data split mid-line, mid-CRLF, mid-header-fold, and mid-boundary-marker across separate feed() calls, plus StreamReader::compact(). Along the way fixed a header-value bug (space after ':' wasn't skipped) and a double cursor-offset subtraction bug in two places caused by forgetting that ZString::getLength() is already cursor-relative, not the raw buffer size. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Brad ArantandClaude Sonnet 5
|
94d4a30e88 |
Fix CGI form/cookie data handling and CGI URL encode/decode
- CGIFormData: fix broken setSize()+read() double-length bug that corrupted every parsed body with a leading run of garbage bytes (setSize(contentLength) pre-set length before read()/write() appended on top of it); switch content-type checks from exact equality to startsWith() since a real multipart Content-Type always carries a boundary= parameter; implement the previously empty multipart/form-data branch (ordinary fields only, file uploads deferred to a separate upload mechanism per TODO.txt); restore operator[] real lookup logic (was hardcoded to echo the key back); default-initialize formdata to NULL; fix mismatched include guard (leftover from an IMFFormData.h copy-paste); drop the unused IMFHeader.h include that pulled in a broken, unrelated StreamReader.h dependency. - CGIFormattedData: parameterize the split delimiter (was hardcoded to &, which silently broke cookie parsing since HTTP_COOKIE uses ; not &) and trim whitespace on each split token. - MString::toCGI()/fromCGI(): were unimplemented stubs (always returned an empty string); implemented proper CGI percent-encode / percent-decode + space plus-encoding, and fixed the one caller that discarded fromCGIs return value instead of using it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Brad Arant
|
28feaff3d7 |
Rewrite JString for real JSON compliance
The previous implementation was a narrow path-accessor, not a JSON parser:
numbers/booleans/null were unhandled entirely (self-acknowledged by a
// TODO: Handle numbers... comment), string escape sequences were never
encoded or decoded, updating an existing non-string-valued key threw a
hard-coded Fix me exception, and skipping past an uninteresting key threw
Unrecognized data type for anything but string/array/object. Full
analysis given to the user before this rewrite; nothing here was guessed.
Added:
- JSONType enum + getType()/exists()/isNull() for type introspection.
- Typed read access: getString() (escape-decoded), getNumber(), getInt(),
getBool() - each validates the value at the path is actually that type.
- Typed write access: setString() (escapes automatically), setNumber(),
setInt(), setBool(), setNull() - and Proxy operator= overloads for
double/int/long/bool so produces a real JSON number, not
a quoted string the way the old single MString-typed operator= did.
- jsonEscape()/jsonUnescape(): full escape codec including \uXXXX and
UTF-16 surrogate pairs (astral characters, e.g. emoji) decoded to
correct UTF-8.
- skipValue()/skipString()/skipContainer()/skipNumber() and their
capture (span-returning) counterparts: escape-aware structural
skipping, so a string value containing a literal { } [ ] , or : can
no longer desync object/array nesting depth - this affected both
locate()s internal key-skipping and find()s array/object extraction
(the latter still relied on the base ZString::getContainer(), which
has this exact bug and was not touched - see the ZString commit in
this same session - so a JString-specific escape-aware capture was
added instead of depending on it for this purpose).
- set() rewritten to use skipValue() to correctly locate and replace an
EXISTING values full span regardless of its current type, instead of
only handling the case where the existing value was already a quoted
string.
Bugs found and fixed along the way, surfaced by writing real tests
against realistic JSON rather than just getting it to compile:
- The array-index-then-further-path case (e.g. b[2].c) always returned
true immediately after reaching the index, so any path segment past
the index was silently ignored - dead code after an unconditional
return/throw meant the intended continuation logic could never run.
- The array selector syntax (label[key=value]) was only reachable as a
top-level special case, never as part of a normal nested path (e.g.
items[id=x].val) - unified both under the same bracket handling.
- Once reachable, the selector-match subkey check (find(key) on each
array element) always searched from the document root, not the
current array element, because find() unconditionally calls reset().
Split into public find() (resets) and internal findRelative() (does
not), and pointed both selector-matching call sites at the latter.
- path[0].getTokenExclude(...) in the array-selector path parser
indexed into a list vector that split() never populated - undefined
behavior. Removed the erroneous [0].
Verified via a dedicated test program (not included in this commit)
covering: string escape round-trip, numbers, booleans, null, exists(),
\uXXXX including a real surrogate pair decoded to correct 4-byte UTF-8,
nested dotted+bracketed navigation, a string value containing every
structural JSON character, replacing an existing key with a different
type, multi-key documents, pretty() not corrupting structural characters
inside a string, and the array selector syntax on both a matching first
and second element - 30/30 assertions pass. Full CoreUtils compile pass
unchanged apart from JString.cpp itself now succeeding (14/20 clean, same
7 pre-existing StreamReader.h-blocked failures as always, nothing new
broken).
|
||
|
Brad Arant
|
14697e2af2 |
Fix three real bugs found during a full ZString review
operator< computed end2 as valuex.getData() + valuex.getLength() - mixing an absolute pointer (getData(), the buffer start) with a cursor-relative length (getLength(), remaining-from-cursor). For any ZString with a non-zero cursor offset this produces an end pointer short of the true buffer end by exactly the amount already consumed, truncating the comparison early. Fixed to use valuex.data + valuex.length (the true end), matching how end1 was already computed for *this. operator<=, >=, and > were separately broken (flagged by the codes own TODO comment) - they compared via strncmp from data (ignoring cursor entirely), inconsistent with operator< and with each other. Replaced all three with derivations from the now-correct operator< (a <= b is !(b < a), etc.) - guarantees trichotomy by construction instead of three independently-written comparison loops. substring(start, len): when the requested length overran the buffer, computed newlen as endChar - end (the overrun amount) instead of end - startChar (the actual remaining valid length) - backwards, and stored into a char (max 127), silently wrapping for longer strings. Fixed the formula and widened to size_t. trimCRLF(): read *(data + length - 1) unconditionally, including when length is already 0 - one byte before the buffer on an empty string. Guarded both checks on length > 0. Verified: full local compile (identical result to before - 13/20 clean, same pre-existing StreamReader.h-blocked failures, nothing new broken); a dedicated runtime test exercising all three fixes plus a trichotomy check across multiple cursor offsets - all 13 assertions pass. Test file not included in this commit. |
||
|
Brad Arant
|
4082e82466 |
Fix Exception default file/line always reporting Exception.h itself
__FILE__/__LINE__ are preprocessor macros - as default argument values in the header they expand once, at the header declaration site, not at each call site. Every throw relying on the defaults (all but File.cpp, which passed them explicitly) reported Exception.h itself instead of the real error location. Switched the defaults to std::source_location::current(), which the C++20 standard specifically defines to evaluate at the call site when used as a default argument - unlike macros. Kept the same 4-parameter signature so every existing call site, including ones that pass __FILE__/__LINE__ explicitly, is unaffected. Verified two ways: full-repo compile pass (identical result to before, 13/20 clean, same pre-existing StreamReader.h-blocked failures, nothing new broken); a real runtime test - a throw relying on defaults from a known line correctly reported that exact file/line via the caught exceptions .file/.line members. Test file not included in this commit. |
||
|
Brad Arant
|
29c9a5b855 |
Fix MString::fromHex() silently swallowing the odd-length-input error
The Exception on the odd-length-input path was constructed but never thrown (missing throw keyword) - the temporary was destroyed with no effect, so execution fell through to an out-of-bounds nextChar() read past the exhausted buffer instead of raising an error. Added the missing throw. Verified via full local compile: MString.cpp builds clean standalone; a full-repo pass shows only the pre-existing, already-known StreamReader.h-missing failures (7 files, unrelated to this change), confirmed via each error log individually. |