tls and master diverged from a July 2024 fork point and never
reconciled - tls carries ~19 months of real socket/TLS stability
fixes (uninitialized tlsInfo, TCPServer::connectionRequest()
unconditionally initializing TLS, the WANT_READ write busy-spin, the
console authentication fix from this session), while master
independently picked up a Subscription/SubscriptionManager rework
(MString-based ownership tracking: owner/ownership/selection/time
fields and a working subscriptionOwner() method) that tls never had.
Every file both branches touched was resolved by keeping tls's side
entirely (Command.h, Socket.cpp, TCPServer.h, TCPSession.cpp/h,
TCPSocket.h were pure reformatting on master's side, or an
alternate/superseded fix for a bug tls already fixed a different way -
e.g. master's TCPSession uuid/alias handling vs tls's existing
uuid_unparse-based fix), with two deliberate exceptions:
- TCPServer.cpp's constructor was missing the
commands.add(subscriptions, "publish"/"subscribe"/"catalog"/...)
wiring master had - without it, SubscriptionManager existed but was
completely unreachable through the normal command dispatch. Restored
on top of tls's TLS-aware constructor.
- SubscriptionManager.cpp's "invite" command handler had a real
incompatibility: master's version assigned into an `invitee` member
typed as coreutils::MString via a stray `(coreutils::MString *)`
cast that doesn't match the member's actual (non-pointer) type, and
called session.server.getSessionByAlias() against a TCPServer
reference - tls made TCPSession::server a pointer (part of removing
the separate TCPSession2 class), so that call site no longer
compiles as written. Took tls's cleaner local-variable version
instead (functionally identical - looks up a session by alias and
forwards an invite message) and removed the now-unused `invitee`
member from SubscriptionManager.h.
One additional compile fix needed after resolving conflicts:
Subscription::subscriptionOwners is an MString (master's ownership
design), but TCPSession::alias is void* (tls's identity mechanism,
unchanged and depended on elsewhere) - MString has an implicit
constructor from void* already used successfully elsewhere in this
same file (the "publish" handler's Subscription constructor call),
but operator== doesn't accept the same implicit conversion, so
removeSessionSubscriptions()'s ownership check needed an explicit
coreutils::MString(session.alias) construction to compile.
Also drops output/main (a stray checked-in compiled binary, already
removed on tls) and the doxygen docs/html output (already removed on
tls, was carried on master since the fork).
Verified: full ServerCore rebuild (./compile) succeeds with zero
errors or warnings beyond the one compile fix documented above.
Thread/EPoll lifecycle:
- Thread::~Thread() now actually joins/frees its worker thread (was
empty - the OS thread kept running against already-freed EPoll/Socket
objects on any early teardown, a confirmed "double free" crash)
- EPoll::~EPoll() calls stop() if never explicitly stopped, for the
same reason
- epoll_wait() now uses a 1s timeout instead of blocking forever, so a
worker thread can actually notice isStopping() and let itself be
joined
- EPoll::processCommand() iterated threads by value, silently
double-freeing the real vector's Thread the moment each copy went
out of scope
- ThreadScope gained onThreadStart()/onThreadEnd() hooks (actually
wired up now - the constructor that was supposed to store the scope
discarded it) for per-thread C-library init (e.g. MySQL client)
Socket/TCPSocket/TLS:
- Socket::writeSocket(): give up and shut down a connection after 50
consecutive zero-progress sends instead of retrying forever -
SSL_ERROR_WANT_READ on a write needs a READ to make progress, but
resetSocket() re-arms EPOLLWRNORM regardless, which an
already-writable TCP socket satisfies immediately - a real stuck
connection generated 55GB, then 12.9GB of debug log this way before
being caught
- Socket::writeSocket() now honors sendData()'s actual return value
(previously discarded, entry popped unconditionally after one call) -
a large response could get silently truncated past what fit in one
write()/SSL_write() call
- TLS::registerSocket()/sendData()/receiveData(): `initialized` now
only goes true once SSL_set_fd() has actually succeeded, and
sendData()/receiveData() check it first - fixes a real SIGSEGV where
a new session's own queued greeting could reach SSL_write() on a
different epoll thread before this session's handshake setup had run
- TCPServer::connectionRequest() now performs the TLS handshake ON THE
ACCEPTED SESSION, not the listening server itself (was building SSL
state on the wrong object, leaving every session's own `ssl`
uninitialized)
- TCPServer::receiveData() always uses the plain (non-TLS) read path -
the listening socket is never itself a TLS peer
- TCPSocket's block-mode literal reading used an unset `blockLength`
field instead of `blockSize`, and could busy-spin forever on a
multi-packet literal
- TCPSocket line scanning replaced an unbounded strcspn() (assumes a
NUL-terminated buffer; read past a non-NUL-terminated receive buffer
into adjacent heap memory) with a bounds-checked memchr() scan
Protocol/session handling:
- CommandList::processRequest() now case-folds the command keyword
before lookup - SMTP/POP3/IMAP commands are spec-required to be
case-insensitive; a client sending "Mail From:" instead of
"MAIL FROM:" previously failed to dispatch at all
- TCPSession::protocol() no longer drops blank lines while a handler
has grabbed input (e.g. __SMTP_DATA reading a message body, an IMAP
APPEND literal) - those handlers depend on seeing blank lines as
real content, not commands; this silently corrupted any message body
containing a blank line
- TCPSession's uuid alias fixed: streaming a raw uuid_t via operator<<
triggered libstdc++'s unsigned-char* overload (strlen() on 16 binary
bytes, a real stack-buffer-overflow read), and `alias` pointed at a
stack-local that was already destroyed by the time the constructor
returned
- INotify::processEvent() only treats event->name as a NUL-terminated
string when event->len > 0 (per inotify(7)) instead of unconditional
strlen() on a field the kernel doesn't guarantee is even present -
a confirmed heap-buffer-overflow read that had been misdiagnosed as
a MySQL thread-safety issue
connectionRequest() had a literal "if(true) { registerSocket(...);
acceptSocket(); }" guarding the TLS handshake setup for every accepted
connection, regardless of whether this TCPServer was ever given a
TLSInfo. TLS::createContext() only sets `ctx` when tlsInfo is non-NULL
(correctly guarded there) - so for any plain, non-TLS TCPServer, `ctx`
is left uninitialized, and this unconditional call reached
TLS::registerSocket() -> SSL_new(ctx) with garbage, segfaulting the
whole process on the very first connection.
Found while building APIServer (a new non-TLS HTTP server on top of
TCPServer) - confirmed via gdb (SIGSEGV inside SSL_new, called from
TLS::registerSocket, called from this exact line) that this affects ANY
non-TLS ServerCore-based server's first accepted connection, not
anything specific to the new code being tested against it.
Fixed by checking `tlsInfo` (already a TCPSocket-protected member this
class inherits) instead of the unconditional `true` - matches the guard
TLS::createContext() already uses correctly for the same distinction.
Verified live: before the fix, a non-TLS ApiServer segfaulted on the
first curl request every time; after, GET/HEAD/POST requests across
several real routes complete correctly and the server stays up for
repeated requests, including one that deliberately throws (confirming
the crash is really gone, not just less frequent).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Brad Arant
barant