Files
ServerCore/TCPServer.cpp
T
Matt Arant d058322cc6 Move TCPSession lifetime to shared_ptr ownership, fix the whole double-delete/UAF class of crash
Every crash root-caused earlier tonight (Subscription::subscribers,
TCPServer::sessions, the Socket::write()/eventReceived() re-arm race) was
the same underlying shape: some thread calling a method - or Thread::run()
calling delete - on a TCPSession* while another thread concurrently freed
that same object, because nothing actually guaranteed the object stayed
alive for the duration of a call. Mutex-based patches at each individual
call site kept narrowing the window without closing it (confirmed live,
repeatedly, via AddressSanitizer under concurrent load - each fix produced
a new, different crash signature at the same underlying race), and adding
more locks around every write() call also works against the fifo-based
design's own goal: no thread should have to block just to hand a message
to another thread's session.

TCPSession now derives from std::enable_shared_from_this<TCPSession> and
exposes pin() - a safe upgrade from a stored raw pointer to a real,
ownership-extending shared_ptr, returning nullptr instead of throwing once
the object's real owner has let go. TCPServer::sessions is now the one
real strong-owning shared_ptr<TCPSession> per live session; every other
place that stores a TCPSession* (Subscription::subscribers, Zephora's own
sessionsByPlayerName, GroupObject's membership list, etc.) is just a
non-owning observer and must pin() before touching the object.

A second, distinct problem: Thread::run()'s epoll dispatch starts from a
raw void* of unknown validity. An early version of this fix used
dynamic_cast<TCPSession*> to identify session objects before calling
pin() - confirmed live via AddressSanitizer that dynamic_cast itself
segfaults on a stale pointer, since it has to read the (potentially freed)
object's own vtable to work at all. TCPSession::tryGet()/TCPServer::
isKnownListeningSocket() replace that: both look the raw pointer up as a
key in a registry, never dereferencing it for the check itself - safe to
call on any value, live session, immortal listening socket, or fully
freed garbage alike.

Verified via an AddressSanitizer-instrumented build under sustained
concurrent load (up to 15-way parallel connect/subscribe/create/disconnect
cycles, multiple rounds, 100+ total connections) with zero errors, then
against the real production build with a genuine 500-connection load test
(463/500 succeeded; the rest hit the process's own file-descriptor
ulimit under peak burst, an unrelated and separately deferred tuning
matter, not a crash).

Files touched: TCPSession.h (enable_shared_from_this, pin(), the tryGet()
registry), TCPServer.h/.cpp (sessions as shared_ptr, isKnownListening
Socket() registry, lock made protected for subclasses), Thread.cpp (the
whole dispatch rewrite), Subscription.cpp/SubscriptionManager.cpp (every
subscriber write() site now pin()s first), ConsoleServer.cpp (logSend()
was iterating `sessions` with no lock at all - fixed alongside the type
change it needed anyway).
2026-09-08 02:24:09 +00:00

222 lines
9.5 KiB
C++

#include "TCPServer.h"
#include "EPoll.h"
#include "Exception.h"
#include "Log.h"
#include "TCPSession.h"
namespace core {
TCPServer::TCPServer(EPoll &ePoll, IPAddress address, TLSInfo *tlsInfo, std::string delimiter, int depth, std::string text)
: TCPSocket(ePoll, tlsInfo, text), commands(delimiter, depth) {
// Wires the subscription (pub/sub) command set into this server's
// own command dispatch - previously missing entirely here (the
// subscriptions member existed but nothing ever routed these verbs
// to it), so publish/subscribe/catalog/etc were unreachable on any
// TCPServer built after the TLS work, even though SubscriptionManager
// itself was still fully implemented.
commands.add(subscriptions, "publish");
commands.add(subscriptions, "unpublish");
commands.add(subscriptions, "subscribe");
commands.add(subscriptions, "unsubscribe");
commands.add(subscriptions, "catalog");
commands.add(subscriptions, "event");
commands.add(subscriptions, "invite");
setDescriptor(socket(AF_INET, SOCK_STREAM, 0));
int yes = 1;
setsockopt(getDescriptor(), SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(int));
if (bind(getDescriptor(), address.getPointer(), address.addressLength) < 0)
throw coreutils::Exception("Error on bind to socket: " + std::to_string(errno));
if (listen(getDescriptor(), 20) < 0)
throw coreutils::Exception("Error on listen to socket");
// See isKnownListeningSocket()'s own doc comment - registers this
// listening socket as safe for Thread::run() to dispatch to
// directly, distinct from the shared_ptr-owned TCPSession registry.
{
std::lock_guard<std::mutex> guard(listeningSocketsMutex());
listeningSockets().insert(this);
}
}
TCPServer::~TCPServer() {
coreutils::Log(coreutils::LOG_DEBUG_2) << "Closing server socket " << getDescriptor() << ".";
close(getDescriptor());
}
void TCPServer::connectionRequest() {
coreutils::Log(coreutils::LOG_DEBUG_2) << "Connection request is being received on socket " << getDescriptor() << ".";
lock.lock();
TCPSession *session = accept();
// The shared_ptr constructed here IS the session's real, strong
// ownership from now on (see sessions' own doc comment). A second
// reference to the SAME shared_ptr (never a second, independent
// std::shared_ptr<TCPSession>(session) call on the same raw
// pointer - that would create a second, competing control block
// and a guaranteed double-free) also goes into the tryGet()
// registry, which is what Thread::run()'s epoll dispatch actually
// uses to safely reach this object (see TCPSession::tryGet()'s
// own comment on why).
if (session) {
auto sessionPtr = std::shared_ptr<TCPSession>(session);
sessions.push_back(sessionPtr);
TCPSession::registerLive(session, sessionPtr);
}
// Must run ON THE SESSION, not `this` (the listening server) - the
// unqualified registerSocket()/acceptSocket() calls this replaced
// built the SSL* handshake state on the SERVER's own TLS-base
// member instead of the newly accepted session's, leaving every
// session's own `ssl` uninitialized and segfaulting the first time
// real data arrived on it (SSL_read() against a never-SSL_new()'d
// pointer - confirmed via gdb). Sessions already correctly inherit
// this server's SSL_CTX at construction (TCPSession's own
// TLS(SSL_CTX*)-taking constructor) - performTLSHandshake() just
// needed to actually be invoked on that object.
if (session && tlsInfo)
session->performTLSHandshake();
lock.unlock();
}
void TCPServer::receiveData(coreutils::ZString &buffer) {
// See this override's own doc comment in TCPServer.h - the
// listening socket always uses the plain read()/ENOTCONN detection
// path, never TLS::receiveData(), regardless of whether this server
// is TLS-configured.
Socket::receiveData(buffer);
}
TCPSession * TCPServer::accept() {
try
{
TCPSession *session = getSocketAccept(ePoll);
session->setDescriptor(::accept(getDescriptor(), (struct sockaddr *)&session->ipAddress.addr, &session->ipAddress.addressLength));
// if(blackList && blackList->contains(session->ipAddress.getClientAddress())) {
// session->shutdown();
// Log(LOG_WARN) << "Client at IP address " << session->ipAddress.getClientAddress() << " is blacklisted and was denied a connection.";
// return NULL;
// }
// if(whiteList && !whiteList->contains(session->ipAddress.getClientAddress())) {
// session->shutdown();
// Log(LOG_WARN) << "Client at IP address " << session->ipAddress.getClientAddress() << " is not authorized and was denied a connection.";
// return NULL;
// }
return session;
}
catch (coreutils::Exception e) {
coreutils::Log(coreutils::LOG_EXCEPT) << "Major error on session initialization. Error is '" << e.text << "'.";
}
catch (...) {
coreutils::Log(coreutils::LOG_EXCEPT) << "Unnspecified error on session initialization.";
}
return NULL;
}
void TCPServer::removeFromSessionList(TCPSession *session) {
lock.lock();
for (auto cursor = sessions.begin(); cursor < sessions.end(); ++cursor)
if (cursor->get() == session)
{
// Drops the one strong-owning reference. If nothing else is
// currently pin()'d to this session, the object is actually
// destroyed right here, synchronously - same timing as
// before this change for the common (no-contention) case.
// If some other thread IS mid-call via its own pin()'d
// shared_ptr right now, destruction is safely deferred until
// that reference is released too, instead of the crash
// (double-delete / use-after-free, confirmed live via
// AddressSanitizer under concurrent load) that raw-pointer
// ownership produced here before.
sessions.erase(cursor);
break;
}
lock.unlock();
// Removes this pointer from the tryGet() lookup registry too (see
// TCPSession::tryGet()'s own comment). Safe to do after releasing
// `lock` and regardless of whether the erase() above was actually
// the last reference: tryGet() correctly reports "gone" the moment
// a session's real refcount hits zero, via the weak_ptr's own
// expiration - this call is just tidying the registry map itself,
// not something later correctness depends on.
TCPSession::unregisterLive(session);
}
void TCPServer::sessionErrorHandler(std::string errorString, std::stringstream &out) {
throw coreutils::Exception(errorString);
}
TCPSession * TCPServer::getSocketAccept(EPoll &ePoll) {
return new TCPSession(ePoll, this);
}
void TCPServer::output(std::stringstream &out)
{
out << "Use the 'help' command to list the commands for this server." << std::endl;
}
int TCPServer::processCommand(coreutils::ZString &request, TCPSession &session)
{
// Was unguarded, like every other method below - `sessions` is
// mutated under `lock` by connectionRequest()/removeFromSessionList()
// (a session accepting or disconnecting on another thread), but
// every READ of it here had no lock at all. Confirmed live via
// dmesg: a real segfault at a stable, deterministic instruction
// offset (same crash signature as the Subscription::subscribers
// race fixed earlier this session, in SubscriptionManager.cpp) -
// Socket::write() -> resetSocket() dereferencing a null EPoll
// reference, consistent with iterating `sessions` while another
// thread concurrently erased and freed one of its entries.
// sendToAll() in particular is what most real gameplay broadcasts
// (combat, position, chat) go through, so this is very likely the
// actual dominant cause of that crash, not just the ItemDataBase
// subscriber list this session's earlier fix addressed.
std::lock_guard<std::mutex> guard(lock);
int sequence = 0;
for (auto &sessionx : sessions)
{
session.out << "|" << ++sequence;
sessionx->output(session.out);
session.out << "|" << std::endl;
}
return 1;
}
void TCPServer::sendToAll(std::stringstream &data) {
std::lock_guard<std::mutex> guard(lock);
for (auto &session : sessions)
session->write(data.str());
data.str("");
}
void TCPServer::sendToAll(std::stringstream &data, TCPSession &sender) {
std::lock_guard<std::mutex> guard(lock);
for (auto &session : sessions)
if (session.get() != &sender)
session->write(data.str());
data.str("");
}
void TCPServer::sendToAll(std::stringstream &data, TCPSession &sender, SessionFilter filter) {
std::lock_guard<std::mutex> guard(lock);
for (auto &session : sessions)
if (filter.test(*session))
if (session.get() != &sender)
session->write(data.str());
data.str("");
}
std::shared_ptr<TCPSession> TCPServer::getSessionByAlias(void *alias) {
coreutils::Log(coreutils::LOG_DEBUG_1) << alias;
std::lock_guard<std::mutex> guard(lock);
for (auto &session : sessions)
if (session->alias == alias)
return session;
return nullptr;
}
}