Fix premature socket re-arm race between write() and eventReceived()

Socket::write() unconditionally called resetSocket() (via EPOLL_CTL_MOD)
from whatever thread called it, whenever it could grab `lock` - including
while this same socket's OWN eventReceived() call was still mid-flight on
another thread (past receiveData()'s lock_guard, but before its own
trailing resetSocket()/return). Re-arming makes the fd immediately
eligible for a third thread to pick up a fresh event and, if it decides
to shut the connection down, delete the object outright (Thread::run()
does exactly that on a false return) - while the original thread is still
touching it. A real heap-corruption crash inside libc's malloc internals
was confirmed live via dmesg under concurrent load, consistent with
exactly this: a socket freed while another thread still held a reference
to it.

inHandler existed as a flag already but only fed a diagnostic log line -
never actually gated anything. Made it a real std::atomic<bool> (was
`volatile`, which gives no actual atomicity or cross-thread visibility)
and used it two ways:
- write() now skips its own resetSocket() call entirely while inHandler
  is true - that call's own eventReceived() will re-arm once it's
  actually done, seeing the fifo entry write() just queued.
- eventReceived() itself was clearing inHandler well before its own
  resetSocket() call (several statements up, right after EPOLLIN/
  EPOLLWRNORM handling) - moved the clear to immediately precede that
  call, closing the exposed window down to the minimum achievable
  without restructuring how write() re-arms sockets altogether.

This is a distinct bug from the two subscriber/session-list races fixed
earlier tonight (SubscriptionManager.cpp, TCPServer.cpp) - found while
stress-testing those fixes turned up a third, different crash signature
(heap corruption, not the null-EPoll-reference pattern the earlier two
produced).
This commit is contained in:
Matt Arant committed 2026-09-07 23:52:42 +00:00
1 parent 0bb926a264
commit 8009564a7a
2 files changed
+38 -3

No files matched your search

+6 -1
View File
@@ -4,6 +4,7 @@
#include "ZString.h"
#include <queue>
#include <mutex>
#include <atomic>
#include <sys/epoll.h>
namespace core {
@@ -199,7 +200,11 @@ namespace core {
int descriptor = -1;
std::mutex outlock;
bool readHangup = false;
volatile bool inHandler = false;
// Was `volatile bool` - volatile gives no actual thread-safety
// (no atomicity, no cross-thread visibility guarantee), and this
// flag is now load-bearing (see Socket::write()'s own check of it)
// rather than just feeding a diagnostic log line.
std::atomic<bool> inHandler{false};
//-------------------------------------------------------------------------------------
// the writeSocket is called when epoll has received a write request for a socket.