Fix premature socket re-arm race between write() and eventReceived()
Socket::write() unconditionally called resetSocket() (via EPOLL_CTL_MOD) from whatever thread called it, whenever it could grab `lock` - including while this same socket's OWN eventReceived() call was still mid-flight on another thread (past receiveData()'s lock_guard, but before its own trailing resetSocket()/return). Re-arming makes the fd immediately eligible for a third thread to pick up a fresh event and, if it decides to shut the connection down, delete the object outright (Thread::run() does exactly that on a false return) - while the original thread is still touching it. A real heap-corruption crash inside libc's malloc internals was confirmed live via dmesg under concurrent load, consistent with exactly this: a socket freed while another thread still held a reference to it. inHandler existed as a flag already but only fed a diagnostic log line - never actually gated anything. Made it a real std::atomic<bool> (was `volatile`, which gives no actual atomicity or cross-thread visibility) and used it two ways: - write() now skips its own resetSocket() call entirely while inHandler is true - that call's own eventReceived() will re-arm once it's actually done, seeing the fifo entry write() just queued. - eventReceived() itself was clearing inHandler well before its own resetSocket() call (several statements up, right after EPOLLIN/ EPOLLWRNORM handling) - moved the clear to immediately precede that call, closing the exposed window down to the minimum achievable without restructuring how write() re-arms sockets altogether. This is a distinct bug from the two subscriber/session-list races fixed earlier tonight (SubscriptionManager.cpp, TCPServer.cpp) - found while stress-testing those fixes turned up a third, different crash signature (heap corruption, not the null-EPoll-reference pattern the earlier two produced).
This commit is contained in:
1 parent
0bb926a264
commit
8009564a7a
2 files changed
+38
-3
No files matched your search
@@ -4,6 +4,7 @@
|
||||
#include "ZString.h"
|
||||
#include <queue>
|
||||
#include <mutex>
|
||||
#include <atomic>
|
||||
#include <sys/epoll.h>
|
||||
|
||||
namespace core {
|
||||
@@ -199,7 +200,11 @@ namespace core {
|
||||
int descriptor = -1;
|
||||
std::mutex outlock;
|
||||
bool readHangup = false;
|
||||
volatile bool inHandler = false;
|
||||
// Was `volatile bool` - volatile gives no actual thread-safety
|
||||
// (no atomicity, no cross-thread visibility guarantee), and this
|
||||
// flag is now load-bearing (see Socket::write()'s own check of it)
|
||||
// rather than just feeding a diagnostic log line.
|
||||
std::atomic<bool> inHandler{false};
|
||||
|
||||
//-------------------------------------------------------------------------------------
|
||||
// the writeSocket is called when epoll has received a write request for a socket.
|
||||
|
||||
Reference in new issue
Block a user