SMTP: STARTTLS, smuggling fix, and RFC 5321 conformance

Security:
- SMTP smuggling (CVE-2023-51764 family): only <CRLF>.<CRLF> ends DATA
  (strict CRLF framing); bare CR/LF in the text is stored as CR LF.
  Before, '<LF>.<LF>' and four other variants ended the message and the
  rest ran as fresh commands - a second, forged message was delivered.
- STARTTLS on port 25 (RFC 3207), using the configured tls cert; AUTH
  is only offered/accepted once encrypted (538 5.7.11 otherwise).
- The 250 reply no longer discloses the queue file's full path.
- Queue file names %-encode the addresses ('/' or '|' in an address
  broke the name or pointed it into another directory).

Conformance (RFC 5321, 1870, 2034, 2920, 3207, 3463, 3848, 4616, 4954,
6152):
- 500 for unknown/empty/over-long commands (no reply at all before -
  clients hung); 501 for missing/invalid arguments (bare MAIL/RCPT/HELO
  were out-of-bounds reads).
- HELO/EHLO required before MAIL; nested MAIL refused; EHLO/HELO/RSET/
  end-of-DATA reset the transaction.
- MAIL: SIZE= (552 when over the limit), BODY=7BIT|8BITMIME, AUTH=;
  555 for anything else; case-insensitive keywords; address syntax
  checked.
- RCPT: postmaster (bare and any case) accepted; case-insensitive
  domain and local-part; 550 5.1.1 unknown user vs 550 5.7.1 relay
  denied; aliases accepted (they were rejected at RCPT before); 1000
  recipient cap (452 4.5.3).
- ENHANCEDSTATUSCODES advertised and used on every reply.
- AUTH PLAIN added alongside LOGIN; strict base64; 535 5.7.8 on
  failure; 503 before EHLO, during a transaction, or when already
  authenticated (the last one sent no reply before).
- Received: per RFC 5321 4.4 with client IP, protocol per RFC 3848
  (ESMTP/ESMTPS/ESMTPSA) and queue id; Return-Path: added at local
  delivery.

tests/smtp_conformance.py: 112 checks. origin/master passes 37 of the
72 it can reach before the server stops answering; this passes 112/112.
tests/imap_pop3_regression.py: IMAPS/POP3S unaffected (3/3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0121tAhQVe4qbCwYgVjKQS4B
This commit is contained in:
barantandClaude Opus 5.5 authored and brad Arant committed 2026-11-02 11:56:41 -08:00
1 parent 407cc2e915
commit d3d7e5f3e5
27 files changed
+1821 -358

No files matched your search

+130 -136
View File
@@ -2,6 +2,7 @@
#include "SMTPServer.h"
#include "DKIMVerifier.h"
#include "DMARCChecker.h"
#include "SMTPSyntax.h"
#include "Log.h"
#include <cctype>
#include <chrono>
@@ -69,12 +70,13 @@ namespace mail {
}
// RFC 5321 4.4: every SMTP server MUST prepend a Received trace
// header to a message it processes - this codebase never wrote one
// at all before. Deliberately no "for <recipient>" clause: one
// already-written queue file gets hardlinked to every recipient (see
// queueMail() below), and RFC 5321 4.4 itself says that clause
// SHOULD be omitted once there's more than one anyway.
// RFC 5321 4.4: every SMTP server MUST prepend a Received: trace
// header to each message it accepts. Grammar (4.4):
// "Received:" "from" Extended-Domain "by" Domain Opt-info ";" date-time
// with the client's address literal as TCP-info, the protocol per
// RFC 3848 (SMTP / ESMTP, +S for TLS, +A for authenticated), and this
// message's queue id. No "for <recipient>" clause: one queued copy is
// linked to every recipient, and 4.4 says to omit it then anyway.
coreutils::MString __SMTP_DATA::receivedHeader(SMTPSession &session, SMTPServer &server) {
time_t now = time(nullptr);
struct tm tmResult;
@@ -82,131 +84,126 @@ namespace mail {
char dateBuf[64];
strftime(dateBuf, sizeof(dateBuf), "%a, %d %b %Y %H:%M:%S %z", &tmResult);
std::string protocol = session.esmtp ? "ESMTP" : "SMTP";
if (session.esmtp && session.isTLS()) protocol += "S";
if (session.esmtp && session.authState == USER_KNOWN) protocol += "A";
coreutils::MString header;
header << "Received: from " << session.clientDomainName << CRLF;
header << "\tby " << server.hostName.c_str() << " with SMTP;" << CRLF;
header << "Received: from " << session.clientDomainName
<< " ([" << session.ipAddress.getClientAddress().c_str() << "])" << CRLF;
header << "\tby " << server.hostName.c_str() << " (BARANTMail) with " << protocol.c_str()
<< " id " << session.queueId.c_str() << ";" << CRLF;
header << "\t" << dateBuf << CRLF;
return header;
}
int __SMTP_DATA::processCommand(coreutils::ZString &request, SMTPSession &session, SMTPServer &server) {
switch(session.mode) {
if (session.mode == WAIT_FOR_DATA) {
// RFC 5321 4.1.1.4: "DATA" CRLF - no parameters.
if (!SMTPSyntax::argument(line(request)).empty()) {
session.out << "501 5.5.4 Syntax: DATA (no parameters)" << CRLF;
return 1;
}
if (session.state == MAIL) {
session.out << "503 5.5.1 Need RCPT command" << CRLF;
return 1;
}
if (session.state != RCPT) {
session.out << "503 5.5.1 Need MAIL command" << CRLF;
return 1;
}
session.queueId = generateMailFileName().str();
session.out << "354 End data with <CR><LF>.<CR><LF>" << CRLF;
// The trace header seeds the message; SPF's Received-SPF:
// (computed at MAIL FROM) goes directly after it, where MTAs
// conventionally put it.
session.mailData = receivedHeader(session, server);
session.mailData << session.spfHeader.c_str();
session.dataLineTooLong = false;
session.mode = RECEIVE_DATA;
grabInput(session);
return 1;
}
case WAIT_FOR_DATA:
// ---- RECEIVE_DATA: one line of message text per call. Lines are
// framed on CR LF only (SMTPSession sets strictCRLF), so "." here
// can only be the real <CRLF>.<CRLF> terminator - a bare-LF or
// bare-CR variant inside the text is never mistaken for it (SMTP
// smuggling, CVE-2023-51764 family).
if (request != ".") {
if (session.mailData.getLength() >= SMTPServer::maxMessageSize)
return 1; // over the limit: keep reading, reject at the end
// RFC 5321 4.5.2 transparency: remove one leading '.'.
std::string text = line(request);
if (!text.empty() && text[0] == '.')
text.erase(0, 1);
// A bare CR or LF can't end a line here; store it as a proper
// CR LF (RFC 5321 2.3.8 forbids them in the text), so nothing
// downstream - local mailbox readers, or the next MTA when this
// message is relayed - ever sees a bare line ending from us.
if (text.find_first_of("\r\n") != std::string::npos) {
std::string normalized;
for (char c : text) {
if (c == '\r' || c == '\n') normalized += "\r\n";
else normalized += c;
}
text.swap(normalized);
}
text += "\r\n";
session.mailData << text.c_str();
return 1;
}
switch(session.state) {
// ---- end of data
clearGrab(session);
if (session.dataLineTooLong) {
session.out << "500 5.5.6 Message contains a line that is too long" << CRLF;
} else if (session.mailData.getLength() >= SMTPServer::maxMessageSize) {
session.out << "552 5.3.4 Message size exceeds fixed maximum message size" << CRLF;
} else {
std::string message = session.mailData.str();
case RCPT:
session.out << "354 Enter the mail message terminated by <CRLF>.<CRLF>" << CRLF;
// RFC 5321 4.4 requires a Received trace header on every message
// this server processes - previously absent entirely. Written
// once here as the seed of mailData so the body lines that
// follow just accumulate after it; omits a "for <recipient>"
// clause since one already-written file gets hardlinked to every
// recipient (see queueMail() below) and RFC 5321 4.4 itself says
// that clause SHOULD be left out once there's more than one.
// session.spfHeader was computed once already, back at MAIL
// FROM time (see __SMTP_MAIL.cpp) - reused here rather than
// re-running the SPF DNS lookups a second time for the same
// transaction. Placed directly after Received:, matching
// where real-world MTAs conventionally put it.
session.mailData = receivedHeader(session, server);
session.mailData << session.spfHeader.c_str();
session.mode = RECEIVE_DATA;
grabInput(session);
// session.setTimer(120.0f);
break;
// DKIM verification + DMARC alignment, now that the full
// message (including whatever From:/DKIM-Signature headers
// the client sent) is actually available - neither can run
// any earlier than this.
std::vector<DKIMVerifier::SignatureResult> dkimResults = DKIMVerifier::verify(message);
std::string fromDomain = extractFromDomain(message);
DMARCChecker::Policy dmarcPolicy;
DMARCChecker::Result dmarcResult = DMARCChecker::evaluate(
fromDomain, session.spfDomain, session.spfResult, dkimResults, dmarcPolicy);
std::string authResults = DMARCChecker::authenticationResultsHeader(
session.spfResult, session.spfDomain, dkimResults, dmarcResult, fromDomain, server.hostName);
case MAIL:
session.out << "503 Please use RCPT first" << CRLF;
break;
// RFC 7489 6.3: pct= applies the published disposition to only
// that percentage of otherwise-failing messages - the
// remainder still gets the Authentication-Results header
// recording the real (failing) result, just isn't rejected
// for it. p=none/quarantine never reject outright either way
// (quarantine has no separate mailbox to route into here, so
// it degrades to "accept, but the header says fail" - still a
// meaningful signal for whatever reads the header downstream).
bool sampled = (dmarcPolicy.percent >= 100) || ((rand() % 100) < dmarcPolicy.percent);
bool reject = (dmarcResult == DMARCChecker::Result::Fail) && dmarcPolicy.policy == "reject" && sampled;
default:
session.out << "503 Please use MAIL first" << CRLF;
break;
}
break;
case RECEIVE_DATA:
if(request != ".") {
// RFC 5321 4.5.2 transparency: a line starting with '.' that
// isn't the bare terminator has been stuffed with an extra
// leading '.' by a compliant sender - strip exactly one before
// storing, or every such line comes out with a spurious
// leading dot baked permanently into the stored message body.
coreutils::ZString line = request.startsWith(".")
? coreutils::ZString(request.getData() + 1, request.getLength() - 1)
: request;
// Cap accumulation at the advertised SIZE limit rather than
// growing mailData without bound - still consuming every line
// the client sends either way (RFC 5321 doesn't support a
// synchronous mid-DATA reply; the client isn't reading
// anything back until its own terminating "." goes out), so
// this only bounds our own memory use. The client actually
// finds out via the 552 reply below, once that terminator
// arrives.
if (session.mailData.getLength() < SMTPServer::maxMessageSize)
session.mailData << line << CRLF;
}
else {
session.mode = WAIT_FOR_DATA;
session.state = READY;
clearGrab(session);
if (session.mailData.getLength() >= SMTPServer::maxMessageSize) {
session.out << "552 5.3.4 Message size exceeds fixed maximum message size" << CRLF;
} else if(session.recipientList.size() > 0) {
if(session.mailData.getLength() > 0) {
std::string message = session.mailData.str();
// DKIM verification + DMARC alignment, now that the full
// message (including whatever From:/DKIM-Signature headers
// the client sent) is actually available - neither can run
// any earlier than this.
std::vector<DKIMVerifier::SignatureResult> dkimResults = DKIMVerifier::verify(message);
std::string fromDomain = extractFromDomain(message);
DMARCChecker::Policy dmarcPolicy;
DMARCChecker::Result dmarcResult = DMARCChecker::evaluate(
fromDomain, session.spfDomain, session.spfResult, dkimResults, dmarcPolicy);
std::string authResults = DMARCChecker::authenticationResultsHeader(
session.spfResult, session.spfDomain, dkimResults, dmarcResult, fromDomain, server.hostName);
// RFC 7489 6.3: pct= applies the published disposition to only
// that percentage of otherwise-failing messages - the
// remainder still gets the Authentication-Results header
// recording the real (failing) result, just isn't rejected
// for it. p=none/quarantine never reject outright either way
// (quarantine has no separate mailbox to route into here, so
// it degrades to "accept, but the header says fail" - still a
// meaningful signal for whatever reads the header downstream).
bool sampled = (dmarcPolicy.percent >= 100) || ((rand() % 100) < dmarcPolicy.percent);
bool reject = (dmarcResult == DMARCChecker::Result::Fail) && dmarcPolicy.policy == "reject" && sampled;
if (reject) {
coreutils::Log(coreutils::LOG_INFO) << "Rejected message from " << session.from
<< " - DMARC fail, p=reject for " << fromDomain.c_str() << ".";
session.out << "550 5.7.1 Message rejected: failed DMARC policy for " << fromDomain.c_str() << CRLF;
} else {
coreutils::MString withAuthResults(authResults);
withAuthResults << session.mailData;
session.mailData = withAuthResults;
coreutils::MString ID = queueMail(server, session.from, session.recipientList, session.mailData);
if(ID != "") {
coreutils::Log(coreutils::LOG_INFO) << "Queued message " << ID << ".";
session.out << "250 OK Queued message " << ID << CRLF;
}
else
session.out << "550 Mail message too big" << CRLF;
}
} else
session.out << "550 Mail message was empty" << CRLF;
} else
session.out << "250 OK Queued message " << CRLF;
}
break;
}
if (reject) {
coreutils::Log(coreutils::LOG_INFO) << "Rejected message from " << session.from
<< " - DMARC fail, p=reject for " << fromDomain.c_str() << ".";
session.out << "550 5.7.1 Message rejected: failed DMARC policy for " << fromDomain.c_str() << CRLF;
} else {
coreutils::MString withAuthResults(authResults);
withAuthResults << session.mailData;
session.mailData = withAuthResults;
queueMail(server, session.queueId, session.from, session.recipientList, session.mailData);
coreutils::Log(coreutils::LOG_INFO) << "Queued message " << session.queueId.c_str() << ".";
// Just the id - the reply used to include the queue file's
// full path on this server.
session.out << "250 2.0.0 OK: queued as " << session.queueId.c_str() << CRLF;
}
}
// RFC 5321 4.1.1.4: the transaction is over whatever the outcome.
session.resetTransaction();
return 1;
}
@@ -214,34 +211,31 @@ namespace mail {
struct timespec timex;
clock_gettime(CLOCK_REALTIME, &timex);
std::stringstream temp;
temp << timex.tv_sec << "." << std::setfill('0') << std::setw(9) << timex.tv_nsec;
temp << timex.tv_sec << "." << std::setfill('0') << std::setw(9) << timex.tv_nsec;
return coreutils::MString(temp.str());
}
coreutils::MString __SMTP_DATA::queueMail(SMTPServer &server,
coreutils::MString &sender,
std::vector<coreutils::MString> &recipientList,
coreutils::MString &mailData) {
void __SMTP_DATA::queueMail(SMTPServer &server, const std::string &id,
coreutils::MString &sender,
std::vector<coreutils::MString> &recipientList,
coreutils::MString &mailData) {
coreutils::MString fileName;
fileName << server.mailFileSystem.getMailPath() << "/.queue/" << generateMailFileName();
fileName << server.mailFileSystem.getMailPath() << "/.queue/" << id.c_str();
coreutils::File mailFile(fileName, O_CREAT | O_WRONLY, 0660);
mailFile.write(mailData);
for(coreutils::MString recipient: recipientList) {
coreutils::MString newName;
// Also encode the envelope sender, not just the recipient -
// SMTPServer::inCreate() needs the real MAIL FROM address to
// hand outbound (non-local) mail to SendMail, and the filename
// is the only thing that survives from this queuing step to
// that point.
newName << fileName << "|" << recipient << "|" << sender;
link(fileName.c_str(), newName.c_str());
// <id>|<recipient>|<sender>: SMTPServer::inCreate() needs both
// addresses and the file name is the only thing that carries
// them there. Both are %-encoded - "/" and "|" are legal in an
// address but would break the name (see SMTPSyntax).
coreutils::MString newName;
newName << fileName << "|" << SMTPSyntax::encodeQueueField(recipient.str()).c_str()
<< "|" << SMTPSyntax::encodeQueueField(sender.str()).c_str();
link(fileName.c_str(), newName.c_str());
}
unlink(fileName.c_str());
return fileName;
}
}