Normally, the Halon platform operates in a traditional store-and-forward MTA fashion. However, in some cases, such as providing outbound filtering for a network of computers, it is desired to run the software in a more proxy-like setup.
In-line delivery
Although the Halon system operates more efficiently in queued mode, because the system can schedule more freely, sometimes there is a need for queue-less in-line delivery, using Deliver(["queue" => false]); (EOD per recipient) and $arguments["mail"]->send() (EOD per message). It is also possible to combine queued delivery in the Halon system and in-line delivery for a client, using an external SMTP proxy that reacts to callbacks from a post-delivery script such as the following.
http("https://smtp-proxy/callback", [], [], ["code" => $errorcode, "msg" => $errormsg, "id" => $messageid]);
Delete(); // no more retries
While developing such a proxy might seem like a daunting task, customers have done so in reasonably short time, using high-level frameworks, such as Twisted.
Semi transparent operation
One aspect of proxying is being more transparent. Please keep in mind that many of the recommended methods for dealing with spam and blacklisting are hampered as moving towards a more transparent mode of operation.
Original source
When delivering an email out from the gateway, the source address can be changed using Try([ "sourceip" => [] ]); (pre-delivery). By enabling the nonlocal_source system setting, any address (such as the connecting/sending client's IP) can be used, as follows.
if ($senderip)
SetSourceIP($senderip, ["nonlocal_source" => true]);
if ($senderhelo)
SetHELO($senderhelo);
Please keep in mind that any router and firewall must be configured in such a way, that the returning traffic is routed back to the Halon system.
Original destination
We argue that "forcing" an email to be sent to the MX of the $recipientdomain (RCPT TO), by simply relying on the normal behaviour of a "lookup-mx" transport, is a powerful security/anti-spam feature. However, in some cases, it might be necessary to send an email to the original destination IP address, to which the sending client connected. This requires a load balancer in front of the Halon system, which can append the destination IP address as a header to such an email. If there is no load balancer, a simple SMTP proxy can be set up on, for example, Linux, using the SO_ORIGINAL_DST socket option. The header can be read from the DATA script, as follows.
$metadata = ["orig_dest" => $arguments["mail"]->getHeader("x-orig-dest")];
// $metadata needs to be added to $arguments["mail"]->queue() and History()
And the header can be used in the pre-delivery script, when sending the message, as follows.
$metadata = GetMetaData();
if ($metadata["orig_dest"])
Try(["host" => $metadata["orig_dest"]]);
In theory, even authentication information from the AUTH context can be forwarded to SetSASL() (pre-delivery), although that seems like a poor idea from a security perspective.
Fully transparent operation
If the semi transparent operation method is not suitable, an external proxy can be used to handle all communication between client and server; with the Halon system "on the side", inspecting the messages and coming up with verdicts.
ProxSMTP
The open source ProxSMTP by Stef Walter (currently at Red Hat) is ideal for a fully transparent setup, and is available as a package for most Linux and *BSD distributions.
Using Halon's modified version
For high performance systems, using our proxsmtp version is recommended, enabling direct communication from the proxsmtp to a Halon node, without invoking an external program. The file proxsmtpd.conf (for example, /usr/local/etc/proxsmtpd.conf) is configured like in the following.
FilterType: smtp
FilterCommand: IP-OF-HALON
...
Using our proxsmtp with a Halon cluster
Our proxsmtp does not have built-in support for load balancing traffic against a cluster of Halon nodes, but this can be easily accomplished by installing a separate load balancer, for example, the HAProxy (for example, /etc/haproxy/haproxy.cfg) and using a configuration similar to the following.
listen smtp 127.0.0.1:25
mode tcp
option tcplog
option smtpchk
balance roundrobin
server halon1 10.2.0.30:25 check
server halon2 10.2.0.31:25 check
And then using the listener for the load balancer as the FilterCommand in the proxsmtp.
Fully transparent outbound anti-spam
The Halon platform can be used as a fully transparent spam filter, with outbound traffic destined for port 25 being re-routed to the system by firewalls or routers.
Considerations
We recommend implementing outbound anti-spam as a store-and-forward MTA; that is, configure it as a relay/smarthost, or as a semi transparent proxy whenever possible, because transparent filtering comes with a few disadvantages. Most importantly, transparent filtering disables TLS, to be able to intercept the outbound traffic. Hence, if the sending server is configured to use, for example, DANE, email delivery for such messages will fail. From an anti-spam effectiveness perspective, this solution performs slightly worse than alternative methods, because it makes smart queue scheduling and other delivery techniques, such as dedicated bulk IP's and source hashing, impossible to use.
In other words, before deploying a transparent spam filter, consider the alternatives. If doing it because of SPF, consider using SRS to rewrite the sender address, or using semi transparent proxy mode, instead. However, we do realise that fully transparent filtering is needed in some cases; for example, when deployed as a drop-in replacement in front of servers that are managed by someone else, such as VPS/cloud hosting.
Installation
The typical setup consists of (at least) two proxy servers running our proxsmtp version and two Halon (spam filtering) nodes.
Proxy server
The proxy server can be any Linux distribution of your choice. Follow the instructions to install and configure our proxsmtp and an HAProxy.
Halon (spam filtering) nodes
- On the Configuration -> Server -> Listeners page, configure a listener on a desired port (as configured in proxsmtp/HAProxy).
- Click on the newly configured listener ID, and, under General in the Allow XCLIENT from IP(s) field, type the comma-separated IP addresses of each proxy server.
- On the Configuration -> </> Code editor page (also accessed from the context columns on the Configuration -> Server -> Script mappings page), use HSL to configure the desired contexts for the new listener.
The RCPT and EOD contexts can be used to configure a listener; but in the following example, we will only use the EOD context.
/* All imports are included in the default configuration */
import { get_rate } from "modules/rates.hsl";
import { is_whitelisted } from "modules/whitelist.hsl";
import { is_blacklisted } from "modules/blacklist.hsl";
$transactionid = $transaction["id"];
$sender = $transaction["senderaddress"];
$senderdomain = $sender["domain"];
$recipients = $transaction["recipients"];
$remoteip = $connection["remoteip"];
$mail = $arguments["mail"];
$metadata = [];
// Blacklist
$blacklisted = is_blacklisted($remoteip, $sender);
if ($blacklisted)
Reject("Blacklisted ($transactionid)");
if (!is_whitelisted($remoteip, $sender, "anti-spam")) {
// bulk
$outboundbulk = get_rate("outbound-bulk");
if ($outboundbulk["entry"] === "ip") $outboundbulk["entry"] = $remoteip;
if ($outboundbulk["entry"] === "email") $outboundbulk["entry"] = $transaction["sender"];
if ($outboundbulk["entry"] === "domain") $outboundbulk["entry"] = $senderdomain;
// spam
$outboundspam = get_rate("outbound-spam");
if ($outboundspam["entry"] === "ip") $outboundspam["entry"] = $remoteip;
if ($outboundspam["entry"] === "email") $outboundspam["entry"] = $transaction["sender"];
if ($outboundspam["entry"] === "domain") $outboundspam["entry"] = $senderdomain;
// Remote IP rates
$outboundbulkip = get_rate("outbound-bulk-ip");
$outboundbulkip["entry"] = $remoteip;
$outboundspamip = get_rate("outbound-spam-ip");
$outboundspamip["entry"] = $remoteip;
$isbulk = false;
// Defer high volumes of bulk and spam
if ($outboundbulk and $outboundbulk["entry"] !== "username") {
if (ScanRPD(["outbound" => true]) === 50 or (ScanRPD(["outbound" => true]) !== 100 and ScanSA() > 4)) {
// IP rate limit
if (rate("outbound-bulk-ip", $outboundbulkip["entry"], $outboundbulkip["count"], $outboundbulkip["interval"]) === false)
Defer($outboundbulkip["entry"]." is only allowed to send ".$outboundbulkip["count"]." bulk messages per 1 hour, try again later ($transactionid)");
// Domain rate limit
if (rate("outbound-bulk", $outboundbulk["entry"], $outboundbulk["count"], $outboundbulk["interval"]) === false)
Defer($outboundbulk["entry"]." is only allowed to send ".$outboundbulk["count"]." bulk messages per 1 hour, try again later ($transactionid)");
$isbulk = true;
}
}
if ($outboundspam and $outboundspam["entry"] !== "username" and !$isbulk) {
if (ScanRPD(["outbound" => true]) === 100 or ScanSA() > 6) {
// IP rate limit
if (rate("outbound-spam-ip", $outboundspamip["entry"], $outboundspamip["count"], $outboundspamip["interval"]) === false)
Defer($outboundspamip["entry"]." is only allowed to send ".$outboundspamip["count"]." spam messages per 1 hours, try again later ($transactionid)");
// Domain rate limit
if (rate("outbound-spam", $outboundspam["entry"], $outboundspam["count"], $outboundspam["interval"]) === false)
Defer($outboundspam["entry"]." is only allowed to send ".$outboundspam["count"]." spam messages per 8 hours, try again later ($transactionid)");
}
}
}
// Anti-virus checks
if (!is_whitelisted($remoteip, $sender, "anti-virus")) {
if (ScanRPD(["outbound" => true, "extended_result" => true])["virus_score"])
Reject("Rejected by virus filter ($transactionid)");
if (ScanKAV())
Reject("Rejected by virus filter ($transactionid)");
if (ScanCLAM())
Reject("Rejected by virus filter ($transactionid)");
}
Accept("250 2.0.0 Ok: queued as $transactionid");
// Function overrides
function Accept($reason) {
global $recipients, $sender, $metadata, $transactionid;
foreach ($recipients as $recipient) {
History("DELIVER", $recipient["address"], [
"sender" => $sender,
"metadata" => $metadata,
"transportid" => $recipient["transportid"],
"reason" => $reason
]);
}
builtin Accept();
}
function Defer($reason, ...$args) {
global $recipients, $sender, $metadata;
foreach ($recipients as $recipient) {
History("DEFER", $recipient["address"], [
"sender" => $sender,
"metadata" => $metadata,
"transportid" => $recipient["transportid"],
"reason" => $reason
]);
}
builtin Defer($reason, ...$args);
}
function Reject($reason, ...$args) {
global $recipients, $sender, $metadata;
foreach ($recipients as $recipient) {
History("REJECT", $recipient["address"], [
"sender" => $sender,
"metadata" => $metadata,
"transportid" => $recipient["transportid"],
"reason" => $reason
]);
}
builtin Reject($reason, ...$args);
}
Logging server
For easier administration and supervision, we recommend using the logging server.
Router configuration
For a fully transparent operation setup to work, any firewall and router, through which the outbound traffic is routed, must re-route outbound packets (and returning traffic) to the proxy servers, using a port 25 destination. The valid configuration for a specific router brand varies; the following examples apply for Juniper Networks:
Comments
0 comments
Article is closed for comments.