Detecting hackers/spammers sending mail on your system

# Detecting Hackers or Spammers sending email through your system.

- Finding the source/cause of a spam outbreak (identifying the hacked account or setting issue)
- Reading msg*.rec log entries...
- Settings to help auto detect spammers
- Settings to stop forgery/spoofing of your domain
- General spam settings page...

## Finding the source/cause of a spam outbreak

Use the command
tellmail send_top
to find any account sending a lot of email note down the top 1-2 accounts, next open the web admin to the msg.rec log file page and search for the problem accounts to confirm if what they are sending is spam or not, if it's spam change the passwords.
Next use:
tellmail showq or in web admin use Status & Reporting/Queue stats or Mail queue to examine some messages
to find if there is a bunch of email stuck in the outgoing queue, spam messages tend to get stuck so this may help you identify some spam, then search the msg.rec logs again to find the Received log entry to find why the spam message was recieved.
In some instances, the outgoing message is a bounce or other internally generated message, in this case you need to find the log entries just 'before' the message in question to find what caused it, so search on the time hh:mm: to find the log entries that are related.

## Reading msg*.rec log entries.

Log entries for an incoming message look like this:

     9 21:35:30[130766127] Rcpt 208.117.50.44 <return@address.com> <myaccount@fake.co.nz> 0  ""
     9 21:35:31[130766127] Received 208.117.50.44 return@address.com <myaccount@fake.co.nz> 120395 <XJkzvvWDQXeUsz7INbIxGQ@ismtpd0003p1lon1.sendgrid.net> "Relay=islocal, nrcpt=1, f="Booking.com" <email.campaign@sg.booking.com>, s=[Chris - Whakapapa Village and Adelaide! There's a deal with your name on it!]"
     9 21:35:32[130766127] Aspam 208.117.50.44 return@address.com <myaccount@fake.co.nz> 120395 <XJkzvvWDQXeUsz7INbIxGQ@ismtpd0003p1lon1.sendgrid.net> "notrust *****: 5.8 sd=5.8 l=0.00 nok=2/0 m=2 nf=0 Close 0.05(X-SpamContent:clean) 0.95(X-myrbl:Color=brown) 0.91(isclickimage2) 0.82(isclickimage1) 0.20(X-Phrase:clean) 0.37(dkimok) 0.37(genuine) 0.60(spfpass) 0.46(X-NotAscii:utf) 0.53(X-LangGuess:English) 0.49(X-Verify-Helo:+OK) SanScore 0.0 5.8 Sval 5.8"
     9 21:35:32.00 [130766127] Spam 208.117.50.44 <return@address.com> <myaccount@fake.co.nz> 120395 <XJkzvvWDQXeUsz7INbIxGQ@ismtpd0003p1lon1.sendgrid.net> "[o4.sg.booking.com] SpamDetect"
     9 21:35:32.00 [130766127] Stored 208.117.50.44 <return@address.com> <myaccount@fake.co.nz> 120395 <XJkzvvWDQXeUsz7INbIxGQ@ismtpd0003p1lon1.sendgrid.net> "[o4.sg.booking.com] Stored locally /home/surgemail/netwin.co.nz/hc/gf/chrisp/mdir/new/1470796532.2114_15646.netwin.netwinsite.co"
    Cpu time used      0 cpu seconds

For an outgoing message it looks like this:

     9 21:39:48[130766226] Rcpt 115.188.8.177 <myaccount@fake.co.nz> <user@destination.com> 0  ""
     9 21:39:49[130766226] Received 115.188.8.177 myaccount@fake.co.nz <user@destination.com> 566 <6a197213-738a-76f8-44d7-3e98ddb34224@netwin.co.nz> "Relay=smtpauth=myaccount@fake.co.nz, nrcpt=1, s=[test]"
     9 21:39:49[130766226] NOSPAM 115.188.8.177 myaccount@fake.co.nz <user@destination.com> 566 <6a197213-738a-76f8-44d7-3e98ddb34224@netwin.co.nz> "trusted origin so skipping spam filtering g_smite_skip_relay"
     9 21:39:53.00 [130766226] Sent 115.188.8.177 <myaccount@fake.co.nz> <user@destination.com> 566 <6a197213-738a-76f8-44d7-3e98ddb34224@netwin.co.nz> "[115-18-8-177.jetstream.xtra.co.nz] Delivered to remote host 12.9.234.50 used SSL - 250 message sent ok "

In both cases the line you want when hunting a hacked account is the 'Received' entry, this shows you why the message was accepted by your server.

- smtpauth=user@xyz.com (The user gave valid smtp auth credentials, which are listed.)
- g_relay_allow_ip (The sender is from a trusted ip address you have listed)
- islocal (The message is to a local account/domain, or was being forwarded by one)

To find this look for the Relay=... text in the line, e.g. in this example the message is relayed because the sender authenticated correctly as myaccount@fake.co.nz

     9 21:39:49[130766226] Received 115.188.8.177 myaccount@fake.co.nz <user@destination.com> 566
       <6a197213-738a-76f8-44d7-3e98ddb34224@netwin.co.nz> "Relay=smtpauth=myaccount@fake.co.nz, nrcpt=1, s=[test]"

The number in square brackets lets you find all related log entries, e.g. search on 130766226 in the above example
In some cases you will need to find events just before a log entry, to find what caused the message to be sent, to do this search on the date and time, e.g. "9 21:39:4" in the above example...

## Settings to help auto detect spammers.

One or more of your users will have their account hacked and abused to send spam sooner or later, you can make it much harder, and you can detect it and stop it early using these settings and policies.
# Login guesses per IP before it is automatically and permenently locked out. Use tellmail unlock ip.address to fix...
G_HACKER_MAX "10"
# If hacker attempts to login to one of these then the ip is instantly locked out. (Don't use accounts that exist)
G_HACKER_POISON "root@*,administrator@*"
# Only allow smtp logins if the user has previously logged in via imap/pop from the same address
G_SAFE_SMTP "true"
# Alert users when logins occur from unknown addresses that are not from australia or usa...
G_SAFE_WARNING "true"
g_safe_country "us,au"
# Max messages an authenticated user can send per 30 minutes, e.g. 5000
G_SPAM_USER_MAX "2000"
# Max outgoing messages per ipaddress/return path pair, 30 minutes, e.g. 5000
G_SPAM_FROM_MAX "2000"
# Detect local users sending 'spam like' email and send a report to the manager.
G_OUTGOING_N "5"
# White list for people you know send mail that looks a bit dodgy.
G_OUTGOING_WHITE "bob@here.com,1.2.3.4"
# send manager an email if a local user sends more than 300 message in a day...
G_USER_SEND_WARNING "300"
g_user_send_ip "300"
# Apply some more strict password checking, and alert users with simple passwords...
G_CREATE_PASS_DIGIT "true"
G_CREATE_PASS_RECHECK "true"
G_HACK_TOUSER "true"

Lastly you should run this command to find any accounts with trivially crackable passwords.

tellmail test_weak

We've built a faster more extensive password checker into nwauth, if you have the latest version of it you can do this:
nwauth -version
+OK nwauth 4.3e capa=cluster2
nwauth -testweak
+DATA Info: testing for 999 common passwords, pass 1
+DATA Cracked: crack2@xyz.com
+DATA Info: testing for variations on common passwords, pass 2
...

## Settings to stop spoofing of your domains.

The problem with spoofing is that traditional email allowed it and real legitimate systems still make use of that. Also your own users may send email from their own accounts via other systems (gmail etc)... so it's impossible to stop all forgeries unless your users are prepared to 'play the game' and only send email from your servers (or at least a list of known systems), so that's why there is such a lot of settings to do this, each one addresses a subset of forgery behaviour.

g_spf_nofriend "true"

# Maybe this next one, but some real emailers will get blocked as they still don't obey spf properly.
g_friends_obey_spf "true"

g_spf_enforce_local "true"

Check your have spf entries for your domains. https://netwinsite.com/spf.htm#define

# Optional settings depending on your tolerance for bouncing real messages.

g_from_exact "true" # Bounce if the from doesn't match the authenticated sender, stops your users pretending to be each other.

g_from_noforge "true"

g_from_noforgeme "true" # checks for the special case of 'from=to' and blocks it if it originates from outside your system.

g_from_check "true" # enables some checking of the from address.

g_from_nofriend "true" # prevent friend matches for apparently forged return addresses. So they don't bypass spam handling.

# And one of these

g_from_bounce "true" # Bounce messages that are probably forgeries.

g_from_stamp "true" # stamp forgeries in the spam headers but don't actually stop them.

### An extreme setting you may use:

SECURITY_SUFFIX - Make logins fail unless the user knows the suffix
One other method to protect your server is to make the login username different from the email address. You can do this on a per domain level, lets say you have a domain MYDOMAIN.COM and you want the users to login with username=JOHN@SECRET.MYDOMAIN.COM
g_from_rewrite was="*@secret.mydomain.com" to="%1@mydomain.com"
g_from_rewrite_header "true"
g_from_rewrite_sender "true"
vdomain name="mydomain.com"
security_suffix "secret.mydomain.com"
...

note: I don't recommend using the security suffix on most systems, it is the most complex of the settings and the most likely to cause disruption, it does have benefits but I would only use it for small sites where extreme security measures are justified.