This is the second part of an article on configuring a mail server. We will discuss testing your mail server’s settings and configuring its security using AppArmor, iptables, and Fail2ban. These instructions are applicable to Debian and Ubuntu.

In this article:

Testing mail server settings
Configuring Firewall iptables
Configuring Fail2ban
RoundCube Security
Creating AppArmor profiles

The process of installing and configuring a mail server is described in the first part of the article "How to set up a mail server in Debian and Ubuntu with ispmanager to protect against spam".

Testing mail server settings

You can check your mail server settings with the free MxToolbox service to analyze DNS records, identify mail servers, and troubleshoot mail delivery problems.

Here's how to use MxToolbox:

Go to the MxToolbox website.
Enter the domain name of your mail server (e.g. unixweb.info) or the e-mail and click Check Email Health. Change the domain name or email to your own.

The service will present information about the state of the DNS records, whether there are external blacklists, and PTR ,SPF, DKIM settings.

Customizing Firewall iptables

A firewall helps control incoming and outgoing connections by blocking unwanted traffic and preventing possible attacks.

The default iptables rules after installing ispmanager:

cat /etc/ispiptable.conf
# Generated by iptables-save v1.8.7 on Sat May 18 23:58:32 2024
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:ispmgr_allow_ip - [0:0]
:ispmgr_allow_sub - [0:0]
:ispmgr_deny_ip - [0:0]
:ispmgr_deny_sub - [0:0]
:ispmgr_limit_req - [0:0]
# Using to filter traffic
-A INPUT -j ispmgr_deny_ip
-A INPUT -j ispmgr_allow_ip
-A INPUT -j ispmgr_allow_sub
-A INPUT -j ispmgr_deny_sub
-A INPUT -m set --match-set ispmgr_limit_req src -j DROP
# Allow access to port 1501 for a subnetwork
-A ispmgr_allow_sub -p tcp -m tcp --dport 1501 -j ACCEPT
COMMIT
# Completed on Sat May 18 23:58:32 2024

From the contents of the iptables /etc/ispiptable.conf file, we can see that all the ports are open. This is not safe. To run ispmanager safely, you need to apply stricter rules.

Here's how to customize iptables:

Open ports:

SMTP. Ports 25, 465, 587, protocol: TCP
IMAP. Ports 143, 993, protocol: TCP
POP3. Ports 110, 995, protocol: TCP

Be sure to close all incoming connections, allow only established connections, and then open ports for the following services:

Ispmanager. Port 1500, Protocol: TCP
HTTP. Port 80, Protocol: TCP
HTTPS. Port 443, Protocol: TCP
SSH. Port 22, Protocol: TCP
FTP. Port 21, Protocol: TCP
FTP passive mode, Range 49152:65534, Protocol: TCP
DNS Port 53, Protocol: UDP

To access these ports:

1.Add rules to grant access to mail server ports.

For IPv4:

# Allow incoming connections on port 21 for FTP
sudo iptables -A INPUT -p tcp --dport 21 -j ACCEPT
# Allow incoming connections on ports 49152-65534 for passive FTP mode
sudo iptables -A INPUT -p tcp --dport 49152:65534 -j ACCEPT
# Allow incoming connections on port 22 for SSH
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
# Allow incoming connections on port 80 for HTTP
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
# Allow incoming connections on port 443 for HTTPS
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT
# Allow incoming connections on port 1500 fo ispmanager
sudo iptables -A INPUT -p tcp --dport 1500 -j ACCEPT
# Allow incoming connections on port 53 for DNS
sudo iptables -A INPUT -p udp --dport 53 -j ACCEPT
# Allow incoming connections on port 25 for SMTP
sudo iptables -A INPUT -p tcp --dport 25 -j ACCEPT
# Allow incoming connections on port 465 for SMTPS
sudo iptables -A INPUT -p tcp --dport 465 -j ACCEPT
# Allow incoming connections on port 587 for Submission
sudo iptables -A INPUT -p tcp --dport 587 -j ACCEPT
# Allow incoming connections on port 143 for IMAP
sudo iptables -A INPUT -p tcp --dport 143 -j ACCEPT
# Allow incoming connections on port 993 for IMAPS
sudo iptables -A INPUT -p tcp --dport 993 -j ACCEPT
# Allow incoming connections on port 110 for POP3
sudo iptables -A INPUT -p tcp --dport 110 -j ACCEPT
# Allow incoming connections on port 995 for POP3S
sudo iptables -A INPUT -p tcp --dport 995 -j ACCEPT
# Allow established connections
sudo iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow incoming connections to localhost
sudo iptables -A INPUT -i lo -j ACCEPT
# Set the default policy for the INPUT chain to DROP
sudo iptables -P INPUT DROP

For IPv6:

# Allow incoming connections on port 21 for FTP.
sudo ip6tables -A INPUT -p tcp --dport 21 -j ACCEPT
#  Allow incoming connections on ports 49152-65534 for passive FTP mode.
sudo ip6tables -A INPUT -p tcp --dport 49152:65534 -j ACCEPT
# Allow incoming connections on port 22 for SSH.
sudo ip6tables -A INPUT -p tcp --dport 22 -j ACCEPT
# Allow incoming connections on port 80 for HTTP.
sudo ip6tables -A INPUT -p tcp --dport 80 -j ACCEPT
# Allow incoming connections on port 443 for HTTPS.
sudo ip6tables -A INPUT -p tcp --dport 443 -j ACCEPT
# Allow incoming connections on port 1500 for ispmanager.
sudo ip6tables -A INPUT -p tcp --dport 1500 -j ACCEPT
# Allow incoming connections on port 53 for DNS.
sudo ip6tables -A INPUT -p udp --dport 53 -j ACCEPT
# Allow incoming connections on port 25 for SMTP.
sudo ip6tables -A INPUT -p tcp --dport 25 -j ACCEPT
# Allow incoming connections on port 465 for SMTPS.
sudo ip6tables -A INPUT -p tcp --dport 465 -j ACCEPT
# Allow incoming connections on port 587 for Submission.
sudo ip6tables -A INPUT -p tcp --dport 587 -j ACCEPT
# Allow incoming connections on port 143 for IMAP.
sudo ip6tables -A INPUT -p tcp --dport 143 -j ACCEPT
# Allow incoming connections on port 993 for IMAPS.
sudo ip6tables -A INPUT -p tcp --dport 993 -j ACCEPT
# Allow incoming connections on port 110 for POP3.
sudo ip6tables -A INPUT -p tcp --dport 110 -j ACCEPT
# Allow incoming connections on port 995 for POP3S.
sudo ip6tables -A INPUT -p tcp --dport 995 -j ACCEPT
# Allow established connections and the associated packets.
sudo ip6tables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow local loopback connections (localhost).
sudo ip6tables -A INPUT -i lo -j ACCEPT
# Set the default policy for the INPUT chain to DROP (block all incoming connections).
sudo ip6tables -P INPUT DROP

2.Check that the rules have been added.

For IPv4:

sudo iptables -L -vn

For IPv6:

sudo ip6tables -L -vn

3.Save your changes.

For IPv4:

sudo iptables-save > /etc/ispiptable.conf

For IPv6:

sudo ip6tables-save > /etc/ispip6table.conf

Now your mail server is protected from unauthorized access and will run smoothly.

Configuring Fail2ban for Exim and Dovecot

Fail2ban automatically responds to suspicious activity and prevents brute-force attacks.

Setting up Fail2ban.

In the ispmanager interface:

Go to "Server" → "Software Installation".
Select Fail2ban and install it.

In the command line:

sudo apt-get install fail2ban

Configuring Fail2ban from the ispmanager interface:

Log in to the ispmanager interface.
Go to the Monitoring and Logs section.
Select "Network Services".

Network Services:

1.Check the desired service you want to protect, for example, sshd. Click the "Fail2ban Rules" button. The "Service Rules" page opens.

2.Select the appropriate service, that is, sshd in this example. Then, activate the rule by clicking the appropriate button as shown in the examples.

Service Rules:

To prevent Fail2ban from blocking your IP:

Go to the Monitoring and Logs section.
Select "Network Services".
Click on the " Fail2ban Settings" button. The "Global Settings Fail2ban" page will open, where you can set the number of attempts and the ban time.

Then:

input your IP in the "Never Ban" field,
and check the "Apply to all rules" box.

4.Click Save.

Restart Fail2ban in the ispmanager interface:

Go to "Server" → "Manage Services".
Select Fail2ban and restart it.

What issues may arise? There may be something critical lacking in your OS, rsyslog may not be installed. Here is how you can get rule Dovecot working in Fail2ban:

1.Install rsyslog:

sudo apt install rsyslog

2.Create the /var/log/mail.log file and set permissions:

sudo touch /var/log/mail.log && sudo chown syslog:adm /var/log/mail.log

3.Restart rsyslog and dovecot again:

sudo systemctl restart rsyslog dovecot

4.Verify that logs are running:

sudo tail -f  /var/log/mail.log
Jun 17 22:31:02 mail dovecot: master: Warning: Killed with signal 15 (by pid=3037075 uid=0 code=kill)
Jun 17 22:31:03 mail dovecot: master: Dovecot v2.3.16 (7e2e900c1a) starting up for imap, pop3 (core dumps disabled)
Jun 17 22:31:03 mail dovecot: config: Warning: /etc/dovecot/conf.d/90-plugin.conf line 12: Global setting mail_plugins won't change the setting inside an earlier filter at /etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid this warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf line 47)

If the main log is running, I don't see the point in analyzing it in detail as it is not critical.

5.Restart fail2ban:

sudo fail2ban-client restart

After performing these steps, all the rules are working. Check the status of Fail2ban as shown in the following:

sudo fail2ban-client status
Status
|- Number of jail:      5
`- Jail list:   dovecot, exim-isp, exim-spam, sieve, sshd

Configuring the Fail2ban configuration on the command line

Make changes in the file /etc/fail2ban/jail.local:

# ispmanager start
[sshd]
maxretry = 5
enabled = true

backend = systemd
[exim-isp]
port   = smtp,465,submission
logpath = %(exim_main_log)s
maxretry = 3
enabled = true

[exim-spam]
port   = smtp,465,submission
logpath = %(exim_main_log)s
maxretry = 3
enabled = true

[dovecot]
port    = pop3,pop3s,imap,imaps,submission,465,sieve
logpath = %(dovecot_log)s
backend = %(dovecot_backend)s
maxretry = 3
enabled = true

[sieve]
port   = smtp,465,submission
logpath = %(dovecot_log)s
backend = %(dovecot_backend)s
maxretry = 3
enabled = true
# ispmanager end

To prevent your Fail2ban IP from blocking your IP, add your IP to section ignoreip in the[DEFAULT] configuration file as shown in the example below. Replace 1.2.3.4 with your current IP address:

[DEFAULT]
bantime = 600
ignoreip = 1.2.3.4
maxretry = 3

Save the changes and restart the rules in Fail2ban using the sudo fail2ban-client reload command.

4.Restart Fail2ban using the command:

sudo service fail2ban restart

Fail2ban will now monitor the Exim and Dovecot logs, blocking suspicious IP addresses, thereby securing your mail server.

RoundCube Security

RoundCube is a webmail client that provides access to e-mail through a web interface and open-source software the vulnerabilities of which are regularly published in databases such as exploit-db.com. The measures covered in this chapter help minimize the risk of attackers exploiting these vulnerabilities.

Restricting access to RoundCube offers:

Security. Access restriction helps prevent unauthorized access to mailboxes and sensitive information.

Spam and phishing protection. Access restriction helps prevent attackers from using RoundCube to send spam or phishing emails in bulk.

Resource efficiency. Access restriction helps manage server load by preventing redundant requests to RoundCube.

Compliance with security standards. Many organizations are required to meet certain security and data privacy standards. Restricting access to email clients helps meet these requirements.

Examples of exploits:

Recommendations for security in RoundCube.

Regularly update RoundCube and all its dependencies to prevent any vulnerabilities from getting exploited.

In this example, I'll look at methods for restricting access by IP address, configuring the web server Nginx .htaccess file, changing settings in the RoundCube itself, and other general recommendations.

Web server configuration (Nginx).

For Nginx, you can use the allow directive and deny in the configuration file:

cat /etc/nginx/vhosts-includes/roundcube-nginx.conf
location /roundcube {
    allow 1.2.3.4;
    allow 192.168.1.0/24;
    deny all;
    alias /var/lib/roundcube;
    index index.php;
}
location ~* ^/roundcube/(.+\.(jpg|jpeg|gif|css|png|js|ico|html|xml|txt))$ {
    alias /var/lib/roundcube/$1;
    error_page 404 @apache;
}
location ~ ^/roundcube/(.+\.php)$ {
    allow 1.2.3.4;
    allow 192.168.1.0/24;
    deny all;
    alias /var/lib/roundcube/$1;
    fastcgi_pass unix:/var/run/php-fpm.www-data.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $request_filename;
    fastcgi_param PHP_VALUE "display_errors=off \n display_startup_errors=off";
    include fastcgi_params;
    error_page 502 = @apache;
    error_page 404 = @apache;
}
location @apache {
    error_log /dev/null crit;
    proxy_pass http://127.0.0.1:8080;
    proxy_redirect http://127.0.0.1:8080 /;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

To restrict access to RoundCube, simply use the following settings: replace the 1.2.3.4 and 192.168.1.0/24 administrator IP addresses.

To prevent accidental configuration changes during updates, run the command:

sudo chattr +i /etc/nginx/vhosts-includes/roundcube-nginx.conf

Configuring RoundCube itself. In the RoundCube /etc/roundcube/config.inc.php configuration file, you can add IP address verification:

$allowed_ips = array('1.2.3.4', '192.168.1.0/24');
if (!in_array($_SERVER['REMOTE_ADDR'], $allowed_ips)) {
    die('Access denied');
}

Please note that updates to RoundCube may overwrite any changes to this file. To protect the configuration file from accidentally getting changed, run the following command in the terminal:

sudo chattr +i /etc/roundcube/config.inc.php

Customizing the .htaccess file.

You can specify the permitted IP addresses in the file .htaccess:

<IfModule mod_rewrite.c>
    RewriteEngine on
    RewriteCond %{REMOTE_ADDR} !^1\.2\.3\.4$
    RewriteCond %{REMOTE_ADDR} !^192\.168\.1\.[0-9]+$
    RewriteRule ^ - [F,L]
</IfModule>

This scenario is relevant when RoundCube is running Apache. With the default ispmanager settings, file .htaccess will not work.

This chapter describes only the basic methods for increasing security with RoundCube. If you have any questions, post them in the comments.

AppArmor profiles for Exim, Dovecot, and RoundCube

AppArmor isolates each application and restricts application access to system resources. This reduces the risks of unauthorized access.

Scripts for creating AppArmor profiles. Here are my own scripts for automatically configuring AppArmor profiles on GitHub. These are not ispmanager products and we accept no liability for their operation or security.

The AppArmor script and profiles are distributed under the GPL-3.0 license. By downloading the script, you do so at your own risk and are responsible for its use.

Summary and recommendations

Tips for further configuring and maintaining your mail server:

In this two-part article, we went over the steps for creating and configuring a secure, functional mail server. Now, you can configure your mail server to ensure your emails don’t get sent to spam.

We used:

AppArmor, which restricts what applications can do and increases the overall level of protection.

A network filter with iptables configuration to manage network connections and prevent unauthorized access.

Fail2ban to protect against brute-force attacks by automatically blocking malicious IP addresses.

Greylisting, DNSBL, SpamAssassin to fight spam.

ClamAV for antivirus protection by scanning for and protecting from malware. It also restricts access to RoundCube, only letting whitelisted users in.

Four routines to keep your mail server secure:

Regular updates. Ensure that all components of the mail server and security system are updated regularly to protect against emerging threats.

Monitoring. Use monitoring tools to track server performance and security and respond to suspicious activity.

Backup. Set up regular backups of your mail server data so that you do not lose any data.

Establish a user policy and training. Create and enforce strict security policies for users. At least ensure that users create complex passwords, set up two-factor authentication, and know how to use email services securely.

Here are more of my articles about information security:

How to test and secure a mail server in Debian and Ubuntu with ispmanager

Testing mail server settings

Customizing Firewall iptables

Configuring Fail2ban for Exim and Dovecot

RoundCube Security

AppArmor profiles for Exim, Dovecot, and RoundCube

Summary and recommendations