SurgeMail has a number of features to make the migration from an existing mail server to your new SurgeMail installation easier. These can be combined to your choice to make your mail migration as painless as possible for both you and your users. However migrating from one mail server to another should always be done carefully and should be fully controlled. Important considerations are factors such as simplicity of upgrade procedure, length of downtime and ability to rollback.
Unless you have
specific requirements, the recommended method for upgrading to
SurgeMail is POP / IMAP intercept mode. We aim to make the
migration as simple and safe as possible. If you have any
suggestions on how we can improve the migration process please
let us know:
Note: In all cases we recommend you start by making a backup copy of your existing mail server's mailbox files :-)
This is a simple zero downtime method for migrating all active accounts from any arbitraty mailserver to SurgeMail. This method configures SurgeMail to be the new mailserver for all users. When a user logs into SurgeMail and a local account does not exist, SurgeMail will login to the old server check the account is valid. If the account is valid SurgeMail will create an account and retrieve mail for this account storing it locally. From now on the user will use SurgeMail as their primary mailserver and mail for them will be delivered locally. Mail delivered to SurgeMail for users that have not yet logged in will be forwarded on to your old mail server.
Detailed examples and documentation can be found here on using this method. Note! migration cannot be used in conjunction with CRAM-MD5 as that prevents surgemail from storing the old password. CRAM-MD5 is best not used as SSL offers better security.
You can also use this command line tool to import a single account or test importing an account. Examine migration.log if it is not successful:
tellmail imap_import newuser@domain oldimaphost olduser oldpass delete|keep create|nocreatee.g. to import an account, without deleting the messages from the old one, use:
tellmail imap_import email@example.com mailsever.xyz.com firstname.lastname@example.org SECRET keep create
If you wish to test an account
by logging in to it, and then you wish to delete the account and
test it again you would need to issue this command so that
surgemail will know it needs to retry:
tellmail pstat_delete "email@example.com"
To make it do all accounts again use:
tellmail pstat_delete "*"
(this can result in
duplicate messages, normally you would only do this if you had
been testing and you wanted to delete the mailbox folders manually
and start again!)
SurgeMail has a number of features to make the migration from an existing non SurgeMail mailserver to SurgeMail easier:
SurgeMail has the option "fallback_relay" (configurable per domain) that allows mail to be delivered to a different host if the user does not exist locally. This can be used whilst testing that a system is operational and if desired for the gradual migration of users from an existing server to the current server. The two ways of configuring this are as follows:
1. Configure SurgeMail
to handle the domain served by your existing server. Keep your
MX record pointing at your existing server. On the existing
server cc (or redirect) mail to your SurgeMail configuration.
For the account(s) in question start using SurgeMail to retrieve
and send mail. In this configuration rollback is easy by simply
switching your mail client back to the original server.
Note: If you are sending to servers that do a reverse lookup you will need to get SurgeMail to relay outbound through your existing server.
2. Configure SurgeMail
to handle the domain served by your existing server. Switch your
MX record to the SurgeMail server. Now if the account in
question is defined in the SurgeMail user database, SurgeMail
will process mail for this user and the users mail client will
need to point to SurgeMail. If the account is not present all
mail will be forwarded to your existing mailserver and the users
mail client will need to point to your existing mail server.
Note: Again if you are sending to servers that do a reverse lookup you will need to get your existing server to relay outbound mail through the SurgeMail server.
There probably is no need to is no need to import your existing user database as SurgeMail is very likely to have an authentication module that will use your existing database directly. Having said that many administrators do choose to convert their existing user database to NWAuth. This is especially so if using system based authentication as NWAuth does not require a UNIX system account for every user in your authentication database.
If converting your sendmail configuration your existing /etc/passwd + /etc/shadow account database can be imported to NWAuth format without users having to change their password or the encrypted password being decoded either by manually copying the fields or using the following script xferusers.pl.
SurgeMail uses maildir format to store delivered mail. SurgeMail has the inbuilt ability to convert standard drop files to maildir format. In addition SurgeMail will convert dmail bin files and mbx files to SurgeMail mdir format. The settings that control this process are the surgemail.ini vdomain settings: dmail_drop_path, dmail_bin_path and dmail_hash.
eg: If you were converting a Windows DMail configuration the upgrade settings would be as follows, these are automatically added if you do SurgeMail install based on a dmail.conf.
dmail_bin_path "D:\somewhere\binfiles" dmail_drop_path "D:\somewhere\dropfiles" dmail_hash "0"
eg So unhashed UNIX Sendmail drop files can be converted as follows:
dmail_drop_path "/var/spool/mail" dmail_hash "0"
The default SurgeMail behaviour is to check for mail to convert when a user logs in using POP or IMAP. This means that normally the conversion load is evenly spread over the timeframe that users login. However larger accounts may see a delay in checking their mail while the mail conversion takes place. This can be prevented by doing the conversion of all "outstanding old mail" in batch mode using the following command:
This command requires dmail_bin_path and dmail_drop_path and dmail_hash settings to have been setup correctly prior to running the command.
For pine format unix drop files (Each folder is contained in a single file with each message starting with the line 'From ...', you can use this command to import mail folders (other than the inbox which the dmail conversion commands will handle) This commands requires SurgeMail 3.7c4 or later.
tellmail dropfile_import firstname.lastname@example.org /home/john/mail
As each mail server configuration tends to be unique and specifically configured we do not support a generic upgrade ability from an arbitrary mail server. We aim to supply the capability to easily migrate the necessary elements of your configuration to surgemail.
Having said that, SurgeMail does have the ability to upgrade an existing DMail configuration. It does this by generating a surgemail.ini configuration based on the dmail.conf configuration. This reuses the authentication database that DMail used and adding the settings described in "Delivered Mail Conversion" above. Then when a user logs in mail existing mail is converted to the correct SurgeMail location.
SurgeMail has been
designed to make migration between two SurgeMail servers of
similar or of different platforms very easy but again there are
several different ways to migrate. We recommend the Mirroring
migration method, see instructions here.
NOTE: If you upgrade to the latest surgemail
version in the process of transferring to a new server,
then you will need to purchase updates (if you don't currently
have updates). Alternatively, you can install a matching
'old' version of surgemail on the new system. (Old releases are
The easiest and currently the recommended way of moving SurgeMail between servers of the same platform is by copying the whole SurgeMail directory tree. To do this, install a default configuration on the new server, then copy the entire SurgeMail directory tree (plus /etc/surgemail.ini) to the new server, move your license key from one host to the other using tellmail deactivate / activate, and update your mx record.
"Moving" your SurgeMail key:
Alternatively the relevant data files can be copied. This is the recommended way when moving between platforms or you wanting a clean install on the same platform. To do this you will need to install a default configuration on the new server, then copy the following configuration, mail and data files from the old installation to the new installation, move your license key, and update your MX record.
To copy SurgeMail data directories you will need to copy:
On UNIX be sure to correct the file ownerships to 'mail' if you coppied the files as root! (chown -R mail /usr/local/surgemail)
In both the above two cases neither the old or new SurgeMail installations should not be running whilst copying files. If this is not acceptable in terms of interruption of service you can use migration using pop intercept mode or migration using mirrorring. The optimal procedure is still being finalised for zero downtime upgrades so please test your chosen migration path on a non live system first and again suggestions for improvement are welcome.
The mirrorring feature can be used to migrate user accounts and mail from one server to another using zero downtime. To do this you need to Setup your new server in the same configuration as your old server by copying the configuration files and install a temporary second license key. Enable mirrorring on both systems. Add an MX record with higher priority for your new server. At this stage both servers will be live and mail can be delivered to either server and it will be mirrored to the other server. When you are confident the new server is working as expected you remove the mx record for the old system, wait for any remaining mail to be mirrored to the new server and take the old server offline.
Several things should be carefully noted. Firstly mirroring only duplicates delivered mail, more specifically files that get mirrored are the files stored in the surgemail/mydomain.com maildir directories for each domain. This means that certain settings or files will not get mirrored as follows:
When running WebMail in POP mode the mail stored in folders is stored in the WebMail work area. When running WebMail in IMAP mode this mail is stored in the SurgeMail folders and is accessible using other IMAP clients. To convert this mail do the following:
Maildir implementations vary slightly in their implementation. SurgeMail is able to import other Maildir formats by running the command "tellmail maildir_convert surgemail_domain.com source_directory" where source directory is a fully specified path with $HASH1 / $HASH2 and $USER, where expansion of directories is required.
tellmail maildir_convert mydomain.com '/home/$USER/Mail' tellmail maildir_convert mydomain.com '/home/$HASH2/$USER/Mail' tellmail maildir_convert mydomain.com '/var/spool/mail/$HASH1/$USER/Mail'
$HASH1 = /fr (for fred)
$HASH2 = /f/r (for fred)
$USER = wild card scan of usernames
eg: Converting all mail stored in the format /var/spool/mail/user1/Mail to /usr/local/surgemail/mydomain.com/bb/wj/user1/
tellmail maildir_convert '/var/spool/mail/$USER/Mail'
Using 5.1 or later you can create a file called 'virtusertable.dat' to translate user names given without a username during pop/imap login into a user and domain name pair, this should only be used when converting users from an old sendmail system where you need to translate the users to multiple different domains, e.g.
fred -> email@example.com
bob -> firstname.lastname@example.org
joe -> email@example.com
The file format is as follows: