Email Steps to ensure your site/server email doesn't end up in spam inboxes

To ensure your site/server sent emails do not end up in users' spam inboxes, you need proper SPF/TXT, DKIM, DMARC and reverse PTR DNS records setup for your domain and server's main hostname (setup via Getting Started Guide Step 1) as outlined below.

There is a distinction between your site domain and server's main hostname:

1. server's main hostname - If you use a web app like Wordpress, Xenforo, IP.Board, Magento, etc to sent emails. Then emails can either be sent as is via Centmin Mod Postfix MTA server via root@host.domain.com where receiving email servers would evaluate the SPF, DKIM, DMARC and PTR DNS records for sending domain @host.domain.com. If server's main hostname i.e. host.domain.com has valid DNS records, then email will be less likely sent to spam box or being rejected by receiving email server. If you use Cloudflare, DO NOT enable orange clooud proxy on server's main hostname DNS record as you prevent receiving mail servers looking up the main hostname's real IP address to verify your mail server.
2. your domain - If your web app allows you to set a from email field and you sent emails from say user@domain.com, then receiving email servers would evaluate the SPF, DKIM, DMARC and PTR DNS records for sending domain @domain.com so your domain.com would need to have those DNS records setup.

Centmin Mod is provide as is, so short of scripted related bugs or issues, any further optimisation to the web stack components - nginx, php-fpm, mariadb mysql, csf firewall etc or web app specific configurations are left to the Centmin Mod user to deal with. So I do not provide any free support for such things as SPF/TXT and DNS records setup. For SPF/DNS, you're best to ask your web host and/or domain registrar and follow below instructions and check out thread at Email - error in PTR, mails goes to gmail Spam folder.

However, Centmin Mod users are free to help each other out and ask questions or give answers on this community forum. My hopes are that this community forum evolves so that more veteran long time Centmin Mod users help new Centmin Mod users out

Generally, to ensure email sent from your Centmin Mod LEMP server's postfix MTA doesn't end up in email recipient's spam box, you need to at least have proper DNS level configurations for

1. SPF / TXT record setup in DNS with your DNS provider. SPF / TXT record is a DNS text record so how you add it will depend on your DNS provider. Some info here, here and here. Also SPF guides from some Domain/DNS providers such as Namecheap.com, Godaddy.com, and check out SPF Wizard - SPF Generator DNS tool. Ensure both your site's domain name and server's main hostname from Getting Started Guide step 1 have valid SPF TXT records
2. Reverse PTR record, you will need to ask your web host for that. Some web hosts and VPS providers use SolusVM control panels and allow you to setup a PTR record yourself. Basically it points your server's IP to your domain name. So when a reverse lookup on the IP is made, it resolves to your domain name. For instance the PTR record for this forum's IP resolves to community.centminmod.com when you do a reverse IP lookup at IP Address Lookup (IPv4 & IPv6). I use Linode.com for my forum hosting and their control panel has the relevant PTR record support for reverse dns.
3. DKIM setup using addons/opendkim.sh addon script for Centmin Mod users' email sent from their @mainhost.yourdomain.com. For emails @yourdomain.com, your 3rd party email provider will have DKIM setup instructions.
4. DMARC record setup. You can use Postmark's DMARC setup service for this. More info on DMARC.
5. Google Postmaster Tools. Sign up your site domains for this service. Postmaster Tools FAQ
6. Ensuring server's native Postfix MTA mailserver's outbound SMTP port is working as some VPS providers now block them by default. So you may need to setup Postfix SMTP relays with a 3rd party SMTP provider. For example, DigitalOcean SMTP Outbound Email Blocked By Default

IP Blacklist Check

Also check if your web site and server IP addresses are in spam blacklists using services like:

• Email Blacklist Check - See if your server is blacklisted
• Blacklist Check
• Blacklist Lookup | Sender Score | Return Path
• IP Reputation Investigation
• ReputationAuthority | WatchGuard Technologies

Straight from Amazon SES best practices guide, you should also check the URL links that you use in your emails against blacklists as well, because some ISPs will block email with blacklisted links. URIBL.com and SURBL.org are two very useful sites that you can use to determine whether your links are blacklisted. Remember to check any links provided to you by a third party or any link shorteners, which obfuscate your domain or links.

Transactional Email Providers

For forum, CMS etc outbound email and best email deliverability, best to use a 3rd party SMTP transactional email provider. Discussion of provider options can be read at Email - Mandrill users will be required to have a paid monthly MailChimp account. I use Amazon SES for this forum's emails.

SPF Setup Guides

• Create a SPF record for your server - mail-tester.com

SPF Wizards

You can use SPF wizards to try and get the correct settings

• SPF Wizard - SPF Generator DNS tool <-- probably best one
• easySPF: an Ajax enabled SPF Wizard
• Create SPF records with MailRadar SPF Wizard
• SPF Wizard
• xNode - SPF Record Creator

SPF Testing Tools

• Also test validity of your SPF txt record at SPF Query Tool
• whatsmydns.net - select TXT record dropdown and enter domain as well as server's main hostname from Getting Started Guide step 1
• solvedns.com - select TXT record dropdown and enter domain as well as server's main hostname from Getting Started Guide step 1

DMARC Testing Tools

• dmarcian - DMARC Inspector

DKIM Testing Tools

• dmarcian - DKIM Inspector

Email Deliverability Testing Tools

• Email Deliverability Tool - Check Your Deliverability Report - MxToolBox - to test against server sent email send test email from server via SSH
  Code (Text):

  echo "test" | mail -s "test" ping@tools.mxtoolbox.com

  and then on web page to search for your email which would be root@host.yourdomain.com where it would be your server's main hostname reported via command
  Code (Text):

  hostname -f

You would get a report similar to one below returned from the search

---

Microsoft & Outlook Email

• How to prevent false positives from happening in Office 365

 

Also DKIM setup is another step to further improve your email deliverability chances. Keep an eye on addons/opendkim.sh addon developments.

 

Added 5th step to sign up your domains for Google Postmaster Tools to monitor your domains spam status reported by users who receive your emails.

i.e. for centminmod.com for past 120 days

 

To elaborate more on 1st post in this thread, more info provided below.

Troubleshooting Proper Email Sending

To troubleshoot and/or ensure server sent emails via Postfix and from @domain.com your domains are delivered to their destination, do the following:

Ensure Proper DNS Records Setup

Ensure you have setup proper SPF, DKIM, DMARC & PTR DNS records for your main hostname i.e. hostname.domain.com and all @domain.com domain names as per outlined and linked instructions at here and Getting Started Guide Step 1 for main hostname (i.e. hostname.domain.com). The addons/opendkim.sh addon in 123.09beta01 will help you setup SPF/DKIM for main hostname of your Centmin Mod server. Without these DNS records for your main hostname and all domain names, the likelihood of your server sent and @domain.com sent emails delivering to target inboxes is lessened and may end up in user's spam box or being rejected.

You can use these online DNS toolboxes to verify you have properly setup SPF, DKIM, DMARC and PTR DNS records

• Network Tools From MxToolBox
• DNS Checker - DNS Propagation Check & DNS Lookup

Testing Server Sent Emails

Use Newsletters spam test by mail-tester.com to send a test email from your server to the email address listed on the mail-tester.com web site - DO NOT leave the site yet though.

To Test Server Sent Email

For example if the site lists email web-1a0xv@mail-tester.com, in SSH you use mail command line to sent a test email with subject "mail-tester" and email contents of "mail-test"

Code (Text):

echo "mail-test" | mail -s "mail-tester" web-1a0xv@mail-tester.com

If you use Amazon AWS SES smtp Postfix relay on Centmin Mod server, then you also need to send from email address that is verified with Amazon AWS SES i.e. user@yourdomain.com

Code (Text):

echo "mail-test" | mail -s "mail-tester" -r user@yourdomain.com web-1a0xv@mail-tester.com

Otherwise, your Postfix /var/log/maillog might have an entry with message like below:

Code (Text):

status=bounced (host email-smtp.us-east-1.amazonaws.com[52.206.145.59] said: 554 Message rejected: Email address is not verified. The following identities failed the check in region US-EAST-1

Which means on Amazon AWS SES dashboard, you need to ensure the email and/or domain is verified for user@yourdomain.com or yourdomain.com


To Test PHP mail() Sent Email

To test PHP mail() function sent email, create a test-mail.php file (can be any name you want) with contents - changing the web-1a0xv@mail-tester.com to whatever you see on mail-tester.com web site

PHP:

<?php
    ini_set('display_errors', 1);
    error_reporting(-1);
    mail ('web-1a0xv@mail-tester.com', 'Postfix Test', 'A test email') || print_r(error_get_last());
?>


save test-mail.php to /home/nginx/domains/domain.com/public/ web root directory and then open up domain.com/test-mail.php on your server. Remember to delete test-mail.php afterwards.

Then click the 'then check your score' button on mail-tester.com site. It will check the email you sent and all it's email headers and give you a report card for the sending domain which is usually your main hostname in default Centmin Mod setups i.e. if sending email is root@hostname.domain.com, then sending domain is hostname.domain.com. You can ignore for subdomains the warning of no MX records for subdomain/hostnames as subdomain/hostnames don't require MX records only the @domain.com domain for receiving emails. This main hostname domain hostname.domain.com would be important for emails sent from your Centmin Mod server so need the required SPF, DKIM, DMARC and PTR DNS records mentioned above.

This checks php mail() from server sent From = root@host.domain.com where host.domain.com is your main hostname. To test php mail() with a proper From = your@domain.com which maybe set in your PHP web app like Xenforo, Wordpress etc. Use a slight variation of the test-mail.php php file below - changing the web-1a0xv@mail-tester.com to whatever you see on mail-tester.com web site and changing YOURFROM@domain.com email address to the one you set in your PHP web app like Xenforo, Wordpress etc:

PHP:

<?php
    ini_set('display_errors', 1);
    error_reporting(-1);
   $headers = "From: YOURFROM@domain.com";
    mail ('web-1a0xv@mail-tester.com', 'Postfix Test', 'A test email',$headers) || print_r(error_get_last());
?>

then open up domain.com/test-mail.php on your server. Then click the 'then check your score' button on mail-tester.com site

You can also test your @domain.com emails via SSH by adding the From email -r

Code (Text):

echo "mail-test" | mail -s "mail-tester" -r user@domain.com your@gmail.com

Then checking your Gmail account including spam box for the email and looking at 'Show Original' for full headers.

You can also test other target emails i.e. your @gmail.com account via SSH command

Code (Text):

echo "mail-test" | mail -s "mail-tester" your@gmail.com

Then checking your Gmail account including spam box for the email and looking at 'Show Original' for full headers.

Example of Gmail received email's 'Show Original' headers where my centminmod.com domain has working SPF, DKIM and DMARC DNS records setup from my 3rd party Google App/Gsuite email provider service. This is from email sent from my server where from email field was @centminmod.com.

• My centminmod.com SPF record. For main hostname, use the SPF tools and generators linked to here to create your SPF record or use addons/opendkim.sh to generator both DKIM + SPF records. For @domain.com your 3rd party email provider would provide those for you i.e. Google Gsuite/Google App, Zoho, Outlook/Microsoft etc.
• My centminmod.com DKIM record you need to specify the selector, so if you setup DKIM record as google._domainkey.centminmod.com, then the selector is google and for mxtoolbox.com DKIM check you'd enter it in format of domain:selector i.e. centminmod.com:google. For main hostname, Centmin Mod users can use addons/opendkim.sh to generate both the DKIM + SPF records. For @domain.com your 3rd party email provider would provide those for you i.e. Google Gsuite/Google App, Zoho, Outlook/Microsoft etc.
• My centminmod.com DMARC record which is just a TXT DNS record so can be looked up as a TXT record as well. You can create DMARC records via free service at https://dmarc.postmarkapp.com/ but DMARC only works when you have both valid working SPF and DKIM DNS records. On the free service site, on the 'Send reports about this domain' field enter your domain.com or subdomain.domain.com (i.e. main hostname host.domain.com) in the field. You can use https://www.learndmarc.com/ to test your DMARC, SPF, DKIM setup is correct for sending email domains.

Notes For Web App Sent Emails

Now if you use a web app like Wordpress, Xenforo, IP.Board, Magento, etc to sent emails. Then emails can either be sent as is via Centmin Mod Postfix MTA server via root@host.domain.com where receiving email servers would evaluate the SPF, DKIM, DMARC and PTR DNS records for sending domain @host.domain.com. If server's main hostname i.e. host.domain.com has valid DNS records, then email will be less likely sent to spam box or being rejected by receiving email server.

If however, your web app allows you to set a from email field and you sent emails from say user@domain.com, then receiving email servers would evaluate the SPF, DKIM, DMARC and PTR DNS records for sending domain @domain.com so your domain.com would need to have those DNS records setup. The above Gmail 'Show Original' email headers screenshot above is an example of such cases.

Using above mail-tester.com online tool is easiest way to check if all those SPF, DKIM, DMARC and PTR DNS records for sending domain @host.domain.com and @domain.com are setup correctly.

Main Host Name Setup

There is a reason the official Centmin Mod Getting Started Guide Step 1 outlines a proper main hostname SPF, DKIM, DMARC and PTR DNS records setup, as it is used for many purposes such as statistics hosting on main hostname vhost, phpmyadmin.sh installer setup of phpmyadmin on main hostname but also for ensuring proper email delivery from main hostname for emails sent from the server.

As such your main hostname for your Centmin Mod server needs to be unique. The main hostname has a separate Nginx vhost config file at /usr/local/nginx/conf/conf.d/virtual.conf where server_name defined there is auto populated to the server main hostname.

You can check if the main hostname is uniquely setup via this grep command in SSH type

Code (Text):

grep -rw server_name /usr/local/nginx/conf/conf.d/ | grep -w "$(hostname)"

Ensure only 1 entry match is found and that is usually the one contained in main hostname's Nginx vhost config file at /usr/local/nginx/conf/conf.d/virtual.conf

example output of 1 entry match you should expect as host.domain.com should only be referenced once in server_name for main hostname's Nginx vhost config file at /usr/local/nginx/conf/conf.d/virtual.conf

Code (Text):

grep -rw server_name /usr/local/nginx/conf/conf.d/ | grep -w "$(hostname)"
/usr/local/nginx/conf/conf.d/virtual.conf:            server_name host.domain.com;

If they don't match or if more than one Nginx vhost file is returned from the grep search, then you need to change and update DNS for the main hostname as per Centmin Mod Getting Started Guide Step 1 and then edit the server_name listed in /usr/local/nginx/conf/conf.d/virtual.conf to the newly changed main hostname.

The main hostname domain i.e. host.domain.com is the domain that server sent emails via Postfix MTA is sent from, so needs to properly setup on server and with proper DNS records.

Postfix pflogsumm Stats

Centmin Mod by default installs pflogsumm tool for Postfix log statistics. Centmin Mod only provides the tool for your usage, interpretation and understanding of the output is left up to you to figure out.

There's a default alias command you can type for today only stats, postfixlog

The alias for postfixlog command is

Code (Text):

alias postfixlog='pflogsumm -d today --verbose_msg_detail /var/log/maillog'

which is basically same as calling pflogsumm directly as

Code (Text):

pflogsumm -d today --verbose_msg_detail /var/log/maillog

There's also other commands you can directly use to by invoking pflogsumm command.

To see yesterday's stats instead of today

Code (Text):

pflogsumm -d yesterday --verbose_msg_detail /var/log/maillog

Report for all /var/log/maillog* in sorted time ascending order

Code (Text):

pflogsumm $(ls -rt /var/log/maillog*)

To filter on specific date of the month, i.e. Mar 26

Code (Text):

grep 'Mar 26' /var/log/maillog | pflogsumm --verbose_msg_detail

Spoiler: Click to show pflogsumm example output report

Code (Text):

grep 'Mar 26' /var/log/maillog | pflogsumm --verbose_msg_detail

Grand Totals
------------
messages

     23   received
     24   delivered
      0   forwarded
      0   deferred
      0   bounced
      0   rejected (0%)
      0   reject warnings
      0   held
      0   discarded (0%)

  24609   bytes received
  27282   bytes delivered
      3   senders
      2   sending hosts/domains
      5   recipients
      3   recipient hosts/domains


Per-Hour Traffic Summary
------------------------
    time          received  delivered   deferred    bounced     rejected
    --------------------------------------------------------------------
    0000-0100           0          0          0          0          0
    0100-0200           0          0          0          0          0
    0200-0300           0          0          0          0          0
    0300-0400           0          0          0          0          0
    0400-0500           3          3          0          0          0
    0500-0600           2          2          0          0          0
    0600-0700           2          2          0          0          0
    0700-0800           2          2          0          0          0
    0800-0900          10         11          0          0          0
    0900-1000           3          3          0          0          0
    1000-1100           1          1          0          0          0
    1100-1200           0          0          0          0          0
    1200-1300           0          0          0          0          0
    1300-1400           0          0          0          0          0
    1400-1500           0          0          0          0          0
    1500-1600           0          0          0          0          0
    1600-1700           0          0          0          0          0
    1700-1800           0          0          0          0          0
    1800-1900           0          0          0          0          0
    1900-2000           0          0          0          0          0
    2000-2100           0          0          0          0          0
    2100-2200           0          0          0          0          0
    2200-2300           0          0          0          0          0
    2300-2400           0          0          0          0          0

Host/Domain Summary: Message Delivery
--------------------------------------
 sent cnt  bytes   defers   avg dly max dly host/domain
 -------- -------  -------  ------- ------- -----------
     16    21656        0     0.1 s    0.1 s  host.domain.com
      6     4891        0     0.9 s    1.1 s  domain.com
      2      735        0     2.0 s    2.1 s  mail-tester.com

Host/Domain Summary: Messages Received
---------------------------------------
 msg cnt   bytes   host/domain
 -------- -------  -----------
     21    21726   host.domain.com
      2     2883   domain.com

Senders by message count
------------------------
     19   root@host.domain.com
      2   user+host@domain.com
      2   nginx@host.domain.com

Recipients by message count
---------------------------
     16   root@host.domain.com
      3   user+postfixx@domain.com
      2   user+host@domain.com
      2   web-zxxxc@mail-tester.com
      1   username@domain.com

Senders by message size
-----------------------
  20991   root@host.domain.com
   2883   user+host@domain.com
    735   nginx@host.domain.com

Recipients by message size
--------------------------
  21656   root@host.domain.com
   2883   user+host@domain.com
   1512   user+postfixx@domain.com
    735   web-zxxxc@mail-tester.com
    496   username@domain.com

message deferral detail: none

message bounce detail (by relay): none

message reject detail: none

message reject warning detail: none

message hold detail: none

message discard detail: none

smtp delivery failures: none

Warnings: none

Fatal Errors: none

Panics: none

Master daemon messages: none

 

Updated above guide's DMARC mention for using https://www.learndmarc.com/ to test your DMARC, SPF, DKIM setup is correct for sending email domains

 

This thread is even more important now as it outline steps above to ensure email deliverability from your server and domains as Google and Yahoo are stepping up their email deliverability requirements from https://netcorecloud.com/blog/bulk-email-marketing-gmail-yahoo-updates/. Be sure to use Google Postmaster to check your domain's email sending reputation https://community.centminmod.com/th...oesnt-end-up-in-spam-inboxes.6999/#post-37647

 

Just got an email reminding me that Google and Yahoo's new spam enforcement requirements for bulk email sending starts enforcement this month of June 2024. Just as well I developed a tool to clean and verify my email address lists for myself and my paid clients GitHub - centminmod/validate-emails: Self-hosted email verification script to clean up bad invalid email address lists. Supports various commercial email verification provider APIs all in one script

Email sender guidelines FAQ - Google Workspace Admin Help