Many of us get too much email. Some of us have figured out how to filter email to deal with it. And a very few of us have figured out how to write those filters from scratch, which basically makes us gods. Let me anoint you.

What’s email filtering?

Some people call this email “sorting” – whatever floats your boat. Whatever you call it, this is the process that many of us use to sort our incoming email into folders automatically for us. For many people, this is likely only useful for our work email, but there’s a population out there (like me) that has to filter personal email as well because we get so much of it. It’s not a popularity thing, it’s usually a robot thing. I have so many monitors and alerts and other processes running on my personal infrastructure that I get a lot of automated emails. I need those sorted properly otherwise it’s just a big pile of junk in my inbox and I miss things.

Filtering can go far beyond just sorting things into folders, however. Some of my filters tag emails so I know they are important, some of my filters automatically forward certain emails to other addresses. And, some of my filters just outright delete emails I don’t want to see but can’t unsubscribe from in the conventional sense.

There’s two places to filter email. The first place is when it arrives on the mail server. The second place is within your mail client. Server-side filtering is very much preferred, and the topic of this post.

Client-side email filtering

If you’re used to setting up filters in Thunderbird or non-work Outlook, or any other email program, then you’re used to client-side filtering. This type of filtering is wasteful because your email client has to download every piece of email and then analyze it to see if it matches one of your filters and, if so, process the email according to your wishes. Then it gets the next piece of email and does it all over again. This is time consuming and can make retrieving large volumes of email sucky.

The only advantage I can see to client-side filtering is that almost all email clients support it, whereas not all email service providers offer server-side filtering. Therefore, you can be assured in almost all cases that you can filter your email on your local machine regardless of whether your email service provider supports filtering.

Server-side email filtering

Filtering on the server has the same steps as filtering on the client. The server has to analyze every email as it comes in to see if it matches a filter and process it if so. However, unlike your laptop, the email server is a beefy beast that is optimized to process emails and can do it much faster that you. Even better, it does the filtering as the email arrives so even when you’re sleeping, your email is getting sorted. This makes retrieving your email much faster because your email client just has to pull it down with no additional overhead wasted on processing it.

There’s good filtering, and there’s the best filtering

Many email providers give you some kind of web interface to check your email. And, if the service supports it, you will also find some rudimentary filtering options in that interface. The basic filtering options usually offer you the ability to filter emails based on things like who it is from, the subject line, and perhaps some other characteristics. That’s good, but not the best.

The best email filtering allows you to filter on any characteristic of an email, not just the subset pre-selected by your email provider. Better email filtering uses Sieve. The best email filtering uses Sieve on the server.

Sieve goes far beyond just being a method to sort emails, it is a scripting language created for just this purpose, complete with its own RFC (5228). In a modern age where generic languages pushing general concepts are all the rage, it is refreshing to use a purpose-built language because specific tools are always more powerful than generic ones.

How to use Sieve

There are several scenarios where you may encounter Sieve. Your email host may offer it, you may run it locally on your system, or perhaps you host your own email and can configure it on your own server. I think email filtering is most useful on the server, and I use Proton Mail which (mostly) supports Sieve, so I will use that in my examples.

I start by navigating to my Proton Mail settings –> filters. I have a few set up. Note how the first two have an Edit button and the last two have an Edit Sieve button. That is because the first two filters were created using the standard filter interface, whereas I created the last two by writing a Sieve script. It is this last two we’re interested in.

Let’s open up the Servers Emails Sieve filter. I have a server that sends a variety of emails, mostly alerts, about events that occur. I use OSSEC on my server to perform HID/NID/FIM functions and I want OSSEC emails filtered to a specific folder. However, there are other emails from my servers that are not OSSEC related, and I want them to stay in my inbox. Incidentally, this is a nice way to keep those emails from going to spam. I rarely set up proper SPF/DMARC/DKIM records for server-level alert emails, so a lot of those end up in spam by default without a filter like this.

require [“fileinto”, “include”, “environment”, “variables”, “relational”, “comparator-i;ascii-numeric”, “spamtest”, “imap4flags”, “envelope”];

if header :contains “Received” [“93.184.216.34.example.com”] { if header :contains “subject” “OSSEC” { fileinto “OSSEC”;
addflag “\Seen”;
}
else { fileinto “Inbox”;
}
return; }

There’s a few basic components of this filter. The first is the require section. This bit tells Sieve what modules to load in order for it to be able to perform the actions in the script. The list you see here is Proton Mail’s default module list, except the fileinto and imap4flags modules. I added those to the list for these reasons:

• fileinto : required to allow Sieve to file the email into a folder.

• imap4flags : required to use the addflag action to mark the message read.

I could have removed some other modules that this particular filter will not use, but I know from experience that I will build on this filter and I see no downside to leaving them in. Now, let’s look at this (really simple) script.

When an email is received Sieve looks at the Received header to see if it contains the string 93.184.216.34.example.com and if so, it Sieve does a second check to see if the subject contains the string OSSEC. If both those conditions match, then Sieve performs the following actions:

• moves the email to my OSSEC folder.

• marks it read. This is not strictly necessary, but it prevents the “unread” counter from showing in my mail client which I prefer. I know I need to manually check my OSSEC folder periodically.

Next, if the email does not contain that header, then it leaves it in the Inbox folder. In theory, this last bit should not be necessary because if an email does not match a Sieve filter then Sieve leaves it alone. But as I mentioned earlier, Proton Mail files these messages into my Spam folder by default, so this is a nice way to prevent that from happening without actually solving the root problem of DNS records. I’m lazy, OK?

Let’s look at a slightly more complex example. I subscribe to a bunch of newsletters and I want them all to go into my Newsletters folder. The easiest way to do this is to filter by sender email address.

require [“fileinto”, “include”, “environment”, “variables”, “relational”, “comparator-i;ascii-numeric”, “spamtest”, “imap4flags”];

if address :contains “from” [“substack.com”, “NakedSecurity@sophos.com”, “news@cyberscoop.com”, “emails@sucuri.net”, “newsletter@flexjobs.com”, “schneier@schneier.com”, “list@wordfence.com”, “ivan@mail.notion.so”, “news@symless.com”] {

fileinto “Newsletters”;
return;

}

In this case, Sieve looks at the from header to see if it contains any of those emails in the array. If it does, then it gets filed into my Newsletters folder. If not, then it leaves it alone.

Pro tip: In my examples, the Newsletter and OSSEC folders have to exist. But you can tell Sieve to create a new folder by using this syntax instead:

if address “From” “someone@example.org” {
fileinto :create “Someone”; }

When Sieve leaves a message alone, that does not necessarily mean that the message will remain in my Inbox. You’ll notice in the screenshot of my filters list that there are three-bar handles to the left of each filter. I can grab those and move them around. The filters will operate in the order they are listed. If an email does not trigger a particular filter, then Sieve moves on to the next filter to see if it can find a match. If you have multiple filters and they’re not being filed as you expect, check if the order of your filters is the problem.

Advanced use of Sieve

These next examples come from the Tiger Technologies support page. I haven’t tried some of them, primarily because I don’t run my own mail server so I don’t have an environment to test properly. But these are all functions that Sieve does for many email providers across the globe.

Plus addressing

One of GMail’s most beloved features is the “plus address” feature. This is the feature that allows you to use something like newsletter+myname@gmail.com as an email address and that email will automatically end up in your myname@gmail.com inbox. Sieve can do that like so:

require [“envelope”, “fileinto”, “mailbox”, “subaddress”];

# file address+amazon@example.com into “Amazon” folder if envelope :detail “to” “amazon” {
fileinto :create “Amazon”; } # file address+ebay@example.com into “eBay” folder if envelope :detail “to” “ebay” {
fileinto :create “eBay”; }

I highly doubt that plus addressing would work on a hosted email account because the email has to arrive in your inbox to trigger any filters. If your email host has not configured Sieve like this on the global server configs, that email would be rejected and not sent to your inbox, so I think that trick is only for people who host their own email.

Rewriting subject lines

Some email providers mark messages as spam by injecting some text, such as SPAM into the subject line. This is done with a combination of technologies. First, some spam filter has to mark the email as spam. Usually, this is Spamassassin - that’s the most widely used spam filter I know of. The second step is to tell Sieve to look for Spamassassin headers and if it finds one, rewrite the subject. This can be done with a script like this which will inject the text [SPAM] into the subject of any email that Spamassassin has scored 7 or higher.

require [“editheader”, “variables”]; # store the original subject in a variable that later rules can use

if header :matches “Subject” “*” {
set “subject” “${1}”; }

if header :contains “X-Spam-Level” “*******” {
# delete the orginal subject...
deleteheader “Subject”;

# ... and replace it with the stored subject plus “[SPAM]”
addheader :last “Subject” “${subject} [SPAM]“; }

Filter an entire domain

You can write the same filters for an entire domain rather than just for a particular inbox. This is how an email provider can support things such as plus addressing for every domain. There is not extra magic to applying filters to the entire domain. You just need to write the rules into pre and post processing files with specific named and put them into your domain’s home directory. This will tell Sieve to run those filters against every incoming email.

domain-after.sieve and domain-before.sieve

Launched!

Sieve is a hugely deep topic and you can write some spectacularly impressive filters to achieve incredibly complex results. As with most purpose-built tools, the real power is in experimentation and experience. I’ve pointed you toward Sieve, and provided some basic examples of the format, but the rest of the fun is up to you. You can try out Sieve filters at Proton Mail but it will cost you – the free plan does not include filters. Or, if you host your own email server, you can just install the Sieve extensions for your email server and start experimenting.

my shorter content on the fediverse: https://the.mayhem.academy/@jdw

---