Trending
Spam filters are invisible until they fail, and then they are very visible: a phishing message sitting in your inbox, or a client's invoice buried in the junk folder. Building one yourself is a good first machine learning project, because the problem is small, the data is text you already understand, and you can get a working model in an afternoon. Here is the whole path, from raw messages to a function that takes an email and returns a verdict.
Strip away the branding and every spam filter does the same thing: it takes a piece of text, converts that text into numbers a model can handle, and outputs a probability that the message is unwanted. Everything else — server-side rules, report buttons, sender reputation — is engineering wrapped around that one prediction.
For a first version, keep it binary: spam or ham, the usual name for "not spam". That means three pieces:
You need labeled examples: messages that already carry a spam or ham tag. Public spam text collections exist and are fine for practice, but exporting a few thousand messages from your own inbox makes the project feel real — just be thoughtful about what you paste into a script, and redact anything personal. A shared or work mailbox may have rules about exporting mail, so check before you do it.
Whichever route you take, check four things before training:
That last point is the one beginners skip. Build your vocabulary from every message and then evaluate, and your test set has quietly influenced the model. The accuracy you get back is fiction.
Lowercasing everything is standard and safe. Beyond that, be careful. It is tempting to strip punctuation and digits, but spam signals often live exactly there: the exclamation marks, the currency amounts, the all-caps subject line, the phone number in the body. Delete them and you make the model's job harder.
A sensible pass looks like this: lowercase, replace URLs with a single placeholder token such as "urltoken" so the model learns "contains a link" instead of memorizing domains, and leave the rest alone. Skip stopword removal. Words like free, now and win are among the most useful features you have, and in a bag-of-words model the extra noise costs you very little.
Real spam writes "V1AGRA" and "cheap meds" on purpose. A word-level model will miss the first one entirely. Character n-grams fix that, and they're a small change once you want to test the improvement.
The simplest representation is a bag of words: a vocabulary of every token in your training set, and one vector per message counting how often each token appears. Word order is discarded, which sounds reckless and works surprisingly well here.
Most people then reach for TF-IDF, for good reason. A word appearing in nearly every message — please, thanks, the — carries little information. A word appearing in a handful of spam messages carries a lot. TF-IDF down-weights the common terms and up-weights the distinctive ones automatically, so you get a stronger feature set without hand-tuning a stopword list.
Two settings are worth knowing. N-grams capture short phrases like "act now" or "click here" instead of isolated words. Sublinear scaling stops one word repeated forty times from dominating a message. Both are usually a single argument in your vectorizer.
Fit a multinomial Naive Bayes model on the vectorized training text, predict on the test set, and compare against the true labels. You will probably see a high accuracy score and feel good about it. Hold that thought.
If most of your messages are legitimate, a model that answers "ham" every single time scores well and is completely useless. Look at the confusion matrix instead: the four counts of what the model got right and wrong for each class. That table tells you whether you built a classifier or a broken clock.
For a spam filter, a false positive — a real message marked spam — is usually worse than a false negative. Someone's password reset or job offer ends up in the junk folder and is never seen. Precision on the spam class is the number to watch. If it's too low, you have a lever: raise the probability threshold so the model only calls something spam when it is genuinely confident.
A model in a notebook is a demo. Make it a tool with a small function: take a string, run the same cleaning and vectorizing steps used in training, return a label plus a confidence score. Print both. A 51 percent spam call and a 98 percent spam call are not the same event, and seeing the probability changes how much you trust the output.
One gotcha catches nearly everyone: save the fitted vectorizer alongside the model, using joblib or a similar serializer. Reload a model without its matching vocabulary and the feature columns won't line up — predictions come back as nonsense. Save them as a pair, load them as a pair.
Keep a held-out set of messages you never touch during development, and run the pipeline against it whenever you change something. Log the messages your filter gets wrong. A folder of misfires is the most valuable data you will ever collect. Retrain when your traffic shifts, because spam changes faster than your model does.
When you want to push further, try these in order: add phrase n-grams, swap in logistic regression or a linear support vector machine and compare precision, add character n-grams for obfuscated words, and tune the threshold on a validation set instead of guessing. Each change is small, and you'll know whether it helped, because your held-out numbers will say so.
The first version takes an evening. The second version — the one you actually trust — is built on the mistakes you logged from the first.
Photo: qimono / Pixabay
