Junk Mail Classification Turned On in Trunk Builds

Wednesday November 13th, 2002

In a posting to netscape.public.mozilla.mail-news, Dan Mosedale writes, "At the end of last week, I checked in code to turn on basic junk mail classification features in the trunk build. There's still lots of work left to do, but it's in a state where folks should feel free to start playing with it, and filing bugs in Bugzilla. See for a short introduction to how it works, as well as credits (getting this up and running has been very much a group effort)."

#1 Filter question

by johann_p

Thursday November 14th, 2002 12:21 AM

It is a bit puzzling - there seems to be a still not activated option "move incoming messages ..." in the junk mail controls dialog, and you can use the condition "Junk status" is "Junk" in the filters dialog to define actions, like moving. This looks pretty redundand - what are the plans? It also seems that filtering based on the junk status does not work yet - I had no luck getting this to work.

#4 Re: Filter question

by rwall

Thursday November 14th, 2002 2:18 AM

Read the FAQ (the page is one click away from the introduction page linked to above):

Taken from that FAQ: <faq> * Why is the "Move incoming messages determined to be junk mail" option disabled?

There are still some known bugs in that code. We don't want anyone testing this feature to inadvertantly lose data. It will be turned on as soon as it's ready. </faq>

A rouch outline of "plans" are available as a TODO list which is also just one click away from the link above:

#9 Read my post, that was not my question

by johann_p

Thursday November 14th, 2002 6:58 AM

My primary question (which was zero clicks away from your answer) was about the seemingly redundant options of moving junk in the (still disabled) junk mail controls and the one based on the junk status in the mail filters. I could not find anything in the pointers you give that would indicated what the differences, intended use or plans are for that. My remark about what does not work is also NOT about the disabled move function in the junk controls the FAQ is about, but about the filter function that can move/label/etc emails based on the assigned junk status, which is something completely different and has nothing to do with the disabled junk mail controls (it is a filter function).

So please, before telling me to read more carefully it might be advisable to do it yourself.

#19 Processing Order?

by jm4n

Friday December 13th, 2002 10:46 PM

From what I gathered, reading the rough "todo" list, classification is done *after* the normal message filters. So a "Move to Folder" if "Junk Status" wouldn't work, since it wouldn't be classified until later.

That's my take anyway -- there was discussion on the todo page about redundant code in that area, and I think some of it may be merged together, so who knows. I'd like to see this merged tightly with the regular filtering personally, but I think they're going a different route with this.

#2 Sharing junk classifications

by bergie

Thursday November 14th, 2002 12:55 AM

What would be interesting next would be a framework for sharing junk mail classifications between systems. If we had a central repository, a company's uniformal junk classifications could be made available to everybody.

#3 I absolutely agree

by johann_p

Thursday November 14th, 2002 1:14 AM

There should be an interface to allow users to portably export and import pretrained classifiers. It should be possible to either reset the classifier and completely reinitialize with the regrained model or incrementally add imported models to the one already present (which is very easy to do with a naive bayes classifier)

#8 I don't agree...

by eiseli

Thursday November 14th, 2002 6:05 AM

Because if you've read the documentation, you will notice that it is very personal. It's statistical, so what might look spam to my inbox (given the mails I recieve) would not necessarily be junk in yours. Or the other way round: what looks like non-junk to my e-mail could look like junk to yours.

#10 Spam poetry

by jogar

Thursday November 14th, 2002 7:06 AM

Yes, you may miss out on terrific poetry fodder!

#11 Thats not the point

by johann_p

Thursday November 14th, 2002 7:08 AM

I did not request that anybody forces you to reuse somebody else's model, if your junk and work mails differ. I requested that one should be able to do so if the difference is not that big (e.g. for working colleages, family members etc.) and unlike the other person, you do not have a lot of archived email and spam to start the training with. From a statistical perspective this makes more sense than to force everybody to start training their naive bayesian model from scratch. Since the naive baesian classifier is incremental by design your later training instances will still capture the special personal aspect of your emails.

#12 ok, got your point ;-) n/t

by eiseli

Thursday November 14th, 2002 8:10 AM


#5 Great

by jonik

Thursday November 14th, 2002 2:59 AM

I did some testing and it seems that the Bayesian filter recognises spam pretty well, even with a very quick "training".

Looking forward to having the rest of the functionality turned on...

#6 View | Not Junk?

by sobaker

Thursday November 14th, 2002 3:21 AM

After some training it recognizes spam just great!

There are some problems..

1. if you move "recognized" message to another folder, it become marked as "?" again 2. from "Try out the view: "View | Not Junk". My mozilla doesn't support it, what's that? ;)

#7 Solved.

by sobaker

Thursday November 14th, 2002 3:40 AM

I've downloaded zip build and now views are working.

#13 Mozilla Mail Sucks

by erik

Thursday November 14th, 2002 11:38 AM

The new Bayesian junk mail filters have made me give Mozilla Mail another go. The reason is that I get way too much spam and yes the junk filter works excelent. But since my default browser is Phoenix and my default email client was Outlook Express (and no I'm not satisfied with OE) it is really a pain to use MM. The UI is unintuitive, the new mail rich text editor cripled and the context menus only repeat the toolbars when it should provide contextual information that is hidden deep in the menu structure.

This just makes me want Thunderbird even more. I just cant wait for a usable mail and news application that supports IMAP, working junk mail filters and an intuitive UI.

#20 Re: Mozilla Mail Sucks

by jm4n

Friday December 13th, 2002 10:52 PM

I gave mozilla mail a go a while back, and hated it. After getting tired of using Pine, and wanting a "real" email client on the windows box, I figured I'd give Moz mail another try (after all, Mozilla became my default web browser quite a while ago).

It has improved vastly. Sure, it has a lot to go, but for standard pop3 message handling it's great. I've been waiting for the bayesian filters since the idea was first announced on Slashdot, and am giving it a try today (no sooner than I installed the latest nightly did 1.3a show up... ugh!)

Things that still bug me:

- Newsgroup handling is horrible. Many, many UI issues, and a couple hanging issues - IMAP support is just as bad from what I've heard - I want to force TEXT-ONLY display of all messages. Yes, all. I hate HTML email :)

But bayesian filtering has me hooked for good now. I haven't used it enough to tell if it's a good implementation, but I trust it is (or will be).

#14 spamcop functionality

by punkrider

Thursday November 14th, 2002 12:25 PM

i would imagine it wouldn't be too hard to create a SpamCop junkmail auto-mailer plugin for MM? With a list of junkmail it would do a batch send to your specified SC address which could be filterd to come back to you for authorization filtered into another folder "SpamCop Auths" or something. I hate to just dump it and go. they should pay for that shit.

#15 Plans to get the statistics visible?

by explo

Thursday November 14th, 2002 3:44 PM

I'm wondering if there are plans to offer user (optionally) the detailed statistics about the analysis of the current mail message. Not that it's incredibly neccessary feature, but some people (including me ;) are curious.


by SubtleRebel

Friday November 15th, 2002 1:05 PM

Why has the checkin of this type of Spam Filtering caused bug 71413 to be marked as WONTFIX?

The purpose of that bug was to enable a user to right click on incoming mail address and select "Block this Address" ( ).

I do not see how this Bayesian filter replaces the feature requested in that bug. If I classify a specific email as junk, does all mail from that sender then automatically get classified as junk too?

#17 Re: WONTFIX?

by higuita

Tuesday November 26th, 2002 12:21 PM

if you want a 100% correct block, the bayesian filter may have to get some more spam to finally learn that ALL emails from that guy are spam... but if you want, you can create a filter rule that moves that emails to the trash and mark then as spam... this way you are teaching the filter and blocking that spam... after a while, even if the guy changes the email, the email rule will break, but at that time the bayesian filter may already have learned that those emails are always spam

#18 So it should not be WONTFIX

by the_Rebel

Wednesday November 27th, 2002 12:31 PM

It is as I thought then. Inclusion of the Bayesian filter does not eliminate the need for a simple means of setting up a filter to block mail from a specific address.