You didn't (and still don't, most likely) have it setup properly - michaeltsai

Please read my reply to your earlier comment. The reason you think SpamSieve's reported accuracy is weighted is that you haven't setup your mail program properly. It's hiding some of the messages from SpamSieve, so that (a) SpamSieve can't catch them and (b) they don't appear in SpamSieve's statistics.

Reply to This

Sunday, February 24 2008 @ 04:30 PM PST

Also - michaeltsai

If you're having trouble setting up SpamSieve, please contact SpamSieve support. You'll probably save yourself a lot of time and frustration. If setup properly, it should be easy to get 96% accuracy on the first day.

Reply to This

Sunday, February 24 2008 @ 04:42 PM PST

You didn't (and still don't, most likely) have it setup properly - rskinner

Thank you for your concern. I read your earlier comment, Micheal. I didn't want to engage in argument until I gave your software a fair shake, so I did not reply.

I'm responding now because I stand by my previous two evaluations. Other potential users can read both of our comments and judge for themselves whether or not to use your software.

My earlier comments stated that the report provided by Spamsieve does not accurately report what Spamsieve can "do for me." Counting the incoming email that is "whitelisted" gives the illusion of some kind of superior accuracy. Actually, well over 90% of my incoming email is from senders who are already in my address book, so for 90+% of my email, Spamsieve offers nothing that cannot be done with the addition of a simple rule. For the first 90% of good email, Spamsieve is not helping me. It is the small percentage of email from unknown users that is the true challenge for any anti-spam software.

To determine accuracy, I choose to divide the actual number of spam emails by the number of "Spamsieve errors" that I am forced to handle manually. This gives me a better idea of whether Spamsieve is actually helping me or hurting me.

For the first couple weeks, Spamsieve was making about one error for every three incoming spams. After one month, it's apparently learned something because it's making only one error in twenty-five incoming spams.

I consider that very good, especially considering that those 25 spams were missed by the filter on my server. THAT is why I sent you the $30 yesterday.

Here are some NEW thoughts I have on the matter:

1. It's nice that you are replying here. It shows that you are an author who cares about his product and does not want it to stagnate.

2. By insisting that I've made an error in setting up your software, you're giving potential users the idea that it's in some way complicated and difficult to use.

Because I LIKE your software and want you to sell a LOT of it so you will be motivated to keep it up to date, I would like to make it clear that the software is EASY to set up properly. Honestly! You make a new rule and that's the end of it.

So there you have it. Personally, I don't know what the beef is. The numbers that I gave you (67% after two weeks, 96% after four weeks) are accurate. Spamsieve reports 99.4%. That's accurate, too. We're just counting different things.

R.

Reply to This

Sunday, February 24 2008 @ 10:50 PM PST

Reply - michaeltsai

It seems that I misunderstood your original post. Regarding (2), most users don't have any trouble setting up SpamSieve. But the ones who do generally get it partially correct, and that's worse than getting it wrong. For example, if they put the SpamSieve rule at the bottom of the rule list instead of the top, it will filter some of the spams, but some will be hidden from SpamSieve. Since it seems to be working a little bit, they think the setup is correct and incorrectly conclude that SpamSieve just isn't very accurate. I apologize for mistakenly thinking you were in this group. I'm rather touchy on that subject, because some of these people have posted negative reviews on VersionTracker, without ever contacting technical support or reading the FAQ, and there's no way for me to contact them.

When choosing a single accuracy number, I still think the SpamSieve one (overall percent correct) makes the most sense, because it also takes into account the number of false positives. It would be trivial to catch all the spam if you let good messages also be marked as spam. Secondly, the whitelist accuracy is not exactly free, because sometimes spammers forge addresses from people on your whitelist. If SpamSieve lets through a spam because it matched the whitelist, that counts as a false negative, so letting a good message through because of the whitelist should count for something. (Also, some users will have lots of good messages that don't match the whitelist, so there needs to be some way to count those.)

All that said, your preferred number, the percentage of spam caught, is certainly a valid metric. I may add it to SpamSieve's statistics window. In your case, this alternate metric can be very different from what SpamSieve reports, because 90% of your mail is good (due to your server filtering out the obvious spams). With more than 80% of the mail on the Internet being spam, for the average user there is probably less difference between the two metrics. In any case, for someone who's really interested in the details of the filtering, I think the answer is to provide a variety of accuracy statistics for the different filters (whitelist, blocklist, Bayesian, etc.) and to show how often each filter was used. SpamSieve tracks this information, but there isn't yet a user interface to display it.

Even accounting for your different accuracy metric, I think the initial 67% accuracy that you saw after two weeks is atypically low. The learning should be much faster than that. In your case, it's moot since the accuracy has since gone up, but if anyone reading this is in a similar situation, please don't wait for weeks before contacting support. There's probably something we can do to help you.

Lastly, thanks for buying SpamSieve and thanks for taking the time to reply.

Reply to This

Monday, February 25 2008 @ 07:03 AM PST