Nov 16 2007
A Filter for Stupidity?
Nov 16, ‘07 — Too long have we suffered in silence under the tyranny of idiocy. In the beginning, the internet was a place where one could communicate intelligently with similarly erudite people.
Then, Eternal September hit and we were lost in the noise. The advent of user-driven web content has compounded the matter yet further, straining our tolerance to the breaking point. It’s time to fight back. Says StupidFilter Project.
What is StupidFilter Project
The solution we’re creating is simple: an open-source filter software that can detect rampant stupidity in written English. This will be accomplished with weighted Bayesian or similar analysis and some rules-based processing, similar to spam detection engines. The primary challenge inherent in our task is that stupidity is not a binary distinction, but rather a matter of degree. To this end, we’re collecting a ranked corpus of stupid text, gleaned from user comments on public websites and ranked on a five-point scale.
Eventually, once the research is completed, we plan to release core engine source code for incorporation into content management systems, blogs, wikis and the like. Additionally, we plan to develop a fully implemented Firefox plugin and a Wordpress plugin.
Project Status
This project is currently in the design and analysis phase. We’ve gathered a fairly large (225K+ comments) database of comments, primarily from Youtube, that ever-inspiring font of stupidity. We’ve implemented a web-based comment ranking system to seed our stupidity corpus and that’s proceeding nicely.
Moderator applications are now open and we’re going through them as quickly as possible. We’re testing CRM114 as a classification platform, initial tests with the bit entropy and correlative classifiers are pretty promising. Additionally, we’ve moved to a new dedicated server better suited to the heavy database work we’re doing. We’re still on track for a late December alpha code release date. StupidFilter.
