PDA

View Full Version : Violator Part 2 - HEY DUMMIES! Meet "Margaret"



BadCat
08-29-2010, 04:15 PM
I've gotten most of the second part of Violator done...just need to do some graphical output and clean things up a bit. This is the real reason I changed the name of DUmpDiver to Violator. It "violates" the DUmmies a lot more.

See, Violator is a "feeder application" for Margaret. As it counts the DUmmies, it reads, stores and indexes EVERY NAUSEATING WORD they type.

The purpose of doing this is to enable psycholinguistic analysis of the stinking commies that post at the DUmp.

Based on the work of Dr. J.W. Pennebaker, the DUmmy posts are analyzed against the Linguistic Inquiry Word Count (LIWC)... http://www.liwc.net/liwcdescription.php

LIWC is a dictionary of words that are categorized by psychological meaning. Pennebaker et al have proven conclusively that word use frequency in a particular category is indicative of overall psychological state.

Here is an example of output from Margaret (sorry it's in XML)...


<forum name="GD">
<category name="anger" terms="121" termsUsed="79372" score="1.69882335314923" />
<category name="comm" terms="127" termsUsed="134292" score="2.87429302198655" />
<category name="death" terms="29" termsUsed="10884" score="0.23295360297934" />
<category name="i" terms="9" termsUsed="38160" score="0.816750228747853" />
<category name="negemo" terms="345" termsUsed="179800" score="3.8483147570457" />
<category name="optim" terms="70" termsUsed="61079" score="1.30729264207783" />
<category name="posemo" terms="265" termsUsed="209241" score="4.47844954437708" />
<category name="posfeel" terms="43" termsUsed="31705" score="0.678591876374494" />
<category name="sad" terms="72" termsUsed="32320" score="0.691754910721452" />
<category name="sexual" terms="49" termsUsed="20159" score="0.431469283577777" />
<category name="swear" terms="32" termsUsed="27159" score="0.581292438746408" />
<category name="we" terms="11" termsUsed="24948" score="0.533969725021002" />
</forum>

This is based on a one day Violator scan covering yesterday (the Glen Beck gathering in DC).

I am analyzing their posts in 12 categories:
anger
comm (communication words)
death
i (talking about themselves in first person)
negemo (negative emotion words)
optim (optimistic words)
posemo (positive emotion words)
posfeel (positive feeling words)
sad
sexual
swear (dirty words)
we (collective)

This line

<category name="anger" terms="121" termsUsed="79372" score="1.69882335314923" />

says "In the anger category, there are 121 terms, there were 79372 usages of these 121 terms. The LIWC score is 1.698..."

The LIWC score is a percentage of the total number of terms divided by the number of times the category terms (like anger) were used. The higher the number, the higher the psychological category.

BadCat
08-29-2010, 04:18 PM
So without further delay...here are the results from Margaret for the one day scan from Sat 8/28 to Fri 8/27 at around 2030.


<?xml version="1.0" encoding="utf-8"?>
<LIWCSummary date="8/29/2010 3:44:06 PM">
<forum name="EDT">
<category name="anger" terms="121" termsUsed="4571" score="1.06787588249861" />
<category name="comm" terms="127" termsUsed="10358" score="2.41983338239348" />
<category name="death" terms="29" termsUsed="1066" score="0.249038654724025" />
<category name="i" terms="9" termsUsed="1797" score="0.419814692813389" />
<category name="negemo" terms="345" termsUsed="14716" score="3.43794825789752" />
<category name="optim" terms="70" termsUsed="5393" score="1.25991131794246" />
<category name="posemo" terms="265" termsUsed="15247" score="3.56200034575723" />
<category name="posfeel" terms="43" termsUsed="2175" score="0.508122958747424" />
<category name="sad" terms="72" termsUsed="3822" score="0.892894688888577" />
<category name="sexual" terms="49" termsUsed="1117" score="0.260953262032585" />
<category name="swear" terms="32" termsUsed="837" score="0.195539731711078" />
<category name="we" terms="11" termsUsed="1590" score="0.371455404325703" />
</forum>
<forum name="GBLT">
<category name="anger" terms="121" termsUsed="140" score="1.21612230715775" />
<category name="comm" terms="127" termsUsed="410" score="3.56150104239055" />
<category name="death" terms="29" termsUsed="22" score="0.191104933981932" />
<category name="i" terms="9" termsUsed="154" score="1.33773453787352" />
<category name="negemo" terms="345" termsUsed="327" score="2.84051424600417" />
<category name="optim" terms="70" termsUsed="217" score="1.88498957609451" />
<category name="posemo" terms="265" termsUsed="617" score="5.35962473940236" />
<category name="posfeel" terms="43" termsUsed="146" score="1.26824183460737" />
<category name="sad" terms="72" termsUsed="87" score="0.755733148019458" />
<category name="sexual" terms="49" termsUsed="152" score="1.32036136205698" />
<category name="swear" terms="32" termsUsed="26" score="0.22585128561501" />
<category name="we" terms="11" termsUsed="139" score="1.20743571924948" />
</forum>
<forum name="GD">
<category name="anger" terms="121" termsUsed="79372" score="1.69882335314923" />
<category name="comm" terms="127" termsUsed="134292" score="2.87429302198655" />
<category name="death" terms="29" termsUsed="10884" score="0.23295360297934" />
<category name="i" terms="9" termsUsed="38160" score="0.816750228747853" />
<category name="negemo" terms="345" termsUsed="179800" score="3.8483147570457" />
<category name="optim" terms="70" termsUsed="61079" score="1.30729264207783" />
<category name="posemo" terms="265" termsUsed="209241" score="4.47844954437708" />
<category name="posfeel" terms="43" termsUsed="31705" score="0.678591876374494" />
<category name="sad" terms="72" termsUsed="32320" score="0.691754910721452" />
<category name="sexual" terms="49" termsUsed="20159" score="0.431469283577777" />
<category name="swear" terms="32" termsUsed="27159" score="0.581292438746408" />
<category name="we" terms="11" termsUsed="24948" score="0.533969725021002" />
</forum>
<forum name="GDP">
<category name="anger" terms="121" termsUsed="11808" score="1.68608154534628" />
<category name="comm" terms="127" termsUsed="17746" score="2.53397722761815" />
<category name="death" terms="29" termsUsed="1234" score="0.176204660142049" />
<category name="i" terms="9" termsUsed="4932" score="0.704247474733051" />
<category name="negemo" terms="345" termsUsed="27230" score="3.88821142274554" />
<category name="optim" terms="70" termsUsed="11210" score="1.60069225299219" />
<category name="posemo" terms="265" termsUsed="32084" score="4.58132116369327" />
<category name="posfeel" terms="43" termsUsed="4269" score="0.609576737557866" />
<category name="sad" terms="72" termsUsed="4848" score="0.692252992195019" />
<category name="sexual" terms="49" termsUsed="2623" score="0.374541996395944" />
<category name="swear" terms="32" termsUsed="3917" score="0.559314144065159" />
<category name="we" terms="11" termsUsed="3620" score="0.516905080805687" />
</forum>
<forum name="LBN">
<category name="anger" terms="121" termsUsed="10503" score="1.49378124488882" />
<category name="comm" terms="127" termsUsed="18338" score="2.60810820420557" />
<category name="death" terms="29" termsUsed="1769" score="0.251594689346693" />
<category name="i" terms="9" termsUsed="5103" score="0.725770322066803" />
<category name="negemo" terms="345" termsUsed="22938" score="3.26233973105395" />
<category name="optim" terms="70" termsUsed="7598" score="1.08061981325957" />
<category name="posemo" terms="265" termsUsed="27195" score="3.86778834187864" />
<category name="posfeel" terms="43" termsUsed="4690" score="0.667031708895415" />
<category name="sad" terms="72" termsUsed="3741" score="0.532060900421695" />
<category name="sexual" terms="49" termsUsed="2380" score="0.338493703021554" />
<category name="swear" terms="32" termsUsed="3004" score="0.427241631880987" />
<category name="we" terms="11" termsUsed="3862" score="0.549270034062707" />
</forum>
<forum name="LNG">
<category name="anger" terms="121" termsUsed="8268" score="1.26253487317388" />
<category name="comm" terms="127" termsUsed="17953" score="2.74144757838543" />
<category name="death" terms="29" termsUsed="1198" score="0.182936233437628" />
<category name="i" terms="9" termsUsed="9757" score="1.48990720338142" />
<category name="negemo" terms="345" termsUsed="22669" score="3.46158720851218" />
<category name="optim" terms="70" termsUsed="8019" score="1.22451223366973" />
<category name="posemo" terms="265" termsUsed="35856" score="5.47526008859733" />
<category name="posfeel" terms="43" termsUsed="6971" score="1.06448120475268" />
<category name="sad" terms="72" termsUsed="2955" score="0.451232529055252" />
<category name="sexual" terms="49" termsUsed="3923" score="0.599047448894671" />
<category name="swear" terms="32" termsUsed="4306" score="0.657532071103863" />
<category name="we" terms="11" termsUsed="3215" score="0.490934883557575" />
</forum>
</LIWCSummary>

Note the date in the Margaret output reflects the time I ran Margaret, not the time of the scan. I'll fix that soon.

BadCat
08-29-2010, 04:24 PM
Note how high the anger scores are for GD and GDP compared to the rest of the scanned fora. Not even the fags in GLBT are that angry...


<forum name="GD">
<category name="anger" terms="121" termsUsed="79372" score="1.69882335314923" />
<forum name="GDP">
<category name="anger" terms="121" termsUsed="11808" score="1.68608154534628" />
<forum name="GBLT">
<category name="anger" terms="121" termsUsed="140" score="1.21612230715775" />

I guess Beck REALLY got their panties in a bunch yesterday.

SarasotaRepub
08-29-2010, 07:42 PM
Too Funny...I bet the DUmmy Mgt. is pissed you can do this to them. :D

Well done!!!

BadCat
08-29-2010, 07:44 PM
Too Funny...I bet the DUmmy Mgt. is pissed you can do this to them. :D

Well done!!!

Well you know I always say "If you don't want everybody to see what you're doing, don't put it on the damn internet"

BadCat
08-30-2010, 05:52 PM
These are the categories and words (terms) used in LIWC...

http://www.yoshikoder.org/code/yoshikoder/dictionaries/LIWC.ykd

A word with an "*" in it is a stem word.

For instance,the entry

<pnode name="accomplish*"/>

Will count "accomplish" "accomplished" "accomplishment"...etc.

If you're looking at the list using Firefox or IE, clicking on the little "-" signs next to a <cnode> will collapse the node into the psychological category.

Clicking on the "+" next to a <cnode> will show the words in that psychological category that Margaret counts.

ralph wiggum
08-30-2010, 06:00 PM
This is hilarious, BC. Nice work.

PoliCon
08-30-2010, 06:06 PM
someone has some demented hobbies. :D good work. :D

BadCat
08-30-2010, 06:09 PM
someone has some demented hobbies. :D good work. :D

Not a hobby.

I do this for a living.

malloc
08-30-2010, 06:37 PM
Not a hobby.

I do this for a living.

What language did you write this in? (Just curious, I do my share of the coding for a living).

Also, is there anyway you can Violator 2/Margret on the CU for comparison?

PoliCon
08-30-2010, 06:39 PM
Not a hobby.

I do this for a living.

tracking idiots on DU is what you do for a living? ;)

BadCat
08-30-2010, 08:37 PM
What language did you write this in? (Just curious, I do my share of the coding for a living).

Also, is there anyway you can Violator 2/Margret on the CU for comparison?

C#.
Margaret uses Lucene.Net.

I'd have to write a plug in to do CU, and I'm not real interested in doing that. I'm going to do one for the Kossacks and the Huffpuffers.

If you want to play with some CU threads, you can cut and paste posts to this site...

http://www.liwc.net/liwcresearch07.php


It won't show as many categories as I'm running though.