Over the weekend I made another script based on the Digg API: Digg CommentSpy
This one tracks live comments posted to Digg, ala LiveMarks. Blue posts are replies to the top thread, yellow are replies to another comment.
I’m now a huge fan of JSON feed based APIs. These scripts have 0 server code, all the work is done in Javascript. This means that scaling the mashup is all on the remote service, not on my puny server.
If you’re using my Digg javascript function, commentspy uses a slightly revised version that allows for ‘min timestamp’ – I use this so that I don’t ask for already seen comments.
I’ve written another of my little javascript web apps, this time based on Digg’s new expansive API.
The API is very well designed and I’ve only found a couple problems with it: the license is very restrictive and getting back some info requires many multiple calls to digg, but I guess it points to the problems with web apis as just being poor proxies for direct database calls.
The Digg application is a friends browser. – It basically asks the question: “Who’s digging you”.
I wrote this script because i’ve noticed that the front page of Digg is guarded by social networks of people who digg each other’s stories. If you want to get your story noticed, you want to have digg friends.
Enter any top digger name, you will see a high ratio of repeat diggers – the same is true of my submissions as well (Thanks Pawfoots!).
As I wrote it, I was interested really in just browsing around the networks, so I thought that was fun enough for an app. It’s kind of like an earlier script I wrote: delimages.
Anyways my app started with a single function basically that gives you back a digg response object with what you want. If you write a Digg script, you can use my little Digg function:
var Digg =
{
apikey : 'http://sandbox.sourcelabs.com/'
};
Digg.request = function (path, count, cbkName, offset)
{
var url = 'http://services.digg.com/' + path;
if (!offset) offset = 0;
var params = $H(
{
'count' : count,
'type' : 'javascript',
'appkey' : Digg.apikey,
'callback' : cbkName,
'offset' : offset
});
url += '?' + params.toQueryString();
var head = document.getElementsByTagName('head')[0];
var request = document.createElement('script');
request.type = 'text/javascript';
request.src = url;
head.appendChild(request);
}
All of these JSON Apis are really hurting my Javascript OOP sensibilities. XMLHttpRequest is cool with you making an anonymous function on the fly as the callback, but with JSON the best approach probably is having a callback name passed in the object return, which isn’t the cleanest.
I tried to make do in my script as best I could though.
Recently I’ve posted about my stories being disappeared from my submitted stories to Digg list.
I noticed another odd thing today:
It’s 298 days old and it must be hard to do bug testing on the zillions of stories they have in their database, but to me this implies they are doing some weird monkeying with their data or the way diggs are displayed.
It’s kind of a funny thing, other sites would kill for the kind of user contribution that Digg receives on an hourly basis. People are pretty devoted to the site. One thing I’ve been noticing recently is that some stories are being smuggled off into the night by Digg’s secret police.
One story I submitted the other day was a creative commons account of a switcher from Kubuntu to OSX. I, like a lot of other Apple converts, am interested in genuine switcher stories and I thought Digg’s Apple friendly crowd would be too. The story was pretty successful even, drawing in a digg every 5 minutes are so, and it soon made the homepage with 35 votes.
But pretty soon my story is gone. Gone from my submitted stories even. It’s still in my Digg Homepage stories
Somewhere I imagine my poor story doing hard labor in a Siberian work camp.
Anyways, I posted the full text of the article to SWiK (creative commons ftw) and pretty soon there were interesting comments on there.
I don’t really mind Digg’s longstanding policy of stealth banning stories. If you can’t play nicely with the group, go play by yourself, that seems like a good compromise. We use stealth bans on SWiK too to try and remove feedback from spammers. But pulling it out of my submitted stories altogether seems like overreaching. That’s my area. Don’t touch that, especially don’t kill good faith stuff I put in there.
Chrisek, a top 10 digger recently went through the same thing, posting info about a new web2.0 tagging site called ‘wis.dm’ – now buried and gone from his submissions too.
Eventually we’ll add a ‘democratic’ way to feature interesting projects on SWiK. Normally I’d study Digg as a model for how a great community site can function, but in this case I think they just annoy valuable submitters.
Ars Technica has an article published yesterday that echoes something I’ve said earlier in The Prisoner’s Dilemma in Digg Story Promotion
The problem has only got more obvious – I’ve heard from various sources that the front page isn’t what it used to be, there is a growing issue with dupes, and the social networks that work together to promote their submitted stories will only get stronger as time goes on if there is no change to the system.
All of these questions are in the background of this story: Digg – who will buy Digg (a story that was buried, can’t see it on the index anymore)
My thoughts on this are that Digg should get back to basics, come up with a new revision that changes the game, and not sell the company – I thought it was a mistake for del.icio.us and I think it’s a mistake for Digg, simply because it’s clear that social software is the only area making noticeable gains in helping people find the information out there.
Mass-digg everything you can see on a page. This can be useful for digging all your friend’s submissions’ at once for example.
ok I thought this should have been on digg, but as is the case most of the time, no one agreed with me:
The Virgin Mary Appears in Second Life
I couldn’t bring myself to add an exclamation point, even though I don’t think I’ve ever seen a single point in time when Digg didn’t have a front page story with an exclamation mark.
But I think it’s cool and I feel like posting it, so here’s the miracle of miracles:

See it? It’s the Virgin Mary!!!! Randomly rendered in a texture! It can be yours for only $30,000 Linden Dollars! ($100 US, must be a lot of inflation in SL)
Second Life in general is doing pretty well despite an aging graphics engine, check out this Google Trends graph of searches for Second Life:
Of course compare this to WoW:
What is the Prisoner’s Dilemma?
Here’s a description from w9a
In game theory, the prisoner’s dilemma is a type of non-zero-sum game in which two players can “cooperate” with or “betray” the other player. In this game, as in all game theory, the only concern of each individual player (“prisoner”) is maximizing their own payoff, without any concern for the other player’s payoff per se. In the classic form of this game, Cooperating is strictly dominated by defecting (i.e., betraying one’s partner), so that the only possible equilibrium for the game is for all players to defect. In simpler terms, no matter what the other player does, one player will always gain a greater payoff by playing defect. Since in any situation playing defect is more beneficial than cooperating, all rational players will play defect.
How does it apply to Digg?
I have long thought about a way to game digg wherein minority voting blocs would approximate the appearance of majority appeal for stories. This tactic though has and is in play currently on digg, with voting blocs ranging from rivals at AOL to Republican / Libertarian online campaigners.
Recently though I took notice of another phenomena that seems to occurs on digg. The tactic is a tit-for-tat strategy for gaining digg front-page promotions.
The way tit-for-tat works in the classic Prisoner’s Dilemna scenario is that even though in one game of prisoner, it would make sense to defect; in an unknown number of games, it’s better to cooperate with other people by trusting them at first and then ceasing to trust them if they betray the trust until they start cooperating again. The tit for tat strategy works so well, all things being equal, it’s never been beaten in computerized simulations of various tactics.
On digg it works through the friends system. The game is that friends are assumed to trust you by digging your stories. You can trust them back by digging their stories. I noticed recently that when I submit stories, some people who have befriended me always digg them. I didn’t even really notice that I had so many friends until I noticed this tab with my 39 friends, most of whom I don’t know at all. The word friends is kind of confusing, because my real friends are all lurkers and don’t bother interacting with digg, or don’t have the time to read digg regularly enough for the friends feature to be interesting, (aside from Kevin Rose for obvious reasons).
Thus friends seem to really be tit-for-tat partners, waiting for you to engage and become their ally in the quest to get digg submissions. A good way to spot a tit-for-tat player is to look for a digg user with a large number of one sided friendships where they are friends with many people and not many return the favor, such as my new tit-for-tat friend chrisek with 253 friends and 75 reciprocating.
I like this theory because it seems that as digg expands, the ecosystem surrounding it must constantly evolve to adapt, and this is an interesting way it is evolving.
Revised tit for tat strategy description to be more accurate.
Here’s a bookmarklet to submit a new story to digg.
(moved from top level page—alex)
This is for the digg script that you wrote.
Hi Alex,
Great tool.
I use wordpress on my website and I don’t think this script would go very well with wordpress. The reason is that I can’t put this in the source files and I will have to use this with the data. Also, WP doesn’t allow you to use javascript with the content.
All that said, there is a plugin that lets you create javascript blocks which could be added to the content.
Is there anyway you could make this script work dynamically. Like, one single script could work for all the posts.
Abhinav Kaiser
http://technopedia.info/tech
abhinavkaiser at gmail dot com
I’ve been playing around with another weekend project, this time what I set out to build is something that will give me a ‘Digg This’ button that I can put on my website.
If you live under a rock and haven’t kept up with my blog posts on Digg, here’s what it’s about.
Let me explain what I mean by a digg button, after all there are ‘digg this’ generators out there already.
But all they do is put a little text link on my site that makes people go off to digg.com if they want to vote for my article.
That’s not what I want. I want the real McCoy, I want the actual real live digg button on my page. I want it to show the count of diggs. I want it to be clickable, and I definitely want to see the vote animation like on Digg.com.
I thought this might be a cool little javascript I could let other people use too, so I whipped up a simple little generator: you can make your own digg this buttons here.
There’s a really interesting story cropping up – some users of Digg are peeking behind the curtain of the mechanism that runs Digg.com.
Specifically a blog called ForeverGeek is outraged that Digg.com might employ any editorial control, when it clearly states their mission is to be a community edited news site.
The whole thing is completely overblown, but it points to the fact that communities feel they have squatter’s rights: wherever they are, they have some part ownership of that property.
I’ve written about Digg before, about the social dynamic of the site and how the community brings out a steady stream of interesting news.
What I didn’t mention however is that the hidden Digg algorithms makes a lot of this magic happen, and it’s a for-profit service, so Digg.com is strongly motivated to keep a high level of service, and not necessarily motivated to make the community part of running the backend service.
As Joshua Schacter has pointed out, there’s a strong negative to allowing users to make community features their home, because things quickly get out of control, it’s much harder to make a utopian society than it is to make a useful tool.
The issue with Digg is that it appears as though there is some hidden editorial control, and some overt editorial control, behind what stories get buried and what stories get promoted.
If you actually look at the mechanism that promotes stories, that by itself is an editorial mechanism—there is no published rule about how many diggs are required to get to the front page, and in fact it’s not that straightforward and Digg.com changes it on a regular basis.
And Digg bans sites explicitly from being submitted, as ForeverGeek discovered, which is likely a big part of their outrage.
The funny thing is that although the story is being treated as a revelation, or of Digg’s selling out to the man nothing much has changed. Digg.com has always used editorial control over the story submission process, it just emphasizes community because the process is driven by the community in a very large part.
Also, Digg is now a huge website, having to deal with a torrent user contributions and filter out spam and people trying to game it, all while remaining snappy and easy to use. Behind this lies a lot of database servers, and code to tie them all together, although the site is still basically simple and generally robust, some errors do occur from time to time.
It will be interesting to see how Digg braves this balance of providing the service that is expected to the 95% of visitors who don’t really know or care about the details behind the digg queue and promotion mechanism and the 5% who care a lot and make venomous posts about perceived injustices in it.
It’s been reported before that Digg.com has surpassed Slashdot on Alexa, but never before in the main ‘reach’ metric—for the first time Digg beat Slashdot the other day.

Slashdot is a good example of why you should probably not rest on your laurels for years and years.

If you look a little deeper into these stats though, it’s clear that Digg’s audience isn’t completely comprised of Slashdot defectors, they have their own audience that’s somewhat distinct, and the style of the stories are different enough that there is room enough for both.
As great as it looks for Digg right now though, there are secondary statistics to reach and page views, like number of people ‘digging’, ranking comments, etc – the metric for how well these are doing is more obscure.
Looking at the graphs for del.icio.us and its biggest followers, as I’ve just posted below, it’s interesting to note that growth for this year is fairly static.

I would have thought that Yahoo would be providing more support for del.icio.us, wasn’t one of the big reasons to go to a big web company to help take social bookmarking to the mainstream?
Digg however is still growing, thanks possibly in part to Kevin Rose’s rush to get features out the door like their much improved comment system, and possibly part of growing to fill the unrealized potential size of tech enthusiast market, digg continues to be first with tech news over everyone else, more now than ever.
The next stage of the social bookmarking game has got to be making services that deliver more value to the mainstream web user and refinining the anti-spam and social heuristics. (LiveMarks unfortunately saw a lot of spam today through del.icio.us user sexkitten], who posted seven and a half thousand spam bookmarks in the space of ten hours.
I have to note however that Alexa has been most unkind to SWiK lately, completely unjustifiably, we’ve just had our best week ever according to our internal stats but Alexa shows us down, which is kind of irritating.
Our user base isn’t well tuned for Alexa, having a third of the users not running Windows and fewer than a third running IE, but it’s still annoying to see our graph go down when we are actually attracting quite a large audience.
Social bookmarking is currently in a high growth pattern. I notice it all the time running LiveMarks, the rate of social bookmarking is increasing every single day.
The more people flock to something, the bigger target it is for abuse. The more people turn to del.icio.us to find useful bookmarks, or digg.com to read the latest news, the more tempting targets those services become for spammers and vandals.
As Clay Shirky has noted, “Social software is stuff that gets spammed.” Perhaps as an addendum to that aphorism it should be noted that successful social services are those that can resist spam successfully.
The spamming foes of social software are just starting to take shape. Witness a new service called ‘TagExplosion’, TagExplosion describes itself:
“[TagExplosion] helps get your program, blog, advertisement or website on the several “Social Bookmarking” sites so you can get on 100,000’s of computers, worldwide. This will help drive more traffic to your website by utilizing the “word-of-mouth” aspect of Social Bookmarking.
Services like these are bound to multiply and exploit every weakness in social bookmarking’s defenses against spam and general social filters such as Digg’s front-page promotion mechanism or del.icio.us’s popularity or aggregation filters.
At the moment, the defenses of the social bookmarking sites are fairly weak. Currently del.icio.us has certain filters that destroy obvious spammers, prevent excessive tagging from aggregating onto every tag page, and prune the front page’s new bookmarks list. But del.icio.us’ only solid defense against automated attack is their signup captcha. As we’ve seen with BlogSpot, a signup captcha is only worth what it costs a spammer to pay an Indian or Filipino company to fill in.
del.icio.us may soon have to contend with spammers that post automatically or through peer to peer schemes such as TagExplosion or hide their spam like steganographers, continuously studying filters to evade and manipulate them.
Digg has similar problems, and is currently using similar techniques with similar vulnerabilities. As Digg is a more socially oriented site, it also uses more socially oriented solutions to fight off the spam problem. Users are encouraged not to post duplicate stories by having to first wade through other users’ submissions, and once posted other users quickly respond and post complaintative comments against problematic links. Users of Digg also help fend off spam by reporting what is wrong to the administrators and to the other users watching the live queue.
As frequent Digg users are well aware, Digg has had a tough battle in the recent times fighting vandalism. So far however, only a tiny minority vandalize the site, spammers are still thankfully somewhat in the dark as to how they might abuse the system.
This is all very reminiscent of the rise of search engines, Compare a search on Altavista for viagra to the same search on Google. Search engines that failed, failed in large part because spammers eventually reverse engineered the algorithms they used to determine relevance and simply rearranged their pages to suit those algorithms. The top result for viagra: ‘t-e-x-a-s-poker.com’ on altavista is the end result of the eventual failure of the first generation algorithms.
It’s likely that to survive and remain useful, social services will have to follow in this mold and emulate Google by switching their filters to factors that are more expensive and more complicated to engineer, such as third party references, longevity, references from known trusted sources, and patterns of human created content. Or they may be forced to become less social, such as with Wikipedia’s recent policies. Even then, it’s certain to be an ongoing battle.