New Year's Resolutions 2009

Revisiting and updating what has gone before:

0) Looking back at e.g., 2005, 2006, 2007, 2008, be aware that I tend to repeat myself in frustration, and try to address the reasons for that.

[Still failing here :-(]

1) Stop arguing with marketers about "conversation". It wastes my time, and it annoys the flack. It's not going to do any good.

[God, what a huge amount of time I've wasted on this, squeaking against bullhorns]

2) Stop being delusional about ever having more influence. That ship has sailed.

[Sigh ...]

3) Keep OUT of the "Net Neutrality" politics. It'll only hurt me.

4) "Life Trumps Blogging"

[Need more work here, though making progress :-(]

On "A Personal Appeal From Wikipedia [*CO*]Founder Jimmy Wales"

There's a Wikipedia fundraising message which is drawing a bit of critical comment for the implicit self-promotion it contains. The efforts to remove Larry Sanger from history as a Wikipedia co-Founder are an ongoing matter. Worse, it plays into Wikipedia's weakness in that what's widely reported in the press tends to be taken as true, even if it's obviously the result of a PR campaign.

One could argue the text is not technically inaccurate. But I would also say it's fair to observe how this feeds into the history-rewriting process, and how there's a system of benefit to a small number of the insiders in this supposedly volunteer democratic process. As Wikimedia's UK PR-flack has stated: "Jimbo applying his rock star factor is one of his most useful jobs for WMF :-)"

Also notable is the appeal's statement that:
"Like a national park or a school, we don't believe advertising should have a place in Wikipedia."

Again, while not inaccurate, it's useful to know that Wales's own attitude towards advertising on Wikipedia in the early days was being quite open to it. And he certainly doesn't have the anti-advertising attitudes that many people think he has (to be fair, it's not all his fault, but he definitely gives certain impressions from which one might easily take a mistaken view).

One good quote, from 2003:

"I know that not many people share my curious political views, but to me, it's much worse to seek money from governments, i.e. to ask them to take money by force from others, than it is to accept advertising money."

But the number of people who will read this commentary is a joke compared to the hype-machine.

My _Guardian_ column on Wikipedia, Sex, Scandal, and problematic material

Sting in the Scorpions tale is the exposure of Wiki's weakness
http://www.guardian.co.uk/technology/2008/dec/18/wikipedia-jimmy-wales

"Anyone who needs to use an old album cover to make a Wikipedia sexual controversy is not trying very hard"

[I didn't pick the title, though I like it for the wordplay - I would have been more pragmatic myself and gone for more SEO-friendly phrasing]

I didn't do the IWF-is-absurd article, as that's been done extensively. (there was one recently by Cory D., and I certainly don't have anywhere near the platform he has). Instead, I used this event as an opportunity to write about the reality of Wikipedia's very real problem with "determining the boundary between provocative and profane."

This column will not further endear me to the Wikimedia Foundation.
But they know the material is true.

[For all columns, see the page Seth Finkelstein | guardian.co.uk.]

Pew Research Center: "Future of the Internet III: How the Experts See It"

The Pew Research Center has released Future of the Internet III: How Experts See It

A survey of internet leaders, activists and analysts shows they expect major technology advances as the phone becomes a primary device for online access, voice-recognition improves, artificial and virtual reality become more embedded in everyday life, and the architecture of the internet itself improves.

They disagree about whether this will lead to more social tolerance, more forgiving human relations, or better home lives.

I was one of the people contacted, and gave my perspective. The press release quotes me for my bubble-popping views on Second Life and its ilk:

The evolution of augmented and virtual reality: ...

"For some reason I've never been able to comprehend, certain pundits can seriously propose that the wave of the future is chatting using electronic hand-puppets. Flight Simulator is not an aircraft, and typing at a screen is not an augmentation of the real world."
- Seth Finkelstein, author of the Infothought blog, writer and programmer

The "electronic hand-puppets" phrase sums it up for me. At the height of the hype, when Second Life was being marketed to various A-listers, I wished I had had the opportunity to attend one of those presentations and bring a ventriloquist's dummy, communicating only by using the dummy (i.e. raise the dummy's hand instead of mine, put it in front of me and move the mouth when I spoke, etc). I'd say the dummy was my "avatar", and I was in Projection Reality. The point would be to illustrate how ridiculous it all is, but I suspect the audience wouldn't get the joke (plus I don't have the status to pull off something like that).

Anyway, I'm quoted a few more times, but the only other really good line I have is "One Laptop Per Child is a classic "Ugly American"-style project."

Reflections on Wikipedia vs CleanFeed, Censorship and British ISP's

Wikinews (a separate project from Wikipedia): British ISPs restrict access to Wikipedia amid child pornography allegations

The Register: Brit ISPs censor Wikipedia over 'child porn' album cover

Someone else can do the pundit argument that the general material is horrible, but does it justify censorship? It's not for me, and nobody cares anyway (well, except for people who'd like to take a line out of context to smear me, but no need to cater to them). This post is for pointing out that this incident gives a golden opportunity to see the technical details of how the UK "CleanFeed" national censorware system works in practice.

Note the "Internet Watch Foundation" which maintains the blacklist, has confirmed the event.

Apparently only one or a few Wikipedia pages are on the UK blacklist, but it seems the effect of a site having even one page on that blacklist is to force all site traffic though a proxy, which assigns it to a single Internet address per Internet service provider. This aspect of putting a huge number of users on a single IP has the effect of severely disrupting Wikipedia's administrative controls for the relevant population.

In the discussion pages on Wikinews, and on Wikipedia, there's a rare instance of true aggregated user research, as people from across the UK are posting what error message they see when they try to access the blacklisted material (e.g. an old album cover by the band "Scorpions" called Virgin Killer), and what's happening from various UK ISP's. And poor Wikinews gets no respect for journalism, as the huckster A-listers are enamoured with hyping Wikipedia as much as possible, even though Wikipedia is a very poor fit for journalism.

I'm not going to speculate where this all will end up now. But it's as big an explosion over national censorware as has even been seen in the Western world.

Update: "CleanFeed" flow chart

Welcome Slashdot Readers

Bennett Haselton, in a Slashdot article today (thanks)

(My own favorite blog that nobody's ever heard of is Seth Finkelstein's InfoThought, which is usually logical and insightful and is only about 25% of the time about how "nobody ever reads this blog, so what's the point". His Guardian columns are also good and usually don't have that subtext, perhaps because it's considered impolite to use a newspaper's column-inches (column-centimeters?) to complain that you have no voice.)

No, because then they wouldn't publish it! :-)

Though on that theme, I recommend the Guardian column I wrote:
"If you want to change the world, a blog may not be the place to start"

Further, the unread blogger sayeth not, due to irony overload :-(

My _Guardian_ column on Google Flu Trends

Why you should be concerned about Google Flu Trends

http://www.guardian.co.uk/technology/2008/nov/27/privacy-searchengines

The search engine has unwittingly hung a big sign on itself advertising services for government surveillance

The title's fine, though when I submitted it I proposed "Google Flu And Monitoring Health". I was aiming for a deliberate ambiguity in the phrase "monitoring health" between the literal sense of seeing where is sickness and more metaphorical sense of good safeguards against misuse of private data. Maybe I was being too clever.

I know many people have written on this topic, but I really tried to capture the double-edged nature here. That is, the conflict between "That's so cool" for technical achievement, and "That's so scary" in terms of potential for abuse. As I think of it: Technology-positive social criticism.

I'm hoping to popularize a phrase I've used here: "surveillance engines"

[For all columns, see the page Seth Finkelstein | guardian.co.uk.]

"The big business of net censorship"

Echo: The big business of net censorship - "Clamping down on free speech on the internet has been a lucrative enterprise for software manufacturers" - Jo Glanville

We know as much as we do because of the great research of organisations such as the OpenNet Initiative and because of the brave detective work done by researchers such as Seth Finkelstein and Ben Edelman. Under the Digital Millennium Copyright Act in the US, no one can legitimately examine the lists of blocked sites or ask for a review.

Daniel Brandt (Scroogle, Google Watch) on Google ranking anomalies

[Below is a guest post from Daniel Brandt, who gives his experiences and speculations below. His views are of course his own and not necessarily my own, but I do believe them worth hearing]

There is definitely some sort of filtering going on in Google's rankings for certain keywords. It took 18 months for any of the pages on my wikipedia-watch.org site to rank better than 200 deep or so for any combination of keywords from those pages. During this time, Yahoo and Live.com were ranking the same pages well for the same terms.

When I test terms on Google, I test with a multi-threaded private tool that checks more than 30 Google data centers on different Class Cs, and shows the rank up to 100 on each one. I can see changes kicking in and out as they propagate across these data centers. The transitions can take several days in normal cases, as when a new or modified page is appropriated into the results.

Wikipedia-watch.org has been a website now for 36 months. During the first half of that period, no pages ranked higher than 200 deep or so, even if you used two fairly uncommon words from that page to search for it (this is documented at wikipedia-watch.org/goohate.html). During the second half of that period, after it took about four months to settle into the transition, the deeper pages ranked okay, and were on a par with Yahoo and Live. But there was still one glaring exception to this rule: the search for the single word "wikipedia" failed to turn up the home page in the first 100 results almost all of the time during this second period.

When it did show up, it always ranked within the top 15. When it didn't show up, it was always greater than 100. There was never anything in between, and I've been watching this curiosity for the last six months now. For the first five of these months, it might kick in for a few hours on all data centers, and then disappear. This happened several times. Twice it kicked in for a few days, and then disappeared from the top 100 again. During the last 30 days, it has been in about half of the total time, for several days each time, and then disappeared again for days. It's always one or the other -- in the top 15 or not even in the top 100. Meanwhile, the deep pages have ranked okay the last 18 months, and have been stable this entire time.

This behavior is something I'm seeing only for the home page, and only on Google but not on Yahoo or Live. It happens almost exclusively when the word "wikipedia" is the solitary search term, or maybe this one word and another term that's also on that page. If you add a third term you begin ranking reasonably well for my home page, presumably because the search is now specific enough to override the filtering. By the way, this home page has a PageRank of 5 and Yahoo counts 3,500 external backlinks to that home page (there's a counting tool at microsoft-watch.org/cgi-bin/ranking.htm). You cannot use Google to count backlinks, because for years now, Google has been deliberately suppressing this information.

I should also add here that for three years running, another site of mine, Scroogle.org, had a tool that compared the top 100 Google results for a search with the top 100 Yahoo results for that same search. This may come as a surprise to some, but the divergence was consistently 80 percent for all searches. In other words, only 20 out of 100 links showed up on both Yahoo and Google for any search, and the other 80 on each engine were unique in their top 100. The overall quality of the results was about even for each engine. To put this another way, there's a lot of wiggle room for a particular engine to vary the top results, and still look like they're providing the most relevant links.

To make this long story shorter, I believe that there is some sort of backend filter that affects which top results are shown by Google. This actually makes some sense, since most searchers never go beyond the first page of results (at 10 links per page). This means Google's reputation and ad revenue depend heavily on the utility of that first page. A filter that favors recency is one component of this, because Google jacks up recent forum and blog posts (and increasingly even news posts). Everyone expects this by now. Static sites such as wikipedia-watch.org must compete in this sort of environment.

In addition to the recency factor, I think there is filter weighting based on what I call "newbie searches." A newbie search is grandpa or grandma searching for single words such as "wikipedia" or "email" that normally return millions of results, which of course is useless to the searcher. Such searches are stupid to begin with, but Google must cater to stupidity in order to push ads, since ad revenue is 99 percent of total revenue. There might even be some sort of rotational weighting for newbie searches.

And call me a tin-foil hatter if you must, but I also believe that "hand jobs" are involved in tweaking this filter. In other words, there is a political dimension to it as well. Regrettably, I cannot prove this. We need more transparency from Google, and we need it now, before the situation becomes even more suspicious.

Uncommon Google items - "Real Sex", Counts, Distinctive Sentences

A few things that have not been echoed widely, and deserve more notice (not that I can change it much, but here's something ...)

Tony Comstock - Taking the Real Sex out of [Real Sex] Searches. (Is the googlebot erotophobic?)

"But last July I noticed that although the word [real] is #5 in Google’s listing of keywords in inbound links, [real] doesn’t appear anywhere in the Googlebot’s listing for our site content keywords. That’s right, the Googlebot doesn’t see the word [real] at the home of real life, real people, real sex."

This is actually really interesting, though I don't really have the tools to investigate it. I lean towards thinking it's some sort of spam or "trusted site" algorithmic issue rather than an anti-sex bias of Googlebot.

David Weinberger - Obama v. Bush: Google counts

Estimated Google hits for [“Barack Obama”] are more than [“George W. Bush”] and [“George Bush”] combined. This strikes me as a clear demonstration that the meaning of those hit numbers is not what one intuitively expects them to be. It's known that the numbers are not full database counts - people read them as full database counts, but they are merely a statistical estimate. I suspect, just off the top of my head, that the results are heavily skewed by a recency bias in what's used for the estimate. I'd believe Barack Obama has been mentioned overall more than George Bush in the very recent past.

Walt Crawford - How Common is Common Language?

An extensive examination where Google is used as a testbed for analyzing the utility of checking phrases in cases of suspected plagiarism. "Even relatively short sentences seem to be unusual most of the time. On the order of 85% in this sample, and I suspect that percentage would be higher in a truly random sample. ... What I believe may be true: If you’re suspicious that a clumsy plagiarist has cut-and-pasted without paraphrasing, almost any medium-length sentence may suggest you should check further. It may be entirely innocent. But it seems surprisingly uncommon for the same, say, 11-word string to show up more than once."

My _Guardian_ column on Google Book Search Settlement

Google's copyright war will have open access advocates up in arms

http://www.guardian.co.uk/technology/2008/nov/06/google-open-access-copyright

.. on the copyright issues surrounding Google's digitising of books

There's some value in enemy-of-my-enemy opposition, where the interests of an advertising near-monopoly are a counterweight to a content cartel. But battles between behemoth businesses should not be mistaken for friendship to libraries, authors or public interest.

[Update: I didn't pick the title, but I don't find it a problem]

I endorse Barack Obama

As I've said before, normally I don't write about pure politics, since if my influence on Internet freedom is marginal, my influence on the electoral process isn't even a speck on the page. And while this election doesn't look close, you never know. I suspect my readership skews liberal and intellectual, but I probably have a few conservative, older, readers in the mix.

Vote for Barack Obama

I've never been one of the worshippers of "The One", but I actually have come to think more favorably of Barack Obama over the past few months. Politically, I am impressed by how he fought off the inevitable Swiftboating attempts, and the competence of the campaign organization overall.

In contrast, McCain's stunts like "suspending" his campaign during the financial crisis - and then doing nothing but grandstanding - refute any argument for his experience or leadership. Obama clearly demonstrated both intelligence and steadiness there.

By all governing measures, Obama has proved to be a better candidate than McCain - the people he surrounds himself with, the Vice President choice, the strategic decisions he's made - and I believe the policies he's advocated (Obama's quip "It's like these guys take pride in being ignorant" sums it up well).

So I endorse Barack Obama.

Catch-up: Google column soon, Digital Sharecropping, Wikia CEO tale

["Life Trumps Blogging", but some collected notes]

1) The obligatory pontification about the Google Book Search settlement, a topic on which all Google interested pundits must write about, will appear in my next Guardian column, in a few days.

2) Briefly noted: The Economist Innovation Awards and Summit

Business Process: Jimmy Wales, Founder, Wikipedia for public collaboration as a form of product and content development.

I have yet to see a more blatant business jargon way of saying "for electronic plantations full of digital sharecroppers".

3) Amazing story from an unreliable source of "Why Jimmy Wales got booted from Wikia's top job". I wouldn't have believed it, and it's been denied, but a reliable source confirmed to me that it's true. There looks to be some very strange backroom politics going on within Wikia (the company aiming to "commercialize the hell out" of Wikipedia concepts and success, though having no significant financial connection to the Wikimedia Foundation).

DMCA link-blogging

I was thinking of taking the DMCA off the blog description, as it's been a while since I wrote about it. But today is a veritable DMCA-fest due to the ten year anniversary. For old times sake:

EFF: Unintended Consequences: Ten Years Under the DMCA
Extensive report of chilling effects (I'm mentioned :-))

Wired Threat Level Blog: 10 Years Later, Misunderstood DMCA is the Law That Saved the Web
A dubious praise of the take-down provisions, and then discusses anti-circumvention (goes into censorware, and N2H2 censorware company, and I'm not mentioned! :-()

Freedom to Tinker DMCA Week, Part I: How the DMCA Was Born
Good history, for those interested in the policy origins and maneuvering.

Public Knowledge: 10 Years of the DMCA
Looks like it's not going to rehash what's been said, but unearth uncommon examples.

Note: I know the DMCA anti-circumvention cycle has started. Sigh ...

Chasing The Wikia Layoffs Story (30%? 10%? Legalism?)

A few days ago, the tech-gossip blog "Valleywag" published an item that the digital-sharecropping ad-farm, err, excuse me, "communities" wiki site company "Wikia" had laid off 30% of its staff. Recall, Wikia is the company Jimmy Wales started to, in words of one article (not by me!), "take the success -- and, indeed, the underlying philosophy -- of Wikipedia," and "commercialize the hell out of it".

So, the story started metastasizing through the relevant bogosphere organs, and Wikia then issued a denial ... or so it looks. However, as a question I asked Jimmy Wales via his Wikipedia user discussion page observed, the language was ... interesting.

Jimmy, speaking as a journalist, I hate to bother you over this story, but it's necessary for me to do "due diligence". I figure since it's all public statements, I'll ask it here rather than emailing you (also some protection for me!). I've read the denials of the Valleywag story about Wikia laying off around 30% of its workforce. However, to nail things down on the record, when Wikia says - "as part of a reorganization, Wikia recently let go less than 10% of its salaried employees" - that raises an alarm bell for me in terms of legalistic language. To wit: 1) Did Wikia let go others who were not SALARIED EMPLOYEES? (as in, for example purposes, but not meaning this mention to be exclusive: contractors). To be precise, 2) If X people received pay for work in September 2008, and Y people are projected to receive pay for work in January 2009, then X - Y is ... (3? 12? what? - note the phrasing is meant to cover the loophole of people staying on for something like just stock options, so not formally "let go"). Thanks for your time on this matter.

The only reply from him was to remove the question with a note "wrong place for this question".

I did some other checking without much result. I was going to let this all pass, since it didn't seem worth the effort, but then today I had occasion to email Jimmy to check out another story, so added it on. We'll see.

It's always unclear how far to push things like this. Wikia could be telling the truth. It's possible. If they claim they simply don't want to talk to me, because I'm an idiot conspiracy mongering FUD'er, I shouldn't go to the wall over minor stuff. On the other hand, if they play it wrong, they can come off looking like vindictive weasels. It's a complicated game.

Oh yeah, I also have a blog, I'm sure they weight that with all the influence and power it commands.