Mathemagenic

David Sifry answers my question about Technorati:

Here's the basics:

1) We spider blogs, and match up their links to your blog - to anywhere on your blog 2) In the inbound blog list, we use the outbound links from the blog homepage, not from the archives 3) We do process RSS feeds an other metadata, but that doesn't affect your inbound blog stats 4) Nightly, we go through the database and re-calculate the number of inbound blogs and links, which helps us double-check our work and also allows us to create the interesting newcomers list, the interesting recent blogs list, etc.

We strice to be accurate all the time. Sometimes things slip through. For example, one of the reasons why your inbound blog count may be down today is because we were doing maintenance of the database last night to remove duplicate blogs - for example, Radio Userland has an obnoxious habit of sending pings to www.weblogs.com for each weblog "category" if you use multiple categories on your blog. Same information, same author, just link spam, basically. So, last night we cleaned out a bunch of that stuff. If you were linked from a bunch of people's blog categories, then you lost those inbound blogs. Then again, so did everyone else. :-)

The last thing to remember is that while we strive for accuracy and completeness, we still do have bugs and have to fix things. If you notice something strange, please don't hesitate to send us feedback (feedback@technorati.com) and let us know.

Thanks for fast reply! And for fixing the category problem (I had it in my stats).

I suggest that you add this explanation somewhere, so people know that inbound blog/link statistics are calculated based on links from homepages of other weblogs. (I guess I'm getting spoiled as a researcher: I want to know the method to trust results :)

See also more advanced version of answer in David's Technorati Tutorial, Part 1

RSS A Love Story by Brian Lamb [via Roland Tanglao: KLogs]

A Beginner's Guide to Blogs by David Wiley [via Brian Lamb]

Last technical question before I move to other things. Does anyone knows how Technorati works? Do they process blog homepages only? Or only items in RSS feeds? Or only things "not older than ..."?

I wonder because I usually observe some fluctuations in numbers of inbound blogs and inbould links. E.g. yesterday I had 100+ inbound blogs and today it's 80+. It would be interesting to know why these things change. I tried Technorati site and weblog of David Sifry with no luck.

I guess this is a quite typical question that user has about systems that digest information: what are the criteria that are used?