Given that many web pages are actually generated through parameters (see http://www.epinions.com/Refrigerators for just one example), the number of different pages a site can offer rises with the number of parameter combinations. How does one define “uniform sampling” in that context?

You should check this Yahoo! service: http://random.yahoo.com/bin/ryl

As far as I know, it produces random pages taken from their index. I used it in the past and, although there certainly shows a bias towards .com pages, it is pretty convenient.

Best, Dani

Say your webgraph is your Markov chain graph. And you want to sample randomly. one idea is to start with an initial seed and run your Metropolis Hastings algorithm with a burn in period and it will give you a state randomly and normally distributed around your seed.

Similar idea can be used for uniform distribution

This concept can be extended to not just pages, but entire subgraphs. Once you get into some parts of the subgraph you can never get out, thus capturing your random walker. However, without a very large index, it would be difficult to identify these parts of the graph.

Another alternative is to buy a copy of the web. See ClueWeb09 for 1B web pages for less than $1K

http://www9.org/w9cdrom/88/88.html

which includes methods for balancing out the tendency for random walks to reach well-connected pages more often than poorly-connected ones,

That may avoid the issue of having the search engine assign a “relevancy” score as well being biased to sites with specific words.

1. pick a random number $RANDOMNUMBER

2. check if there is somebody at
http://www.facebook.com/home.php#/profile.php?id=$RANDOMNUMBER

3. if yes: done,
if not, goto 1

for instance, this page is: http://bit.ly/KSZ44

but it’s probably biased toward geek topics..