Bernardo A. Huberman and Lada A. Adamic
Appeared as the brief communication, "Growth dynamics of the World-Wide Web" in Nature, 399 (1999) 130.
We present a theory for the growth dynamics of the World Wide Web that
takes into account the wide range of stochastic growth rates in the number
of pages per site, as well as the fact that new sites are created at different
times. This leads to the prediction of a universal power law in the distribution
of the number of pages per site which we confirm experimentally by analyzing
data from large crawls made by the search engines Alexa and Infoseek. The
existence of this power law not only implies the lack of any length scale
for the Web, but also allows one to determine the expected number of sites
of any given size without having to exhaustively crawl the Web.