One More Step Toward Infinite Storage

Acccording to a new IDC study reported in Wired, the world had 185 exabytes of storage available last year and will have 601 exabytes in 2010. Meanwhile, the amount of “digital information” generated will grow from 161 exabytes last year to 988 exabytes in 2010.

Their point is that we lack the storage capacity to store everything. This seems to go against the theory that we nearly have infinite storage. But I do not think so. How can they tell how much storage is available? Do they include the NSA? Do they include Echelon? Do they include all of the secret agencies in the world storing massive quantities of data in general?

What might be a more interesting observation is that few people store everything as of now. And I do not expect that people will start storing everything soon. Storage costs must still come down a bit, and software must adapt. But in a few short years, everyone will store copies of everything. And managing all this data, whatever managing means, will become a big deal. And it will not be a nice database problem either because this data will not follow nice database schemas.

One thought on “One More Step Toward Infinite Storage”

  1. As a footnote, Library and Archives Canada is also worried about this, of course. Their mandate is to archive (some) of these exabytes – the ones that matter or can be considered part of the “National Heritage” (http://www.collectionscanada.ca/cdis/index-e.html). So (one of) their problem(s) is – how do we tell what matters and what doesn’t? Given limited management ability / space / archivists etc., do we archive / annotate Daniel’s and Andre’s blogs or Nelly Furtado’s MySpace site?

    As far as absolute numbers of exabytes go, I don’t think that’s an especially good measure for anything. YouTube videos take up quite a lot of space but there aren’t more than a few million. It’s the “objects” and the information about them that matters.

    Although the question of what a “digital object” actually consists of is also in question. Should it be the picture, or the picture with the text or the picture with the text in the blog…?

Leave a Reply

Your email address will not be published. Required fields are marked *

To create code blocks or other preformatted text, indent by four spaces:

    This will be displayed in a monospaced font. The first four 
    spaces will be stripped off, but all other whitespace
    will be preserved.
    
    Markdown is turned off in code blocks:
     [This is not a link](http://example.com)

To create not a block, but an inline code span, use backticks:

Here is some inline `code`.

For more help see http://daringfireball.net/projects/markdown/syntax