STXXL: C++ Standard Template Library for Extra Large Data Sets

I haven’t tried it, but the description is cool:

The The core of STXXL is an implementation of the C++ standard template library STL for external memory (out-of-core) computations, i.e., STXXL implements containers and algorithms that can process huge volumes of data that only fit on disks. While the compatibility to the STL supports ease of use and compatibility with existing applications, another design priority is high performance.

They also have external memory suffix arrays.

Source: geomblog.

Update: Hazel pointed me to TPIE:

TPIE is a software environment (written in C++) that facilitates the implementation of external memory algorithms. The goal of theoretical work in the area of external memory algorithms (also called I/O algorithms or out-of-core algorithms) has been to develop algorithms that minimize the Input/Output communication (or just I/O) performed when solving problems on very large data sets. The area was effectively started in the late eighties by Aggarwal and Vitter and subsequently I/O algorithms have been developed for several problem domains.

Tweet about this on Twitter0Share on Facebook0Share on Google+0Share on Reddit0Email this to someone

Leave a Reply

Your email address will not be published. Required fields are marked *