HPL-2012-181GraHPa: Fault-tolerant and Large-scale Graph Processing; A Biomedical Simulation Use Case
Vaquero, Luis; Navas, Alvaro; Gomez, Ricardo; Cuadrado, Felix; Guijarro, Julio
Keyword(s): graph analysis; large scale; pregel; failure tolerance; scalability; biomedical; simulation
Abstract: Graph analysis has grown as widespread technique to solve/understand a wide variety of our daily problems. Many systems try to build upon these ubiquitous data structures to ease problem modeling. However, none of them fully keeps a highly scalable and failure tolerant approach to easy modeling. Scalability is understood the addition of more resources and a better utilization of the available ones. However, end users should remain unaware of these scalability, resource handling and failure tolerant features. This is specially true for biomedical simulations, where resources can be scarce and users are not interested in dealing with these computational problems, but it is definitely in their interest to leverage these advantages. Here, we present an implementation that tries to cover most of current state of the art limitations. Our implementation offers a conveniently abstract modeling level that hides the distributed nature of the system. Indeed, scalability and failure tolerance are delivered in a transparent manner to users. We validate these features with the implementation of a classic biomedical simulation use case at unprecedented scalability levels.
External Posting Date: August 21, 2012 [Abstract Only]. Approved for External Publication - External Copyright Consideration
Internal Posting Date: July 21, 2012 [Fulltext]