| dc.description.abstract | Recently there has been an increasing demand for efficient mechanisms of carrying out computations that exhibit coarse grained parallelism. Examples of this class
of problems include simulations involving Monte Carlo methods, computations where
numerous, similar but independent, tasks are performed to solve a large problem or
any solution which relies on ensemble averages where a simulation is run under a variety of initial conditions which are then combined to form the result. With the ever
increasing complexity of such applications, large amounts of computational power are
required over a long period of time. Economic constraints entail deploying specialized
hardware to satisfy this ever increasing computing power.
We address this issue in Dispatch, a peer-to-peer framework for sharing computational power. In contrast to grid computing and other institution-based CPU sharing
systems, Dispatch targets an open environment, one that is accessible to all the users
and does not require any sort of membership or accounts, i.e. any machine connected
to the Internet can be the part of framework. Dispatch allows dynamic and decentralized organization of these computational resources. It empowers users to utilize
heterogeneous computational resources spread across geographic and administrative
boundaries to run their tasks in parallel.
As a first step, we address a number of challenging issues involved in designing
such distributed systems. Some of these issues are forming a decentralized and scalable network of computational resources, finding sufficient number of idle CPUs in
the network for participants, allocating simulation tasks in an optimal manner so as to reduce the computation time, allowing new participants to join the system and run
their task irrespective of their geographical location and facilitating users to interact
with their tasks (pausing, resuming, stopping) in real time and implementing security
features for preventing malicious users from compromising the network and remote
machines.
As a second step, we evaluate the performance of Dispatch on a large-scale network consisting of 10−130 machines. For one particular simulation, we were able
to achieve up to 1500 million iterations per second as compared to 10 million iterations per second on one machine. We also test Dispatch over a wide-area network
where it is deployed on machines that are geographically apart and belong to different
domains. | en |
