A transparent checkpointing mechanism

The Dynamite checkpointer

This checkpointer enables the user to transparently add checkpointing facilities to his application, i.e., no modifications to either the user's program, nor the kernel are needed. Supported are

• Solaris (2.5.1, 2.6, 7 & 8) and Intel Linux (2.0.x, 2.2.y and 2.4.z).
• Open files
• Dynamicly loaded libraries

This checkpointer is available under GNU GPL.
Please refer to [1] for details.

You will find the following below:

• Checkpointing: why and how is this done
• Downloading your copy of the checkpointer
• Getting started with your application
• Documentation on the checkpointer
• Dynamite: the checkpointer as part of a dynamic load-balancing environment for PVM
• Hector: the checkpointer as part of a dynamic load-balancing environment for MPI
• History and future prospects
• FAQ Frequently Asked Questions

Checkpointing: why and how is this done

What can a checkpoint facility be useful for?

• Fault tolerance, increase the fault tolerance of your program by checkpointing it at regular intervals
• Enable dynamic scheduling/ process migration, see Dynamite
• Enable rollback transactions
• playback debugging

How does the Dynamite checkpointer work?

• Basically, the address space of the process and its mapped shared libraries are written to file. This is done when the process receives a signal USR1. The handler for this signal resides in the (heavily modified) dynamic loader, which also keeps track of the shared libraries and open files (in order to accomplish this, certain system calls have been wrapped).
• The internals are well documented in [1].
• There is also quite some documentation present in the source itself, which can be extracted using doc++.

Downloading your copy of the checkpointer

N.B. The following links will give you an older version of the checkpointer. Go here for the newest version.

• Download the checkpointer source: ckpt.tar.gz.
• Download the checkpointer testsuite: ckpt-test.tar.gz

Getting started with your application

This information (and more) may be found in the README-file which is included in ckpt.tar.gz:

• Compile: make in source directory.
  First you have to compile the sources, in order to obtain the dynamic loader, which is capable of handling checkpointing facilities.
• (Re)link you application using gcc:
  Solaris: -Wl,-I,/path/to/dynamic/loader/dynld.so
  Linux: -Wl,-dynamic-linker,/path/to/dynamic/loader/dynld.so -rdynamic
• Setup environment variables:
  export DPVM_CKPTDIR=$HOME/store/checkpoint/here
  Only if necessary: DYNAMITE_LD_LIBRARY_PATH

Documentation on the checkpointer

For the moment, please refer to Checkpointing: why and how is this done.

Dynamite: the checkpointer as part of a Dynamic load-balancing environment for PVM

Dynamite has its own page now!

Dynamite provides a dynamic load balancing system on top of PVM. Therefore, it is also called Dynamic PVM or DPVM. Tasks are transparantly migrated between the nodes in order to maintain an optimal load balancing. The implementation has completely been implemented in user-space. Both direct and indirect PVM connections are supported.

Dynamite consists of

• A scheduler, which takes care of a proper workload distribution
• A monitor, which monitors the workload on workstations in the cluster
• A checkpointing/migration mechanism.

The reader is recommended to read [1], [2] and [3].

Hector: the checkpointer as part of a Dynamic load-balancing environment for MPI

Currently we have an alpha version running. However, some thorough testing has yet to be carried out, before we can release this software.

History and future prospects

The checkpoint-enhanced dynamic loader is based on the original code of the Linux dynamic loader, version 1.9.9., ld.so-1.9.9.tar.gz.

The checkpointing library has now been ported to the dynamic loader which comes with glibc-2.1, in order to support programs linked with the glibc-2.1 library properly. See ./ckpt2.html.

FAQ Frequently Asked Questions.

These questions are to be placed in a separate file later.

• [Q]: Why should I use the Dynamite checkpointer, I have heard of so many alternatives?
  [A]: All alternatives we know of do not give the full support we do (e.g. open files), or require either the kernel or the user program to be modified:
  • Eduardo Pinheiro Checkpoint Project, epckpt
  • Libckpt: Transparent Checkpointing under Unix
  • Portable Checkpoint Compiler
  • Managing Checkpoints for Parallel Programs, CoCheck
  • Checkpointing Research At Tennessee (includes many links)
  • Old stuff...
  
  
• [Q]: Is it possible for my program to continue running after checkpointing?
  [A]: Yes, look at noexit.c how to do this (there are many comment lines in there). On the other hand, this procedure only pays off if your executable is really large.
  
  
• [Q]: What exactly is the issue concerning the libc version under Linux?
  [A]: Until recently, programs using Linux glibc-2.1.x and higher were not supported. Therefore you would need a local copy of either libc-5 or glibc-2.0 and compile your application with one of these libraries.
  Now there is a newer version of the checkpointer that should work fine with the newer libc versions under Linux.

References

[1], Download pdf-version

B.J. Overeinder; P.M.A. Sloot; R.N. Heederik and L.O. Hertzberger: A Dynamic Load Balancing System for Parallel Cluster Computing, Future Generation Computer Systems, vol. 12, nr 1 pp. 101-115. May 1996.

[2], Download pdf-version, download postscript-version

K.A. Iskra; F. van der Linden; Z.W. Hendrikse; B.J. Overeinder; G.D. van Albada and P.M.A. Sloot: The implementation of Dynamite - an environment for migrating PVM tasks, Operating Systems Review, vol. 34, nr 3 pp. 40-55. Association for Computing Machinery, Special Interest Group on Operating Systems, July 2000.

[3], Download pdf-version, download postscript-version

K.A. Iskra; Z.W. Hendrikse; G.D. van Albada; B.J. Overeinder and P.M.A. Sloot: Dynamic Migration of PVM Tasks, in L.J. van Vliet; J.W.J. Heijnsdijk; T. Kielmann and P.M.W. Knijnenburg, editors, ASCI 2000, Proceedings of the sixth annual conference of the Advanced School for Computing and Imaging, pp. 206-212. ASCI, Delft, June 2000. ISBN 90-803086-5-x.

[4], Download pdf-version, download postscript-version

K.A. Iskra; Z.W. Hendrikse; G.D. van Albada; B.J. Overeinder and P.M.A. Sloot: Experiments with Migration of PVM Tasks, in ISThmus 2000, Research and Development for the Information Society, Conference Proceedings, pp. 295-304. Instytut Informatyki, Politechnika Poznanska, Poznan, 2000. ISBN 83-913639-0-2.

Page design by Zeger Hendrikse

Most recent modification: September 24, 2002 by dick at science.uva.nl

3 hits in May 2012