Abstract.A distributed system can s u p p o r t fault-tolerant applications by replicating d a t a a n d c o m p u t a t i o n at nodes that have independent failure modes. W e present a scheme called parallel execution threads (PET) which * This work was supported in part by NSF grants CCR-8619886 and CCR-8806358, and RADC contract number F30602-86-C-0032 His research interests include programming language design and implementation, programming environments, and software engineering. Dr. LeBlanc's current research work involves application of these interests in distributed processing systems. As co-director of the Clouds Project, he is studying language concepts and software engineering methodology for utilizing a highly reliable, object-based distributed system. He is also interested in specification-based software development methodologies and tools. Dr. LeBlanc is a member of the Association for Computing Machinery, the IEEE Computer Society and Sigma Xi.can be used to i m p l e m e n t fault-tolerant c o m p u t a t i o n s in an object-based distributed system. In a system that replicates objects, the P E T scheme can be used to replicate a c o m p u t a t i o n by creating a n u m b e r of parallel threads which execute with different replicas of the inv o k e d objects. A c o m p u t a t i o n can be c o m p l e t e d successfully if at least one t h r e a d does not e n c o u n t e r any failed nodes and its c o m p l e t i o n preserves the consistency of the objects. The P E T scheme can tolerate failures that occur during the execution of the c o m p u t a t i o n as long as all threads are not affected by the failures. W e present the algorithms required to i m p l e m e n t the P E T scheme and also address some performance issues.Key words: F a u l t -t o l e r a n t c o m p u t i n g -A t o m i c i t y -Distributed systems and replication 70