Planning under uncertainty is a central problem in the study of automated sequential decision making, and has been addressed by researchers in many di erent elds, including AI planning, decision analysis, operations research, control theory and economics. While the assumptions and perspectives adopted in these areas often di er in substantial ways, many planning problems of interest to researchers in these elds can be modeled as Markov decision processes (MDPs) and analyzed using the techniques of decision theory.This paper presents an overview and synthesis of MDP-related methods, showing how they provide a unifying framework for modeling many classes of planning problems studied in AI. It also describes structural properties of MDPs that, when exhibited by particular classes of problems, can be exploited in the construction of optimal or approximately optimal policies or plans. Planning problems commonly possess structure in the reward and value functions used to describe performance criteria, in the functions used to describe state transitions and observations, and in the relationships among features used to describe states, actions, rewards, and observations. Specialized representations, and algorithms employing these representations, can achieve computational leverage by exploiting these various forms of structure. Certain AI techniques| in particular those based on the use of structured, intensional representations|can be viewed in this way. This paper surveys several types of representations for both classical and decision-theoretic planning problems, and planning algorithms that exploit these representations in a number of di erent ways to ease the computational burden of constructing policies or plans. It focuses primarily on abstraction, aggregation and decomposition techniques based on AI-style representations.

No abstract

Reasoning about change requires predicting how long a proposition, having become true, will continue to be so. Lacking perfect knowledge, an agent may be constrained to believe that a proposition persists indefinitely simply because there is no way for the agent to infer a contravening proposition with certainty. In this paper, we describe a model of causal reasoning that accounts for knowledge concerning cause-and-effect relationships and knowledge concerning the tendency for propositions to persist or not as a function of time passing. Our model has a natural encoding in the form of a network representation for probabilistic models. We consider the computational properties of our model by reviewing recent advances in computing the consequences of models encoded in this network representation. Finally, we discuss how our probabilistic model addresses certain classical problems in temporal reasoning (e.g., the frame and qualification problems).Le raisonnement a propos des modifications nkcessite de prkdire combien de temps une proposition demeurera vraie une fois qu'elle I'est devenue. En I'absence de connaissances parfaites, un agent peut itre force de croire qu'une proposition persiste indefiniment simplement parce qu'il lui est impossible d'inferer avec certitude une proposition contraire. Dans cet article, nous decrivons un modele de raisonnement causal qui tient compte des connaissances concernant la relation entre la cause et I'effet et la tendance des propositions a persister ou non en fonction du temps. Le modele comporte un encodage nature1 sous forme de representation en reseau pour les modeles probabilistiques. Les proprietes informatiques du modele sont prises en consideration a la lumiere des recents developpements dans le domaine de I'estimation des consequences des modeles encodes dans cette representation en rkseau. Enfin, on y discute comment Ie modele probabilistique traite certains problemes classiques de raisonnement temporel (par exemple, les problhes de cadre et de qualification). que, reseau de croyances.Mots clPs : raisonnement tenmorel. raisonnement causal, incertitude, modeles probabilistiques, inference probabilisti-[Traduit par la revue] Comput. Intell. 5. 142-150 (1989)

No abstract