The parameter set in an adaptive control Monte Carlo experiment: Some considerations

Abstract

Comparisons of various methods for solving stochastic control economic models can be done with Monte Carlo methods. These methods have been applied to simple one-state, one-control quadratic-linear tracking models; however, large outliers may occur in a substantial number of the Monte Carlo runs when certain parameter sets are used in these models. Building on the work of Mizrach (1991) and Kendrick (1994, 1995), this paper tracks the source of these outliers to two sources: (1) the use of a zero for the penalty weights on the control variables and (2) the generation of near-zero initial estimate of the control parameter in the systems equations by the Monte Carlo routine. This result leads to an understanding of why both the unsophisticated Optimal Feedback (Certainty Equivalence) and the sophisticated Dual methods do poorly in some Monte Carlo comparisons relative to the moderately sophisticated Expected Optimal Feedback method.

Learning About Learning in Dynamic Economic Models

“…These results in Mizrach (1991) and Amman andKendrick (1994b, 1995) allow one to fully characterize the three components of the cost-to-go function for the simplest one-state, one-control, one unknown parameter, quadratic linear adaptive control problem with a time horizon of two periods. Therefore, in Tucci et al (2010) we have used these results as a starting point to compare the average or representative cost-to-go with different parameter sets and thus to analyze the effects of these different parameter sets on individual runs of a Monte Carlo experiment. 11 The representative cost-to-go helps to sort out the basic characteristics of the different parameter sets.…”

“…Recall that even though the probability that the parameter will take on a specific value is zero, the probability that it will fall in a certain interval is not zero. In studying the effect of changes in the parameter (when 15 For further discussion of these issues see Tucci et al (2010). Probability density function of the initial parameter estimateθ 0|0 whenΣ θθ 0|0 = 0.25,Σ θθ 0|0 = 0.50 andΣ θθ 0|0 = 1.25.…”

“…However, they did not investigate the relationship between this result and the outcomes of the Monte Carlo experiments. In Tucci et al (2010) we have moved in this direction studying more closely this model. However more general models should also be considered…”

Comparison of policy functions from the optimal learning and adaptive control frameworks

“…Also, we have examined the properties of the Beck and Wieland model using the DualPC software in Amman, Kendrick and Tucci (2008) and the problems caused by nonconvexities in this model in Tucci, Kendrick and Amman (2007).…”

“…This is done in the following two sections. In doing the comparison we have employed two variants of adaptive control methods -the first based on the DualPC software, Amman and Kendrick (1999b), and the second using a MATLAB program with a parameterized cost-to-go function for adaptive control following the method outlined in the Amman and Ken-drick (1995) paper and the extension of these results in Tucci, Kendrick and Amman (2007). After describing both of these methods we will present in Section 7 of the paper a comparison of the policy function results obtained with (1) these two methods and (2) the Beck and Wieland method.…”