Exponential Models, Maximum Likelihood Estimation, and the Haar Condition

Crain, Bradford R.

doi:10.2307/2285612

Cited by 18 publications

(30 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Let G be krD(q^ kq )k where the gradient is evaluated at = t . Then G = k t ?~ k by (3). By This proves (12).…”

Section: Discussionsupporting

confidence: 53%

“…(p)k 2 0; 1). 3 We remark that the need for normalizing the k 's can be also interpreted via the notion of \comparison density", as pointed out by an anonymous referee.…”

Section: Description Of the Strategymentioning

confidence: 99%

“…By linear independence of the set of functions we mean that if ( ? 0 ) (x) is constant almost everywhere, then = 0 3. Notice that, since we restricted the range of the basis functions, k (x) ?…”

mentioning

confidence: 99%

See 2 more Smart Citations

Bounds on approximate steepest descent for likelihood maximization in exponential families

Cesa-Bianchi

Krogh

Warmuth

1994

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

An approximate steepest descent strategy converging, in families of regular exponential densities, to maximum likelihood estimates of density functions is described. These density estimates are also obtained by an application of the principle of minimum relative entropy subject to empirical constraints. We prove tight bounds on the increase of the log-likelihood at each iteration of our strategy for families of exponential densities whose log-densities are spanned by a set of bounded basis functions.

show abstract

“…Let G be krD(q^ kq )k where the gradient is evaluated at = t . Then G = k t ?~ k by (3). By This proves (12).…”

Section: Discussionsupporting

confidence: 53%

“…(p)k 2 0; 1). 3 We remark that the need for normalizing the k 's can be also interpreted via the notion of \comparison density", as pointed out by an anonymous referee.…”

Section: Description Of the Strategymentioning

confidence: 99%

See 1 more Smart Citation

Bounds on approximate steepest descent for likelihood maximization in exponential families

Cesa-Bianchi

Krogh

Warmuth

1994

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…Philosophically, this approach imposes on the model only those constraints implied by the observed data. On the flip side, learning the parameters of an exponential family model is a computationally challenging task (Crain 1976, Beran 1979, Wainwright and Jordan 2008 because it requires computing a partition function possibly over a complex state space.…”

Section: Related Prior Workmentioning

confidence: 99%

Inferring Sparse Preference Lists from Partial Information

2020

View full text Add to dashboard Cite

Probability distributions over rankings are crucial for the modeling and design of a wide range of practical systems. In this work, we pursue a nonparametric approach that seeks to learn a distribution over rankings (aka the ranking model) that is consistent with the observed data and has the sparsest possible support (i.e., the smallest number of rankings with nonzero probability mass). We focus on first-order marginal data, which comprise information on the probability that item i is ranked at position j, for all possible item and position pairs. The observed data may be noisy. Finding the sparsest approximation requires brute force search in the worst case. To address this issue, we restrict our search to, what we dub, the signature family, and show that the sparsest model within the signature family can be found computationally efficiently compared with the brute force approach. We then establish that the signature family provides good approximations to popular ranking model classes, such as the multinomial logit and the exponential family classes, with support size that is small relative to the dimension of the observed data. We test our methods on two data sets: the ranked election data set from the American Psychological Association and the preference ordering data on 10 different sushi varieties.

show abstract

“…On the other hand, as already mentioned above, for given data and exponential family MLE may fail to exist. In particular, Crain [11,12] pointed out to problems with the maximum likelihood estimation when the number of parameters is too large for the sample size. He also gave a sufficient condition for MLE to exist almost surely -the Haar condition.…”

Section: Introductionmentioning

confidence: 99%