Dual optimization for convex constrained objectives without the gradient-Lipschitz assumption

Bompaire, Martin; Bacry, Emmanuel; Gaı̈ffas, Stéphane

doi:10.48550/arxiv.1807.03545

Cited by 4 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is obviously detrimental since, in practice, time sequences are increasingly abundant and large. On the other hand, improving the computational effectiveness of estimation procedures for Hawkes processes is a current direction of research (Bompaire et al, 2018).…”

Section: Discussionmentioning

confidence: 99%

Inference of multivariate exponential Hawkes processes with inhibition and application to neuronal activity

Bonnet¹,

Herrera²,

Sangnier³

2022

Preprint

View full text Add to dashboard Cite

The Hawkes process is a multivariate past-dependent point process used to model the relationship of event occurrences between different phenomena. Although the Hawkes process was originally introduced to describe excitation interactions, which means that one event increases the chances of another occurring, there has been a growing interest in modeling the opposite effect, known as inhibition. In this paper, we propose a maximum likelihood approach to estimate the interaction functions of a multivariate Hawkes process that can account for both exciting and inhibiting effects. To the best of our knowledge, this is the first exact inference procedure designed for such a general setting in the frequentist framework. Our method includes a thresholding step in order to recover the support of interactions and therefore to infer the connectivity graph. A benefit of our method is to provide an explicit computation of the log-likelihood, which enables in addition to perform a goodness-of-fit test for assessing the quality of estimations. We compare our method to classical approaches, which were developed in the linear framework and are not specifically designed for handling inhibiting effects. We show that the proposed estimator performs better on synthetic data than alternative approaches. We also illustrate the application of our procedure to a neuronal activity dataset, which highlights the presence of both exciting and inhibiting effects between neurons.

show abstract

Section: Discussionmentioning

confidence: 99%

Inference of multivariate exponential Hawkes processes with inhibition and application to neuronal activity

Bonnet¹,

Herrera²,

Sangnier³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Maximum likelihood estimation A different paradigm consists in maximising the log-likelihood of the sample path, see Daley and Vere-Jones [14]. To the best of our knowledge, the fastest parametric approach, in the case where the kernels are a sum of exponentials with fixed decay rates, is that in Bompaire, Bacry, and Gaïffas [9]; we use this algorithm as a parametric baseline in our numerical examples. Lemonnier and Vayatis [27] use Bernstein polynomials to give a density argument to justify the choice of a linear combination of exponential decays.…”

Section: Mhp Estimation Methodsmentioning

confidence: 99%

“…• SumExp: an SBF exponential MHP model, fitted using the algorithm in Bompaire et al [9]. This is an interesting benchmark to evaluate the quality of the fit for the decay parameter β in a non SBF exponential using our method.…”

Section: Numerical Experimentsmentioning

confidence: 99%

Gradient-based estimation of linear Hawkes processes with general kernels

Cartea¹,

Cohen²,

Labyad³

2021

Preprint

View full text Add to dashboard Cite

Linear multivariate Hawkes processes (MHP) are a fundamental class of point processes with selfexcitation. When estimating parameters for these processes, a difficulty is that the two main error functionals, the log-likelihood and the least squares error (LSE), as well as the evaluation of their gradients, have a quadratic complexity in the number of observed events. In practice, this prohibits the use of exact gradient-based algorithms for parameter estimation. We construct an adaptive stratified sampling estimator of the gradient of the LSE. This results in a fast parametric estimation method for MHP with general kernels, applicable to large datasets, which compares favourably with existing methods.

show abstract

“…Such a function is then used to design an algorithm which is closely related to BPG. Since then, many other extensions of BPG have been developed to deal with the issue of lack of Lipschitz gradient [5,8,10,12,13,16,23,25]. For example, in [12,13] an inertial variant of BPG has been proposed, which relies on a Nesterov's momentum type update.…”

Section: Introductionmentioning

confidence: 99%

First-Order Algorithms Without Lipschitz Gradient: A Sequential Local Optimization Approach

Zhang¹,

Mei²

2020

Preprint

View full text Add to dashboard Cite

First-order algorithms have been popular for solving convex and non-convex optimization problems. A key assumption for the majority of these algorithms is that the gradient of the objective function is globally Lipschitz continuous, but many contemporary problems such as tensor decomposition fail to satisfy such an assumption. This paper develops a sequential local optimization (SLO) framework of first-order algorithms that can effectively optimize problems without Lipschitz gradient. Operating on the assumption that the gradients are locally Lipschitz continuous over any compact set, the proposed framework carefully restricts the distance between two successive iterates. We show that the proposed framework can easily adapt to existing first-order methods such as gradient descent (GD), normalized gradient descent (NGD), accelerated gradient descent (AGD), as well as GD with Armijo line search. Remarkably, the latter algorithm is totally parameter-free and do not even require the knowledge of local Lipschitz constants.Theoretically, we show that for the proposed algorithms to achieve gradient error bound of ∇f (x) 2 ≤ ǫ, it requires at most O( 1 ǫ × L(Y )) total access to the gradient oracle, where L(Y ) characterizes how the local Lipschitz constants grow with the size of a given set Y . Moreover, we show that the variant of AGD improves the dependency on both ǫ and the growth function L(Y ). The proposed algorithms complement the existing Bregman Proximal Gradient (BPG) algorithm, because they do not require the global information about problem structure to construct and solve Bregman proximal mappings. Additionally, based on several numerical experiments, we demonstrate that the proposed methods can perform favourably as compared to the BPG based algorithms in applications such as tensor decomposition.

show abstract

Dual optimization for convex constrained objectives without the gradient-Lipschitz assumption

Cited by 4 publications

References 22 publications

Inference of multivariate exponential Hawkes processes with inhibition and application to neuronal activity

Inference of multivariate exponential Hawkes processes with inhibition and application to neuronal activity

Gradient-based estimation of linear Hawkes processes with general kernels

First-Order Algorithms Without Lipschitz Gradient: A Sequential Local Optimization Approach

Contact Info

Product

Resources

About