Stochastic Gradient Methods for Unconstrained Optimization

Krejić, Nataša; Jerinkic, Natasa Krklec

doi:10.1590/0101-7438.2014.034.03.0373

Cited by 3 publications

(3 citation statements)

References 33 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For solving Equation ( 1 ), a simple optimisation (binary search) algorithm is given in [ 1 ] and is available as part of the software URFinder 3.7 available from the author; see Supplementary Materials . Gradient decent may also be applicable, advanced optimisation methods are described in [ 13 , 14 , 15 ], and computationally faster surrogates for

can be obtained using machine learning.…”

Section: Methodsmentioning

confidence: 99%

From Learning to Consciousness: An Example Using Expected Float Entropy Minimisation

Mason

2019

Entropy

View full text Add to dashboard Cite

Over recent decades several mathematical theories of consciousness have been put forward including Karl Friston’s Free Energy Principle and Giulio Tononi’s Integrated Information Theory. In this article we further investigate theory based on Expected Float Entropy (EFE) minimisation which has been around since 2012. EFE involves a version of Shannon Entropy parameterised by relationships. It turns out that, for systems with bias due to learning, certain choices for the relationship parameters are isolated since giving much lower EFE values than others and, hence, the system defines relationships. It is proposed that, in the context of all these relationships, a brain state acquires meaning in the form of the relational content of the associated experience. EFE minimisation is itself an association learning process and its effectiveness as such is tested in this article. The theory and results are consistent with the proposition of there being a close connection between association learning processes and the emergence of consciousness. Such a theory may explain how the brain defines the content of consciousness up to relationship isomorphism.

show abstract

can be obtained using machine learning.…”

Section: Methodsmentioning

confidence: 99%

From Learning to Consciousness: An Example Using Expected Float Entropy Minimisation

Mason

2019

Entropy

View full text Add to dashboard Cite

show abstract

“…Furthermore, ( 1) is an optimisation problem and falls within a common general class of problems studied in optimisation theory for which a number of efficient algorithms are available. These involve, gradient methods, stochastic gradient methods and derivative free optimisation; see [19], [20] and [21]. Symbol Description a, b, c, .…”

Section: A Comparison Of Float Entropy and Shannon Entropymentioning

confidence: 99%

Quasi‐conscious multivariate systems

Mason

2015

Complexity

View full text Add to dashboard Cite

Conscious experience is awash with underlying relationships. Moreover, for various brain regions such as the visual cortex, the system is biased toward some states. Representing this bias using a probability distribution shows that the system can define expected quantities. The mathematical theory in this article links these facts using expected float entropy (efe), which is a measure of the expected amount of information needed, to specify the state of the system, beyond what is already known about the system from relationships that appear as parameters. Under the requirement that the relationship parameters minimize efe, the brain defines relationships. It is proposed that when a brain state is interpreted in the context of these relationships the brain state acquires meaning in the form of the relational content of the associated experience. For a given set, the theory represents relationships using weighted relations which assign continuous weights, from 0 to 1, to the elements of the Cartesian product of that set. The relationship parameters include weighted relations on the nodes of the system and on their set of states. Examples obtained using Monte-Carlo methods (where relationship parameters are chosen uniformly at random) suggest that efe distributions with long left tails are most important.

show abstract

“…One can distinguish between two main approaches in the sample size scheduling -a predetermined sample size schedule, for example [12] or an adaptive sample size schedule, [6,11,13]. An overview of different sample size scheduling is presented in [7].…”

Section: Introductionmentioning

confidence: 99%

A nonmonotone line search method for stochastic optimization problems

Krejić¹,

Lončar²

2018

Filomat

View full text Add to dashboard Cite

A nonmonotone line search method for solving unconstrained optimization problems with the objective function in the form of mathematical expectation is proposed and analyzed. The method works with approximate values of the objective function obtained with increasing sample sizes and improves accuracy gradually. Nonmonotone rule significantly enlarges the set of admissible search directions and prevents unnecessarily small steps at the beginning of the iterative procedure. The convergence is shown for any search direction that approaches the negative gradient in the limit. The convergence results are obtained in the sense of zero upper density. Initial numerical results confirm theoretical results and show efficiency of the proposed approach.

show abstract

Stochastic Gradient Methods for Unconstrained Optimization

Cited by 3 publications

References 33 publications

From Learning to Consciousness: An Example Using Expected Float Entropy Minimisation

From Learning to Consciousness: An Example Using Expected Float Entropy Minimisation

Quasi‐conscious multivariate systems

A nonmonotone line search method for stochastic optimization problems

Contact Info

Product

Resources

About