Refined MDP-Based Branch-and-Fix Algorithm for the Hamiltonian Cycle Problem

Ejov, Vladimir; Filar, Jerzy A.; Haythorpe, Michael; Nguyen, Giang T.

doi:10.1287/moor.1090.0398

Cited by 14 publications

(18 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Ejov et al (2008b) proposed a branch-and-bound-type algorithm for the HCP by adapting linear constraints (5)-(7). The new hybrid algorithm explained in Sect.…”

Section: Hamiltonian Cycles Through Controlled Markov Chainsmentioning

confidence: 99%

“…Theorem 2.1 demonstrates that a convex subset of the usual "discounted frequency" polyhedral domain defined by constraints (5) and (7) reflects some properties of Hamiltonian policies and can be used as a basis of an algorithm that searches for a Hamiltonian cycle (e.g., see Feinberg 2000;Ejov et al 2008b). However, it can be argued that the polytope defined by (5), (6) and (7) has two deficiencies:…”

Section: H β -Refined Polyhedral Domainmentioning

confidence: 99%

See 1 more Smart Citation

A hybrid simulation-optimization algorithm for the Hamiltonian cycle problem

2009

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a new hybrid algorithm for the Hamiltonian cycle problem by synthesizing the Cross Entropy method and Markov decision processes. In particular, this new algorithm assigns a random length to each arc and alters the Hamiltonian cycle problem to the travelling salesman problem. Thus, there is now a probability corresponding to each arc that denotes the probability of the event "this arc is located on the shortest tour." Those probabilities are then updated as in cross entropy method and used to set a suitable linear programming model. If the solution of the latter yields any tour, the graph is Hamiltonian. Numerical results reveal that when the size of graph is small, say less than 50 nodes, there is a high chance the algorithm will be terminated in its cross entropy component by simply generating a Hamiltonian cycle, randomly. However, for larger graphs, in most of the tests the algorithm terminated in its optimization component (by solving the proposed linear program).

show abstract

“…Recently, Ejov et al (2008b) proposed a branch-and-bound-type algorithm for the HCP by adapting linear constraints (5)-(7). The new hybrid algorithm explained in Sect.…”

Section: Hamiltonian Cycles Through Controlled Markov Chainsmentioning

confidence: 99%

Section: H β -Refined Polyhedral Domainmentioning

confidence: 99%

A hybrid simulation-optimization algorithm for the Hamiltonian cycle problem

2009

Self Cite

View full text Add to dashboard Cite

show abstract

“…Most of the later developments are recapped in monographs by Kallenberg [35], Borkar [6], Piunovskiy [42], Altman [1], and Hernández-Lerma and Lasserre [32], and in surveys by Piunovskiy [43] and Borkar [7]. It has various applications including to the Hamiltonian cycle problem; see Feinberg [21], Filar [28, § §3.3, 3.4], and Ejov et al [15]. The convex-analytic approach is also applicable to average rewards per unit time with expected state-action frequencies playing the role of occupancy measures (Derman [12], Kallenberg [35], Borkar [6], Piunovskiy [42], Altman [1]).…”

mentioning

confidence: 99%

Splitting Randomized Stationary Policies in Total-Reward Markov Decision Processes

Feinberg

Rothblum

2012

Mathematics of OR

View full text Add to dashboard Cite

This paper studies a discrete-time total-reward Markov decision process (MDP) with a given initial state distribution. A (randomized) stationary policy can be split on a given set of states if the occupancy measure of this policy can be expressed as a convex combination of the occupancy measures of stationary policies, each selecting deterministic actions on the given set and coinciding with the original stationary policy outside of this set. For a stationary policy, necessary and sufficient conditions are provided for splitting it at a single state as well as sufficient conditions for splitting it on the whole state space. These results are applied to constrained MDPs. The results are refined for absorbing (including discounted) MDPs with finite state and actions spaces. In particular, this paper provides an efficient algorithm that presents the occupancy measure of a given policy as a convex combination of the occupancy measures of finitely many (stationary) deterministic policies. This algorithm generates the splitting policies in a way that each pair of consecutive policies differs at exactly one state. The results are applied to constrained problems to efficiently compute an optimal policy by computing and splitting a stationary optimal policy.Key words: Markov decision processes; occupancy measures; splitting occupancy measures; constrained Markov decision processes MSC2000 subject classification: Primary: 90C40; secondary: 97K60, 60J20, 60J22 OR/MS subject classification: Primary: dynamic programming/optimal control; secondary: deterministic Markov, finite state, infinite state History: Received April 25, 2008; revised December 11, 2010, and October 16, 2011. Published online in Articles in Advance January 9, 2012.1. Introduction. This paper is concerned with a discrete-time Markov decision process (MDP) with a given distribution of an initial state and with total-reward criteria. It investigates whether and how a stationary policy can be replaced by another policy that is defined as a random selection among policies that are deterministic on a prescribed set of states and coincide with the original stationary policy outside of that set. Contributions are presented for MDPs with finite state and actions sets, for MDPs with countable state sets, and for MDPs with Borel state and action spaces.An MDP is said to be absorbing if its expected lifetime is finite under every policy. In particular, a discounted MDP can be presented as an absorbing MDP. For an absorbing MDP with a fixed initial state distribution, the occupancy measure of a policy specifies the expectation of the number of visits to each measurable set of state-action pairs. The expected total reward for the policy can be expressed as the integral of the one-step reward function with respect to the policy's occupancy measure. Thus, optimizing the expected total rewards is reduced to optimizing a linear function over the set of occupancy measures. This is the basic idea of the convexanalytic approach which provides useful methods for solving MDPs w...

show abstract

“…Markov decision processes are applicable to a wide range of optimization problems. The model introduced in Filar and Krass [15] instigated a new line of research, which has attracted growing attention (see, for example, [4,7,8,10,9,11,14,16,20]).…”

Section: Introductionmentioning

confidence: 99%

“…Feinberg showed that if the graph G is Hamiltonian, the polytope F β (G) has an extreme point, called a Hamiltonian extreme point, for each of its Hamiltonian cycles. Subsequently, Ejov et al [10] described some geometric properties of F β (G) and Eshragh et al [13] transformed F β (G) to a polytope H β (G) to improve algorithmic efficiency. In 2011, Eshragh and Filar [12] partitioned all extreme points of H β (G) into five types, consisting of Hamiltonian extreme points and non-Hamiltonian extreme points of types 1, 2, 3 and 4.…”

Section: Introductionmentioning

confidence: 99%

Hamiltonian Cycles and Subsets of Discounted Occupational Measures

et al. 2020

Self Cite

View full text Add to dashboard Cite

We study a certain polytope arising from embedding the Hamiltonian cycle problem in a discounted Markov decision process. The Hamiltonian cycle problem can be reduced to finding particular extreme points of a certain polytope associated with the input graph. This polytope is a subset of the space of discounted occupational measures. We characterize the feasible bases of the polytope for a general input graph G, and determine the expected numbers of different types of feasible bases when the underlying graph is random. We utilize these results to demonstrate that augmenting certain additional constraints to reduce the polyhedral domain can eliminate a large number of feasible bases that do not correspond to Hamiltonian cycles. Finally, we develop a random walk algorithm on the feasible bases of the reduced polytope and present some numerical results. We conclude with a conjecture on the feasible bases of the reduced polytope.

show abstract

Refined MDP-Based Branch-and-Fix Algorithm for the Hamiltonian Cycle Problem

Cited by 14 publications

References 13 publications

A hybrid simulation-optimization algorithm for the Hamiltonian cycle problem

A hybrid simulation-optimization algorithm for the Hamiltonian cycle problem

Splitting Randomized Stationary Policies in Total-Reward Markov Decision Processes

Hamiltonian Cycles and Subsets of Discounted Occupational Measures

Contact Info

Product

Resources

About