This paper considers the problem of computing an optimal policy for a Markov decision process, under lack of complete a priori knowledge of (1) the branching probability distributions determining the evolution of the process state upon the execution of the different actions, and (2) the probability distributions characterizing the immediate rewards returned by the environment as a result of the execution of these actions at different states of the process. In addition, it is assumed that the underlying process evolves in a repetitive, episodic manner, with each episode starting from a well-defined initial state and evolving over an acyclic state space. A novel efficient algorithm for this problem is proposed, and its convergence properties and computational complexity are rigorously characterized in the formal framework of computational learning theory. Furthermore, in the process of deriving the aforementioned results, the presented work generalizes Bechhofer's "indifference-zone" approach for the ranking & selection problem, that arises in statistical inference theory, so that it applies to populations with bounded general distributions.

No abstract

The original definition of the problem of optimal node visitation (ONV) in acyclic stochastic digraphs concerns the identification of a routing policy that will enable the visitation of each leaf node a requested number of times, while minimizing the expected number of the graph traversals.The original work of [1] formulated this problem as a Stochastic Shortest Path (SSP) problem, and since the state space of this SSP formulation is exponentially sized with respect to the number of the target nodes, it also proposed a suboptimal policy that is computationally tractable and asymptotically optimal. This paper extends the results of [1] to the cases where (i) the tokens traversing the graph can "split" during certain transitions to a number of (sub-)tokens, allowing, thus, the satisfaction of many visitation requirements during a single graph traversal, and (ii) there are additional visitation requirements attached to the internal graph nodes, which, however, can be served only when the visitation requirements of their successors have been fully met. In addition, the presented set of results establishes stronger convergence properties for the proposed suboptimal policies, and it provides a formal complexity analysis of the considered ONV formulations. From a practical standpoint, the extension of the original results performed in this paper enables their effective usage in the application domains that motivated the ONV problem, in the first place.

No abstract

The original definition of the problem of optimal node visitation (ONV) in acyclic stochastic digraphs concerns the identification of a routing policy that will enable the visitation of each leaf node a requested number of times, while minimizing the expected number of the graph traversals.The original work of [1] formulated this problem as a Stochastic Shortest Path (SSP) problem, and since the state space of this SSP formulation is exponentially sized with respect to the number of the target nodes, it also proposed a suboptimal policy that is computationally tractable and asymptotically optimal. This paper extends the results of [1] to the cases where (i) the tokens traversing the graph can "split" during certain transitions to a number of (sub-)tokens, allowing, thus, the satisfaction of many visitation requirements during a single graph traversal, and (ii) there are additional visitation requirements attached to the internal graph nodes, which, however, can be served only when the visitation requirements of their successors have been fully met. In addition, the presented set of results establishes stronger convergence properties for the proposed suboptimal policies, and it provides a formal complexity analysis of the considered ONV formulations. From a practical standpoint, the extension of the original results performed in this paper enables their effective usage in the application domains that motivated the ONV problem, in the first place.

The original definition of the problem of optimal node visitation (ONV) in acyclic stochastic digraphs concerns the identification of a routing policy that will enable the visitation of each leaf node a requested number of times, while minimizing the expected number of the graph traversals.The original work of [1] formulated this problem as a Stochastic Shortest Path (SSP) problem, and since the state space of this SSP formulation is exponentially sized with respect to the number of the target nodes, it also proposed a suboptimal policy that is computationally tractable and asymptotically optimal. This paper extends the results of [1] to the cases where (i) the tokens traversing the graph can "split" during certain transitions to a number of (sub-)tokens, allowing, thus, the satisfaction of many visitation requirements during a single graph traversal, and (ii) there are additional visitation requirements attached to the internal graph nodes, which, however, can be served only when the visitation requirements of their successors have been fully met. In addition, the presented set of results establishes stronger convergence properties for the proposed suboptimal policies, and it provides a formal complexity analysis of the considered ONV formulations. From a practical standpoint, the extension of the original results performed in this paper enables their effective usage in the application domains that motivated the ONV problem, in the first place.

Additive manufacturing methods enable the rapid fabrication of fully functional customized objects with complex geometry and lift the limitations of traditional manufacturing techniques, such as machining. Therefore, the structural optimization of parts has concentrated increased scientific interest and more especially for topology optimization (TO) processes. In this paper, the working principles and the two approaches of the TO procedures were analyzed along with an investigation and a comparative study of a novel case study for the TO processes of a tibial implant designed for additive manufacturing (DfAM). In detail, the case study focused on the TO of a tibial implant for knee replacement surgery in order to improve the overall design and enhance its efficiency and the rehabilitation process. An initial design of a customized tibial implant was developed utilizing reserve engineering procedures with DICOM files from a CT scan machine. The mechanical performance of the designed implant was examined via finite element analyses (FEA) under realistic static loads. The TO was conducted with two distinct approaches, namely density-based and discrete-based, to compare them and lead to the best approach for biomechanical applications. The overall performance of each approach was evaluated through FEA, and its contribution to the final mass reduction was measured. Through this study, the maximum reduction in the implant’s mass was achieved by maintaining the mechanical performance at the desired levels and the best approach was pointed out. To conclude, with the discrete-based approach, a mass reduction of around 45% was achieved, almost double of the density-based approach, offering on the part physical properties which provide comprehensive advantages for biomechanical application.

This paper introduces a novel optimal flow control problem that seeks to convey a specified amount of fluid to each of the nodes of an acyclic digraph with a single source node, while minimizing the total amount of fluid inducted into the network. Two factors complicating the aforementioned task are (i) the presence of nodes with uncontrollable routing of the traversing flow and (ii) a set of precedence constraints regarding the satisfaction of the nodal fluid requirements. It is shown that the considered problem can be naturally formulated as a continuous-time optimal control problem that can be reduced to a hybrid optimal control problem with controlled switching. This property subsequently enables the solution of the considered problem through a Mixed Integer Programming formulation. Additional results in the paper establish the NP-hardness of the considered problem, highlight its affinity to some well known scheduling problems, and offer guidelines that can alleviate the increased problem complexity.