Researchers have used distributed constraint optimization problems (DCOPs) to model various multi-agent coordination and resource allocation problems. Very recently, Ottens et al. proposed a promising new approach to solve DCOPs that is based on confidence bounds via their Distributed UCT (DUCT) sampling-based algorithm. Unfortunately, its memory requirement per agent is exponential in the number of agents in the problem, which prohibits it from scaling up to large problems. Thus, in this article, we introduce two new sampling-based DCOP algorithms called Sequential Distributed Gibbs (SD-Gibbs) and Parallel Distributed Gibbs (PD-Gibbs). Both algorithms have memory requirements per agent that is linear in the number of agents in the problem. Our empirical results show that our algorithms can find solutions that are better than DUCT, run faster than DUCT, and solve some large problems that DUCT failed to solve due to memory limitations.
We address the problem of maritime traffic management in busy waterways to increase the safety of navigation by reducing congestion. We model maritime traffic as a large multiagent systems with individual vessels as agents, and VTS authority as the regulatory agent. We develop a maritime traffic simulator based on historical traffic data that incorporates realistic domain constraints such as uncertain and asynchronous movement of vessels. We also develop a traffic coordination approach that provides speed recommendation to vessels in different zones. We exploit the nature of collective interactions among agents to develop a scalable policy gradient approach that can scale up to real world problems. Empirical results on synthetic and real world problems show that our approach can significantly reduce congestion while keeping the traffic throughput high.
Researchers have introduced the Dynamic Distributed Constraint Optimization Problem (Dynamic DCOP) formulation to model dynamically changing multi-agent coordination problems, where a dynamic DCOP is a sequence of (static canonical) DCOPs, each partially different from the DCOP preceding it. Existing work typically assumes that the problem in each time step is decoupled from the problems in other time steps, which might not hold in some applications. Therefore, in this paper, we make the following contributions: (i) We introduce a new model, called Markovian Dynamic DCOPs (MD-DCOPs), where the DCOP in the next time step is a function of the value assignments in the current time step; (ii) We introduce two distributed reinforcement learning algorithms, the Distributed RVI Q-learning algorithm and the Distributed R-learning algorithm, that balance exploration and exploitation to solve MD-DCOPs in an online manner; and (iii) We empirically evaluate them against an existing multi-arm bandit DCOP algorithm on dynamic DCOPs.
Multiagent sequential decision making has seen rapid progress with formal models such as decentralized MDPs and POMDPs. However, scalability to large multiagent systems and applicability to real world problems remain limited. To address these challenges, we study multiagent planning problems where the collective behavior of a population of agents affects the joint-reward and environment dynamics. Our work exploits recent advances in graphical models for modeling and inference with a population of individuals such as collective graphical models and the notion of finite partial exchangeability in lifted inference. We develop a collective decentralized MDP model where policies can be computed based on counts of agents in different states. As the policy search space over counts is combinatorial, we develop a sampling based framework that can compute open and closed loop policies. Comparisons with previous best approaches on synthetic instances and a real world taxi dataset modeling supply-demand matching show that our approach significantly outperforms them w.r.t. solution quality.
The Vedder (Oligocene) and Kreyenhagen (Eocene) sandstones at the Greeley oil field consist of arkosic to subarkosic arenites and wackes deposited in shallow marine environments. Burial depths of the Vedder sandstones exceed 3150 m and the reservoir temperature is 124°C. The Kreyenhagen sandstones are buried to greater than 3920 m and the reservoir temperature is estimated to be c. 135°C. These sandstones are currently at or very near their deepest burial depths.The textural relationships of the diagenetic minerals suggest syndepositional formation of glauconite, phosphate and pyrite, followed by early precipitation of pore-lining clay coatings and carbonate cements along with framework-grain fracturing and possibly dissolution. With increasing burial, dissolution of the framework grains continued, accompanied by the albitization of feldspars, the formation of K-feldspar and quartz overgrowths, the precipitation of kaolinite and other clays and possibly the precipitation of late carbonate cements. Finally, hydrocarbon migration and the formation of pyrite occurred during late diagenesis.Porosity preservation and reservoir quality are primarily the result of plagioclase dissolution occurring as the strata approached their current burial depths. Mass balance calculations indicate the significant export of aluminium out of the sands. Thus secondary porosity produced by plagioclase dissolution has replaced the primary porosity destroyed by compaction, and now accounts for the majority of the porosity in these rocks.
Abstract-Unprecedented pace of urbanization and rising income levels have fueled the growth of car ownership in almost all newly formed megacities. Such growth has congested the limited road space and significantly affected the quality of life in these megacities. Convincing residents to give up their cars and use public transport is the most effective way in reducing congestion; however, even with sufficient public transport capacity, the lack of last-mile (from the transport hub to the destination) travel services is the major deterrent for the adoption of public transport. Due to the dynamic nature of such travel demands, fixed-size fleets will not be a cost-effective approach in addressing last-mile demands. Instead, we propose a dynamic, incentive-based mechanism that enables taxi ridesharing for satisfying last-mile travel demands. On the demand side, travelers would register their last-mile travel demands in real-time, and they are expected to receive ride arrangements before they reach the hub; on the supply side, depending on the real-time demands, proper incentives will be computed and provided to taxi drivers willing to commit to the lastmile service. Multiple travelers will be clustered into groups according to their destinations, and travelers belonging to the same group will be assigned to a taxi, while each of them paying fares considering their destinations and also their orders in reaching destinations. In this paper, we provide mathematical formulations for demand clustering and fare distribution. If the model returns a solution, it is guaranteed to be implementable. For cases where it is not possible to satisfy all demands despite having enough capacity, we propose a two-phase approach that identifies the maximal subset of riders that can be feasibly served. Finally, we use a series of numerical examples to demonstrate the effectiveness of our approach.
Urban law enforcement agencies are under great pressure to respond to emergency incidents effectively while operating within restricted budgets. Minutes saved on emergency response times can save lives and catch criminals, and a responsive police force can deter crime and bring peace of mind to citizens. To efficiently minimize the response times of a law enforcement agency operating in a dense urban environment with limited manpower, we consider in this paper the problem of optimizing the spatial and temporal deployment of law enforcement agents to predefined patrol regions in a real-world scenario informed by machine learning. To this end, we develop a mixed integer linear optimization formulation (MIP) to minimize the risk of failing response time targets. Given the stochasticity of the environment in terms of incident numbers, location, timing, and duration, we use Sample Average Approximation (SAA) to find a robust deployment plan. To overcome the sparsity of real data, samples are provided by an incident generator that learns the spatio-temporal distribution and demand parameters of incidents from a real world historical dataset and generates sets of training incidents accordingly. To improve runtime performance across multiple samples, we implement a heuristic based on Iterated Local Search (ILS), as the solution is intended to create deployment plans quickly on a daily basis. Experimental results demonstrate that ILS performs well against the integer model while offering substantial gains in execution time.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.