Deep learning models have significantly improved the visual quality and accuracy on compressive sensing recovery. In this paper, we propose an algorithm for signal reconstruction from compressed measurements with image priors captured by a generative model. We search and constrain on latent variable space to make the method stable when the number of compressed measurements is extremely limited. We show that, by exploiting certain structures of the latent variables, the proposed method produces improved reconstruction accuracy and preserves realistic and non-smooth features in the image. Our algorithm achieves high computation speed by projecting between the original signal space and the latent variable space in an alternating fashion.

We develop a mathematical framework for solving multi-task reinforcement learning problems based on a type of decentralized policy gradient method. The goal in multi-task reinforcement learning is to learn a common policy that operates effectively in different environments; these environments have similar (or overlapping) state and action spaces, but have different rewards and dynamics. Agents immersed in each of these environments communicate with other agents by sharing their models (i.e. their policy parameterizations) but not their state/reward paths. Our analysis provides a convergence rate for a consensus-based distributed, entropy-regularized policy gradient method for finding such a policy. We demonstrate the effectiveness of the proposed method using a series of numerical experiments. These experiments range from small-scale "Grid World" problems that readily demonstrate the trade-offs involved in multi-task learning to large-scale problems, where common policies are learned to play multiple Atari games or to navigate an airborne drone in multiple (simulated) environments.Preprint. Under review.

With the wide adoption of mobile devices, it becomes increasingly important to understand how users use mobile apps. Knowing when and where certain apps are used is instrumental for app developers to improve app usability and for Internet service providers (ISPs) to optimize their network services. However, modeling spatio-temporal patterns of app usage has been a challenging problem due to the complicated usage behavior and the very limited personal data. In this paper, we propose a Bayesian mixture model to capture when, where and what apps are used and predict future app usage. To solve the challenge of data sparsity, we apply a hierarchical Dirichlet process to leverage the shared spatio-temporal patterns to accurately model users with insufficient data. We then evaluate our model using a large dataset of app usage traces involving 1.7 million users over 3503 apps. Our analysis shows a clear correlation between the user's location and the apps being used. Extensive evaluations show that our model can accurately predict users' future locations and app usage, outperforming the state-of-the-art algorithms by 11.7% and 11.1%, respectively. In addition, our model can be used to synthesize app usage traces that do not leak user privacy while preserving the key data statistical properties.

Stochastic approximation, a data-driven approach for finding the fixed point of an unknown operator, provides a unified framework for treating many problems in stochastic optimization and reinforcement learning. Motivated by a growing interest in multi-agent and multi-task learning, we consider in this paper a decentralized variant of stochastic approximation. A network of agents, each with their own unknown operator and data observations, cooperatively find the fixed point of the aggregate operator. The agents work by running a local stochastic approximation algorithm using noisy samples from their operators while averaging their iterates with their neighbors' on a decentralized communication graph. Our main contribution provides a finite-time analysis of this decentralized stochastic approximation algorithm and characterizes the impacts of the underlying communication topology between agents. Our model for the data observed at each agent is that it is sampled from a Markov processes; this lack of independence makes the iterates biased and (potentially) unbounded. Under mild assumptions on the Markov processes, we show that the convergence rate of the proposed methods is essentially the same as if the samples were independent, differing only by a log factor that represents the mixing time of the Markov process. We also present applications of the proposed method on a number of interesting learning problems in multi-agent systems, including a decentralized variant of Q-learning for solving multi-task reinforcement learning.

Multi-task reinforcement learning (RL) aims to find a single policy that effectively solves multiple tasks at the same time. This paper presents a constrained formulation for multitask RL where the goal is to maximize the average performance of the policy across tasks subject to bounds on the performance in each task. We consider solving this problem both in the centralized setting, where information for all tasks is accessible to a single server, and in the decentralized setting, where a network of agents, each given one task and observing local information, cooperate to find the solution of the globally constrained objective using local communication.We first propose a primal-dual algorithm that provably converges to the globally optimal solution of this constrained formulation under exact gradient evaluations. When the gradient is unknown, we further develop a sampled-based actor-critic algorithm that finds the optimal policy using online samples of state, action, and reward. Finally, we study the extension of the algorithm to the linear function approximation setting. IntroductionMulti-task reinforcement learning (RL) aims to find a common policy that effectively solves a range of tasks simultaneously, where each task is the policy optimization problem defined over a Markov decision process (MDP). The MDPs can have different state spaces, reward functions, and transition kernels in general, but may be implicitly or explicitly correlated.The most common mathematical formulation for multi-task RL is to maximize the average cumulative rewards collected by a single policy across all MDPs Zeng et al. ( 2021); Jiang et al. (2022); Junru et al. (2022). In this paper, we study a generalized formulation in which we maximize the average cumulative rewards subject to constraints on the performance of the policy for each task. This formulation is a special case of the policy optimization problem for a constrained Markov decision process (CMDP) Altman (1999) and is a flexible framework that allows more fine-grained specification of the performance of the optimal policy in each task. In applications where the tasks exhibit major conflicts of interest and/or the magnitude of the rewards varies significantly across tasks Kalashnikov et al. (2021); Guo et al. (2022), the optimal policy under the average-cumulative-reward formulation may excel in some tasks at the cost of compromised performance in others Hayes et al. (2022). The constrained formulation provides a way to mitigate this task imbalance. Illustrative numerical simulations are given in Section 7.Under the constrained multi-task formulation, we consider centralized and decentralized learning paradigms. "Centralized" in this context means that information of all tasks is available at a single server, while "decen-