“…Zero-sum Markov game has been widely studied since the seminal work [Shapley, 1953]. When the transition kernel is unknown, different sampling oracles are utilized to acquire samples, including online sampling , Xie et al, 2020a, Liu et al, 2021, Jin et al, 2021a, Song et al, 2021, generative model sampling [Sidford et al, 2020, Cui and Yang, 2020, Zhang et al, 2020, Jia et al, 2019. For offline sampling oracle, Zhang et al [2021b] provides finite sample bound for a decentralized algorithm with network communication under uniform concentration assumption and Abe and Kaneko [2020] considers offline policy evaluation, again under the uniform concentration assumption.…”