Large-scale multiple-input multiple-output (LS-MIMO) is considered as a promising key technology for future 5G wireless communications due to its very high spectrum and energy efficiency. However, one challenging problem to achieve these benefits is a practical signal detection algorithm in the uplink. In this paper, we propose a low-complexity near-optimal signal detection algorithm using the conjugate gradient (CG) method for uplink multi-user large-scale MIMO systems, which can avoid the complicated matrix inversion required by linear minimum mean square error (MMSE) signal detection algorithm. We also provide the convergence proof of the proposed scheme to guarantee its usability in practice. The analysis indicates that the proposed scheme can reduce the computational complexity by about one order of magnitude. The simulation results of the bit error rate performance verify that the proposed algorithm outperforms the recently proposed Neumann series approximation algorithm, and achieves the near-optimal performance of the classical MMSE algorithm by using only a small number of iterations.