Noise remains the major obstacle to scalable quantum computation. Quantum benchmarking
provides key information on noise properties and is an important step for developing more advanced
quantum processors. However, current benchmarking methods are either limited to a specific subset
of quantum gates or cannot directly describe the performance of the individual target gate. To
overcome these limitations, we propose channel spectrum benchmarking (CSB), a novel method to
infer the noise properties of the target gate, including process fidelity, stochastic fidelity, and some
unitary parameters, from the eigenvalues of its noisy channel. Our CSB method is insensitive to
state-preparation and measurement errors, and importantly, can benchmark universal gates and
is scalable to many-qubit systems. Unlike standard randomized schemes, CSB can provide direct
noise information for both target native gates and circuit fragments, allowing benchmarking and
calibration of frequently used modules in quantum algorithms like Trotterized Hamiltonian evolution
operator in quantum simulation.