Graphene with linear energy dispersion and weak electron-phonon interaction is highly anticipated to harvest hot-electrons in a broad wavelength range from ultraviolet to terahertz. However, the limited absorption (~2.3%) and serious backscattering of hot-electrons associated with single-layer graphene result in inadequate quantum yields, impeding their practically broadband photodetection, especially in the mid-infrared range. Here, we report a macroscopic assembled graphene (MAG)/silicon heterojunction for ultrafast mid-infrared photodetection. The highly crystalline 2-inch scale MAG with tunable thickness from 10 to 60 nm is produced by scalable wet-assembly of commercial graphene oxide followed by thermal annealing. The MAG/Si Schottky diode exhibits broadband photodetection capability in 1-10 μm at room temperature with fast response (120-130 ns, 4 mm2 window) and high detectivity (1011 to 106 Jones), outperforming single-layer graphene/Si photodetectors by 2 to 8 orders in transient photocurrent. This optoelectronic performance is attributed to the superior advantages of MAG (~40% of light absorption, ~23 ps of carrier relaxation time, and high quasi-equilibrated hot-carrier-multiplication gain), atomic-scale contact interface of MAG and silicon, and impact-ionization avalanche gain (~100 times) from silicon. The MAG provides a long-range platform to understand the hot-carrier dynamics in stacked 2D materials, leading to next-generation broadband silicon-based image sensors.