In order to improve the energy dissipation of the masonry infilled frame structure while decreasing the stiffening and strengthening effects of the infill panels, a new dry stacked panel (DSP) semi-interlocking masonry (SIM) infill panel has been developed. In this paper, the material properties of DSP and a traditional unreinforced masonry (URM) panel have been evaluated experimentally. A series of cyclic tests were performed to investigate the cyclic behaviour of the reinforcement concrete (RC) frame with different infill panels. The failure modes, damage evolution, hysteretic behaviour, stiffness degradation and energy dissipation were compared and analysed. We concluded that DSP is capable of significantly improving the seismic energy dissipation due to its hysteretic behaviour when the frame is in elastic stage without increasing the stiffness of the frame. Therefore, DSP or SIM panels can be considered as frictional dampers. Based on the experimental results, the influence of DSP was examined. Using the parallel model, the hysteretic loops of DSP subjected to different load cases were achieved. The typical full hysteretic loop for DSP could be divided into three distinct stages of behaviour: packing stage, constant friction stage and equivalent strut stage. The connection between the panel and the frame had a great effect on the transferring of different mechanical stages. The constant friction stage was verified to provide substantial energy dissipation and benefits to the ductility of the structure, which, therefore, is suggested to be prolonged in reality.