Fiber-reinforced polymer (FRP) composites have been used in various industries, thus a large amount of FRP wastes have been generated due to the out-of-service of FRP products. Recycling FRP wastes into coarse aggregates to replace natural coarse aggregates (NCA) to form the recycled FRP aggregate concrete (RFAC) is a potential approach to dispose of huge quantities of FRP wastes with low environmental impact. In this paper, waste glass FRP (GFRP) bars were cut into particles of 12 sizes to enable the grading of recycled FRP aggregate (RFA) as similar as possible to that of NAC. The influence of different RFA volume replacement ratios (0%, 30%, 50%, 70%, 100%) on the compressive performance of RFAC was investigated based on uniaxial compression tests of 15 standard cylinders. The results showed that the failure mode of RFAC was different from that of NAC. As the RFA replacement ratio increased, the compressive strength and elastic modulus of the RFAC gradually decreased, but its post-peak brittleness was significantly mitigated compared to NAC. The Poisson’s ratio of RFAC increased with the increase in the RGFA replacement ratio at the elastic stage and was smaller than that of NCA concrete. Both the existing stress–strain models developed for NAC and recycled aggregate concrete (RAC) were found not fit for the RFAC. Thus, a new stress–strain model that was applicable to RFAC was developed by modifying the classical existing model, and a good agreement between the model predictions and test data was reached.