Effects of substituting of wheat flour with coffee cherry pulp powder (CCPP) (coffee by-product as fiber source) at 0, 1.2, 2.3, and 4.7% dry basis (0, 1.25, 2.5, and 5% wet basis) on dough and gluten rheological properties and baking quality were investigated. Rheological properties were analyzed during mixing, compression recovery, and creep-recovery. A rheological approach was adopted to study the viscoelasticity of dough enriched with fiber. The data obtained were analyzed with the Kelvin–Voigt model and the parameters were correlated to bread volume and crumb firmness to assess the effect of incorporating CCPP. A decrease in gluten’s elastic properties was attributed to the water-binding and gelling properties of CCPP. Stiffness of dough and crumb firmness increased as the level of CCPP increased and bread volume decreased. Stiffer dough corresponded with lower compliance values and higher steady state viscosity compared to the control. A follow-up study with 5% CCPP and additives is recommended to overcome the reduction in elastic recovery and bread volume.