Previous studies suggest that the growth of supermassive black holes (SMBHs) may be fundamentally related to host-galaxy stellar mass (M ). To investigate this SMBH growth-M relation in detail, we calculate long-term SMBH accretion rate as a function of M and redshift [BHAR(M , z)] over ranges of log(M /M ) = 9.5-12 and z = 0.4-4. Our BHAR(M , z) is constrained by high-quality survey data (GOODS-South, GOODS-North, and COSMOS), and by the stellar mass function and the X-ray luminosity function. At a given M , BHAR is higher at high redshift. This redshift dependence is stronger in more massive systems (for log(M /M ) ≈ 11.5, BHAR is three decades higher at z = 4 than at z = 0.5), possibly due to AGN feedback. Our results indicate that the ratio between BHAR and average star formation rate (SFR) rises toward high M at a given redshift. This BHAR/SFR dependence on M does not support the scenario that SMBH and galaxy growth are in lockstep. We calculate SMBH mass history [M BH (z)] based on our BHAR(M , z) and the M (z) from the literature, and find that the M BH -M relation has weak redshift evolution since z ≈ 2. The M BH /M ratio is higher toward massive galaxies: it rises from ≈ 1/5000 at log M 10.5 to ≈ 1/500 at log M 11.2. Our predicted M BH /M ratio at high M is similar to that observed in local giant ellipticals, suggesting that SMBH growth from mergers is unlikely to dominate over growth from accretion.