The process of unstable mass transfer in a stellar binary can result in either a complete merger of the stars or successful removal of the donor envelope leaving a surviving more compact binary. "Luminous red nova" (LRN) are the class of optical transients believed to accompany such merger/common envelope events. Past works typically model LRNe using analytic formulae for supernova light curves which make assumptions (e.g., radiation dominated ejecta, neglect of hydrogen recombination energy) not justified in stellar mergers due to the lower velocities and specific thermal energy of the ejecta. We present a one-dimensional model of LRN light curves, which accounts for these effects. Consistent with observations, we find that LRNe typically possess two light curve peaks, an early phase powered by initial thermal energy of the hot, fastest ejecta layers and a later peak powered by hydrogen recombination from the bulk of the ejecta. We apply our model to a sample of LRNe to infer their ejecta properties (mass, velocity, and launching radius) and compare them to the progenitor donor star properties from pre-transient imaging. We define a maximum luminosity achievable for a given donor star in the limit that the entire envelope is ejected, finding that several LRNe violate this limit. Shock interaction between the ejecta and pre-dynamical mass-loss, may provide an additional luminosity source to alleviate this tension. Our model can also be applied to the merger of planets with stars or stars with compact objects.