Flight departure delays cost airlines and airports millions of dollars and become a systematic problem. The on-time performance at an airport is connected to and easily affected by delay propagation from previous operations of flights using the airport. In this paper, we employ both Ordinary Least Square (OLS) and quantile regressions to investigate the impact of various influencing factors on flight departure delay. By using historical flight records and weather information, the impacts of delay propagation-related and other factors are quantified to study the correlations between the explanatory and response variables. Three variables, including previous arrival delay, turnaround buffer time, and the first order of a day, are used to examine the propagation effects. We find that aircraft type, flying on a weekday, and being the first flight of a day have significant impacts on short departure delays. Ground buffer is conducive to mitigating delay propagation. For long delays, however, ground buffer cannot work in an efficient way, and the previous arrival effect is more important. Convective weather and aircraft type are the crucial factors in this situation. Interestingly, flying on a weekday suddenly becomes one of the main components under extreme delays. Meanwhile, propagated delay and airport congestion remain significantly impactful on the on-time performance.