The role of the exchange-correlation potential and the exchange-correlation kernel in the calculation of excitation energies from time-dependent density functional theory is studied. Excitation energies of the helium and beryllium atoms are calculated, both from the exact Kohn-Sham ground-state potential, and from two orbital-dependent approximations. These are exact exchange and self-interaction corrected local density approximation (SIC-LDA), both calculated using Krieger-Li-Iafrate approximation. For the exchangecorrelation kernels, three adiabatic approximations were tested: the local density approximation, exact exchange, and SIC-LDA. The choice of the ground-state exchange correlation potential has the largest impact on the absolute position of most excitation energies. In particular, orbital-dependent approximate potentials result in a uniform shift of the transition energies to the Rydberg states.