Spectral FGS (SFGS), modified from MPEG-4 FGS, was proposed as a scalable video coding technique that has evener image quality and is more suitable for rate allocation for video smoothness. This paper proposed a unified rate allocation algorithm bascd on SFGS/SFGST to make good tradeuffs between image quality, video smoothness, and temporal scalability, according to users' preferences at the buffer fullness constraint. Our rate allocation algorithm was established on the dynamic programming (DP) technique. It becomes easy to tune the preference by adjusting weighting parameters in the optimized cost function. Experiments show that our algorithm is capable of achieving the same video smoothness as our previous work [8], having more diversity in mode selection between SFGS and SFGST, and guaranteeing no buffer overflow or underflow.