We study the dynamics of vortex lattice formation of a rotating trapped Bose-Einstein condensate by numerically solving the two-dimensional Gross-Pitaevskii equation, and find that the condensate undergoes elliptic deformation, followed by unstable surface-mode excitations before forming a quantized vortex lattice. The origin of the peculiar surface-mode excitations is identified to be phase fluctuations at the low-density surface regime. The obtained dependence of a distortion parameter on time and that on the driving frequency agree with the recent experiments by Madison et al. [Phys. Rev. Lett. 86, 4443 (2001)].PACS numbers: 03.75. Fi, 67.40.Db Quantized vortices have long been studied in superfluid 4 He as the topological defects characteristic of superfluidity [1,2]. However, the relatively high density and strong repulsive interaction complicate the theoretical treatments of the Bose-Einstein condensed liquid, and the healing length of the atomic scale makes the experimental visualization of the quantized vortices difficult. The recent achievement of Bose-Einstein condensation in trapped alkali-metal atomic gases at ultra low temperatures has stimulated intense experimental and theoretical activity. The atomic Bose-Einstein condensates(BECs) have the weak interaction because they are dilute gases, thus being free of the above difficulties that superfluid 4 He is subject to. Quantized vortices in the atomic BECs have recently been created experimentally by Matthews et al. By rotating an asymmetric trapping potential, Madison et al. at ENS succeeded in forming a quantized vortex in 87 Rb BEC for a stirring frequency that exceeds a critical value [4]. Vortex lattices were obtained for higher frequencies. The ENS group subsequently observed that vortex nucleation occurs via a dynamical instability of the condensate [6]. For a given modulation amplitude and stirring frequency, the steady state of the condensate was distorted to an elliptic cloud, stationary in the rotating frame, as predicted by Recati et al. [7]. An intrinsic dynamical instability [8] of the steady state transformed the elliptic state into a more axisymmetric state with vortices. However, the origin of that instability and how it leads to the formation of vortex lattices remains to be investigatedThe ENS group found that the minimum rotation frequency Ω nuc at which one vortex appears is 0.65ω ⊥ , where ω ⊥ is the transverse oscillation frequency of the cigar-shaped trapping potential, independent of the number of atoms or the longitudinal frequency ω z [9]. There is a discrepancy between the observations and the theoretical considerations based on the stationary solution of the Gross-Pitaevskii equation(GPE) [10,11]; Ω nuc is significantly larger than its equilibrium estimates.The present paper addresses these issues by numerically solving the GPE that governs the time evolution of the order parameter ψ(r, t):Here g = 4πh 2 a/m is the coupling constant, proportional to the 87 Rb scattering length a ≈5.77 nm. The high anisotropy of the cig...