The use of acoustic levitation in the fields of analytical chemistry and in the containerless processing of materials requires a good stability of the levitated particle. However, spontaneous oscillations and rotation of the levitated particle have been reported in literature, which can reduce the applicability of the acoustic levitation technique. Aiming to reduce the particle oscillations, this paper presents the analysis of the particle stability in a new acoustic levitator device. The new acoustic levitator consists of a piezoelectric transducer with a concave radiating surface and a concave reflector. The analysis is conducted by determining numerically the axial and lateral forces that act on the levitated object and by measuring the oscillations of a sphere particle by a laser Doppler vibrometer. It is shown that the new levitator design allows to increase the lateral forces and reduce significantly the lateral oscillations of the levitated object.