In the present study, an unstructured mixed mesh flow solver was used to conduct a numerical prediction of the aerodynamic performance of the S-76 rotor in hover. For the present mixed mesh methodology, the near-body flow domain was modeled by using body-fitted prismatic/tetrahedral cells while Cartesian mesh cells were filled in the off-body region. A high-order accurate weighted essentially non-oscillatory (WENO) scheme was employed to better resolve the flow characteristics in the off-body flow region. An overset mesh technique was adopted to transfer the flow variables between the two different mesh regions, and computations were carried out for three different blade configurations including swept-taper, rectangular, and swept-taper-anhedral tip shapes. The results of the simulation were compared against experimental data, and the computations were also made to investigate the effect of the blade tip Mach number. The detailed flow characteristics were also examined, including the tip-vortex trajectory, vortex core size, and first-passing tip vortex position that depended on the tip shape.