This paper deals with a nonlinear optimal control approach to helicopter inverse simulation. The reference trajectory is prescribed in prior, and the integral deviation from this trajectory is treated as an additional penalty cost to convert the system optimality to an unconstrained optimal control problem. The resultant two-point boundary value problem has been solved by a multiple-shooting algorithm. The nonlinear helicopter model in this study includes main rotor flap dynamics and a dynamic inflow model. The applications cover the inverse simulation for bob up, turn, and slalom maneuvers. This paper focuses on resolving the convergence issue using the indirect method, the main root causes of which are related to the inherent system instability of the helicopter and with poor initial guesses on state and costate variables. For this reason we will investigate the effect of the shooting node number on convergence and use a hybrid-model approach, where the optimal state and costate variables, calculated using the linear model, are used as initial guesses for those using the nonlinear model. The analyses show good convergence history and capability of tracking the prescribed trajectory. So the results in this paper can provide a valuable motivation for applying indirect methods to nonlinear helicopter flight mechanic analysis.