Aiming at the problem that the trajectory direction cannot be distinguished from the historical data of ship Automatic Identification System (AIS) in the traditional clustering process, a DBSCAN clustering method based on the combination of improved Euclidean distance and cosine similarity is proposed. To measure the differences between arbitrary trajectory clusters in different directions, the improved DP algorithm is used to quickly and accurately extract the ship trajectory feature points, and maintain a high similarity with the original trajectory. The historical AIS data of the waterway near Port area of Zhoushan was selected to analyze and verify the algorithm. The results show that the proposed method can achieve the expected clustering effect, and the results are consistent with the actual situation, which has certain reference value in channel planning, navigation forecast, ship dynamic monitoring and other aspects.