Frequently port safety accidents have resulted in port safety problems becoming the focus of increasing attention. Port safety assessment is an important basis in preventing port safety accidents. Owing to complicated influencing factors of port safety, assessing port safety effectively has been a key research topic in the industry and academe. To determine port safety assessment indexes reasonably and acquire practical assessment results, the port safety assessment index system based on 5M (man, machine, material, management, and milieu) was established first. Then, weights of all assessment indexes were determined by the entropy weight method, and the port safety assessment model was constructed by matterelement theory. Finally, the assessment model was verified by ports in Zhejiang Province of China. Results demonstrate that the material factor presents the highest average weight among all evaluation indexes. The accident rate of goods and qualified rate of pollution control are two indexes with the highest weights. The port safety assessment results based on the entropy weight matter-element model conform to practical situations. Thus, the port safety assessment model based on entropy weight matter-element theory can measure port safety level effectively and evaluate its feasibility in practical cases. The model offsets shortcomings of single theory assessment and provides a new idea and decision-making method for port safety assessment.