Different optimization algorithms are used in fast medicine dispensing system to improve the efficiency of refilling. To achieve rapid replenishing of the manipulator and improve efficiency in pharmacy automation system. An ant colony optimization algorithm, which is named ant colony optimization model with characterization-based speed and multidriver, is proposed. Simulations of different picking points by order-picking mode, particle swarm optimization, particle swarm optimization with characterization-based speed and multi-driver, genetic algorithm, genetic algorithm with characterization-based speed and multi-driver, ant colony optimization, and ant colony optimization with characterization-based speed and multi-driver were carried out, and practical tests of different picking points by order-picking mode and ant colony optimization with characterization-based speed and multi-driver were done in a hospital. The data were collected and comparisons were made. Comparisons among order-picking mode, particle swarm optimization, genetic algorithm, particle swarm optimization with characterization-based speed and multi-driver, genetic algorithm with characterization-based speed and multi-driver, ant colony optimization, and ant colony optimization with characterization-based speed and multi-driver are made. Practical test results in hospital show that the ranges of optimization ratios of the average time values tested by ant colony optimization with characterization-based speed and multi-driver and the order-picking mode are 12.4%-40.3% when the picking points are between 5 and 20. Ant colony optimization with characterization-based speed and multi-driver is suitable for the refilling process in fast medicine dispensing system. It is capable of finding the refilling route which spends the shortest refilling time, and this meets the requirements in hospital.