Black-box attacks generate adversarial examples by querying the target model and updating the noise according to the feedback. However, the current black-box attack methods require excessive queries to generate adversarial examples, increasing the risk of detection by target defense systems. Furthermore, the current black-box attack methods primarily focus on controlling the magnitude of perturbations while neglecting the impact of perturbation placement on the stealthiness of adversarial examples. To this end, we propose a novel edge noise-constrained black-box attack method using the artificial fish swarm algorithm (EFSAttack). EFSAttack introduces the concept of edge noise constraint to indicate the low-frequency region of the image where perturbations are added and employs edge noise constraint to improve the population initialization and population evolution process. The experiments on CIFAR-10 and MNIST show notable improvements in the success rates, query efficiency, and adversarial example invisibility.