The Internet of Things (IoT) protocols have encountered great challenges as the growth of technology has led to many limitations of the performance of the IoT protocols. Message Queuing Telemetry Transport protocol (MQTT) is one of the most dominant protocols in most fields of smart applications, so it has been chosen in this research to be a use case for implementing and evaluating a new proposed Back-off algorithm that is designed to eliminate suspicious and fake messages by calculating an initial frequent rate for each publisher connected to the MQTT broker. The proposed Backoff algorithm was designed to mitigate the traffic load of the uplink traffic by applying an exponential delay factor to suspicious publishers. Another priority scheduling algorithm was proposed to classify publishers as high priority or low priority depending on the new calculated frequent rate. The two algorithms were implemented on the Mosquitto broker and evaluated using a simulation environment by measuring specified performance metrics. The simulated results proved that the Back-off algorithm eliminated network load and introduced an acceptable range of CPU and RAM consumption. The results also concluded that the priority classification algorithm managed to reduce the latency of high-priority publishers.