Due to the rapid growth of the hardware technology, personal computers and workstations are more powerful than before. Instead of using the expensive supercomputer, many personal computers can be connected by a high speed network to form a distributed computing system, so as to decrease the cost of building a high performance computing system.To link all of the dispersed nodes to a cluster, it is very important to setup an agent for achieving the load balancing of the cluster. A Grey Dynamic model-based Load Balancing Mechanism (GMLBM) has been proposed in this paper. It will produce grey prediction for the load data according to the grey theory By applying a few data to get the load model for assigning new tasks according to the load in the predicted group for avoiding the overloading or vacancy of some nodes, so as to eliminate the system bottleneck and improve the system performance. The GMLBM is installed at the agent. The agent detects records and predicts the load of each node in a local group, and selects the node with the lowest load predicted as the node for executing the next task. A simulation has been made to evaluate the performance of the proposed system. By comparing with other load balancing methods, the experimental results show that the method of GMLBM can achieve a better performance than that of round robin and linear extrapolation.