Cloud computing is the recent advancement in the technology of distributed computing that works on the basis of pay as you use model. It comprises of virtual machines that provides both storage and computational facility. The main aim of cloud computing is to offer access to remotely distributed resources to its users. Scheduling of tasks plays a vital role in the efficient working of cloud computing. Sometimes situation comes where multiple tasks with same priority arrives so a good scheduler is one which handles the situation appropriately with proper load balancing. There are various scheduling algorithms proposed in the past. Most of the scheduling algorithms neglect the concept of load balancing. Load balancing in cloud computing is also as much important as task scheduling. As cloud environment consists of virtual machine, it should take care that no virtual machine remains idle and also no virtual machine is under heavy load of the tasks. So, it is important to balance the load equally among the virtual machines to solve the issue of under loading and over loading of virtual machines. In this paper, a credit based scheduling algorithm with load balancing (CBSA_LB) is proposed that balances the load along with the scheduling of tasks. The results are evaluated on the basis of six parameters: processing time, processing cost, response time, makespan time, Throughput and Execution Time. Experimental results show that the proposed technique outperforms the existing techniques (EMOSA and CBSA). .