-Large-scale Cloud systems and big data analytics frameworks are now widely used for practical services and applications. However, with the increase of data volume, the heterogeneity of workloads and resources, together with the dynamicity of massive user requests, the uncertainties and complexity of resource scheduling and service provisioning increase dramatically, often resulting in poor resource utilization, vulnerable system dependability, and negative user-perceived performance impacts. In this paper, we present our understanding of the current and future challenges in this particular area, and discuss both existing and potential solutions to the problems, especially those concerned with system efficiency, scalability and dependability. We first introduce a data-driven analysis methodology for characterizing the resource and workload patterns and tracing performance bottlenecks in a massive-scale distributed computing environment. We then examine and analyze several fundamental challenges and the solutions we are developing to tackle them, including for example, incremental resource scheduling and incremental messaging communication, decentralized scheduling, rapid system failover, and request handling parallelism etc. We integrate these techniques together with the data analytics methodology to drive a novel big data and service engineering to facilitate the system optimization, tuning and verification. We aim to develop and offer innovative methods and mechanisms for future computing platforms that will provide strong support for new big data and IoE (Internet of Everything) applications.