In mobile edge computing (MEC) systems, edge service caching refers to pre-storing the necessary programs for executing certain computation tasks at MEC servers, which is effective to reduce the real-time delay/bandwidth cost on acquiring and installing the programs. Due to the limited caching space at resource-constrained edge servers, it calls for careful design of caching placement to determine which programs to cache over time. This is in general a complicated problem that highly correlates to the computation offloading decisions of computation tasks, i.e., whether or not to offload a task for edge execution. In this paper, we consider an edge server assisting a mobile user (MU) in executing a sequence of computation tasks. In particular, the MU can upload and run its customized programs at the edge server, while the server can selectively cache the previously generated programs for future service reuse. To minimize the computation delay and energy consumption of the MU, we formulate a mixed integer non-linear programming (MINLP) that jointly optimizes the service caching placement, computation offloading decisions, and system resource allocation (e.g., CPU processing frequency and transmit power of MU). To tackle the problem, we first derive the closed-form expressions of the optimal resource allocation solutions, and subsequently transform the MINLP into an equivalent pure 0-1 integer linear programming (ILP) that optimizes only the binary caching placement and offloading decisions. To further reduce the complexity in solving a large-size ILP, we exploit the underlying graphical structures in caching causality and task dependency models, and accordingly devise a reduced-complexity alternating minimization technique to iteratively update either the caching placement or offloading decision by fixing the other. Extensive simulations show that the proposed joint optimization techniques achieve substantial resource savings of the MU compared to other representative benchmark methods considered.