High-resolution real-time imaging at cellular level in retinal surgeries is very challenging due to extremely confined space within the eyeball and lack of appropriate modalities. Probe-based confocal laser endomicroscopy (pCLE) system, which has a small footprint and provides highly-magnified images, can be a potential imaging modality for improved diagnosis. The ability to visualize in cellular-level the retinal pigment epithelium and the chorodial blood vessels underneath can provide useful information for surgical outcomes in conditions such as retinal detachment. However, the adoption of pCLE is limited due to narrow field of view and micron-level range of focus. The physiological tremor of surgeons' hand also deteriorate the image quality considerably and leads to poor imaging results.In this paper, a novel image-based hybrid motion control approach is proposed to mitigate challenges of using pCLE in retinal surgeries. The proposed framework enables shared control of the pCLE probe by a surgeon to scan the tissue precisely without hand tremors and an auto-focus image-based control algorithm that optimizes quality of pCLE images. The control strategy is deployed on two semi-autonomous frameworks -cooperative and teleoperated. Both frameworks consist of the Steady-Hand Eye Robot (SHER), whose end-effector holds the pCLE probe. The teleoperated framework also uses the da Vinci Research Kit (dVRK), which enables the user to remotely control the pCLE probe. The frameworks have been evaluated through experiments and a series of user studies involving 14 participants. Statistical analyses have been conducted and it is shown that the proposed hybrid approach results in higher image quality, smoother motion, and reduced workload in a statistically significant manner.