This study investigates factors affecting handheld human -computer interaction (HCI) for older adults with Age-related Macular Degeneration (AMD). This is largely an uncharted territory, as empirical investigations of HCI concerning users with visual dysfunction and/or older adults have focused primarily on desktop computers. For this study, participants with AMD and visually healthy controls used a handheld computer to search, select and manipulate familiar playing card icons under varied icon set sizes, intericon spacing and auditory feedback conditions. While all participants demonstrated a high rate of task completion, linear regression revealed several relationships between task efficiency and the interface, user characteristics and ocular factors. Two ocular measures, severity of AMD and contrast sensitivity, were found to be highly predictive of efficiency. The outcomes of this work reveal that users with visual impairments can effectively interact with graphical user interfaces on small displays in the presence of low-cost, easily implemented design interventions. Furthermore, results demonstrate that the detrimental influence of AMD and contrast sensitivity on handheld technology interaction can be offset by such interventions. This study presents a rich data set and is intended to inspire future work characterizing and modeling the interactions of individuals with visual impairments with non-traditional information technology platforms and contexts.