Carbohydrates and their conjugates play important roles in many biological processes including fertilization, differentiation, development, immune response, and infection. Their activities are largely dependent on the properties of terminal mono- or disaccharides. Galactose, mannose, fucose, glucose, sialic acid, etc., are commonly used as powerful scaffolds installed on drug molecules for targeting specific tissues including brain, liver, and cancers, and as epitopes for enhancing the targeting of various vaccines. This review focuses on the influence of their structural variations, including changes in sugar type, substituent groups and their positions, as well as length of linker portion, on the targeting of drugs or their efficacy. Particular attention is paid to the targeting properties of mono- and disaccharides applied in drug design and discovery.