Exploring the many surprising facets of timbre through sound manipulations has been a common practice among composers and instrument makers of all times. The digital era radically changed the approach to sounds thanks to the unlimited possibilities offered by computers that made it possible to investigate sounds without physical constraints. In this chapter, we describe investigations on timbre based on the analysis-by-synthesis approach that consists of using digital synthesis algorithms to reproduce sounds and further modify the parameters of the algorithms to investigate their perceptual relevance. In the first part of the chapter, timbre is investigated in a musical context. An examination of the sound quality of different wood species for xylophone making is first presented. Then the influence of physical control on instrumental timbre is described in the case of clarinet and cello performances. In the second part of the chapter, environmental sounds have been investigated in order to identify so-called invariant sound structures that can be considered as the backbone, or the bare minimum, of the information contained in a sound that enables the listener to recognize its source both in terms of structure (e.g. size, material) and action (e.g. hitting, scraping). Such invariants are generally composed of combinations of audio descriptors such as decay, attack, spectral density and pitch. Various investigations on perceived sound properties responsible for the evocations of sound sources are here identified and described through both applied and fundamental studies.