Model-based interpretation of complex and variable images

Abstract

The ultimate goal of machine vision is image understandingöthe ability not only to recover image structure but also to know what it represents. By de¢nition, this involves the use of models which describe and label the expected structure of the world. Over the past decade, model-based vision has been applied successfully to images of man-made objects. It has proved much more di¤cult to develop model-based approaches to the interpretation of images of complex and variable structures such as faces or the internal organs of the human body (as visualized in medical images). In such cases it has been problematic even to recover image structure reliably, without a model to organize the often noisy and incomplete image evidence. The key problem is that of variability. To be useful, a model needs to be speci¢cöthat is, to be capable of representing only`legal' examples of the modelled object(s). It has proved di¤cult to achieve this whilst allowing for natural variability. Recent developments have overcome this problem; it has been shown that speci¢c patterns of variability in shape and grey-level appearance can be captured by statistical models that can be used directly in image interpretation. The details of the approach are outlined and practical examples from medical image interpretation and face recognition are used to illustrate how previously intractable problems can now be tackled successfully. It is also interesting to ask whether these results provide any possible insights into natural vision; for example, we show that the apparent changes in shape which result from viewing three-dimensional objects from di¡erent viewpoints can be modelled quite well in two dimensions; this may lend some support to the`characteristic views' model of natural vision.

Multilevel principal component analysis (mPCA) in shape analysis: A feasibility study in medical and dental imaging

“…The ASM method has been extensively documented in the literature (see, e.g., Refs. [1][2][3][4][5][6][7][8]), and therefore this topic is not discussed here. Instead, we use the approach of mPCA discussed in Ref.…”

“…This is particularly important for repeated images taken from the same person or different "patches" from the same image. Methods that reflect these relationships between images at multiple levels, should function more efficiently than those that do not shape models (ASMs) and active appearance models (AAMs) [1][2][3][4][5][6][7][8] are a common technique of image processing that are used to search for specific features or shapes in images.…”

Initial Results of Multilevel Principal Components Analysis of Facial Shape

“…[1][2][3][4][5][6][7][8]), and therefore this topic is not discussed here. One carries out PCA for the covariance matrix as discussed in Ref.…”

“…Active shape models (ASMs) and active appearance models (AAMs) [1][2][3][4][5][6][7][8] are common techniques in image processing that are used to search for specific features or shapes in images. However, if clustering or multilevel data structures exist naturally in the data set, e.g., as illustrated by the flowchart in Fig.…”

What’s in a Smile? Initial Analyses of Dynamic Changes in Facial Shape and Appearance

“…Indeed, the mPCA approach has previously been shown [21][22][23][24][25] to provide a simple and straightforward method of modeling shape. The mPCA approach is potentially also of much use in active shape models (ASMs) [26][27][28][29][30] and active appearance models (AAMs) [31][32][33][34][35][36].…”