Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

Li, Chuan; Wand, Michael

doi:10.1109/cvpr.2016.272

Cited by 619 publications

(635 citation statements)

References 20 publications

Supporting

Mentioning

634

Contrasting

Order By: Relevance

“…Other methods also combine the CNN model with additional techniques (e.g. conditional random fields, Markov random fields, and graph cut) for specific applications [47][48][49][50][51]. Most recently (after the conference version of our paper was accepted for presentation at the 2017 ARVO conference), a related method based on multiscale convolutional neural networks combined with graph search was published for segmenting the choroid in OCT retinal images [51].…”

Section: Introductionmentioning

confidence: 99%

Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search

Fang¹,

Cunefare²,

Wang³

et al. 2017

Biomed. Opt. Express

426

275

View full text Add to dashboard Cite

Abstract:We present a novel framework combining convolutional neural networks (CNN) and graph search methods (termed as CNN-GS) for the automatic segmentation of nine layer boundaries on retinal optical coherence tomography (OCT) images. CNN-GS first utilizes a CNN to extract features of specific retinal layer boundaries and train a corresponding classifier to delineate a pilot estimate of the eight layers. Next, a graph search method uses the probability maps created from the CNN to find the final boundaries. We validated our proposed method on 60 volumes (2915 B-scans) from 20 human eyes with non-exudative age-related macular degeneration (AMD), which attested to effectiveness of our proposed technique. 1178-1181 (1991). 2. S. Bhat, I. V. Larina, K. V. Larin, M. E. Dickinson, and M. Liebling, "4D reconstruction of the beating embryonic heart from two orthogonal sets of parallel optical coherence tomography slice-sequences," IEEE Trans. Med. Imaging 32(3), 578-588 (2013). 3. P. A. Keane, S. Liakopoulos, R. V. Jivrajka, K. T. Chang, T. Alasil, A. C. Walsh, and S. R. Sadda, "Evaluation of optical coherence tomography retinal thickness parameters for use in clinical trials for neovascular age-related macular degeneration," Invest. Ophthalmol. Vis. Sci. 50(7), 3378-3385 (2009). 4. P. Malamos, C. Ahlers, G. Mylonas, C. Schütze, G. Deak, M. Ritter, S. Sacu, and U. Schmidt-Erfurth, "Evaluation of segmentation procedures using spectral domain optical coherence tomography in exudative agerelated macular degeneration," Retina 31(3), 453-463 (2011). 5. P. P. Srinivasan, L. A. Kim, P. S. Mettu, S. W. Cousins, G. M. Comer, J. A. Izatt, and S. Farsiu, "Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images," Biomed. Opt. Express 5(10), 3568-3577 (2014). 6. J. C. Bavinger, G. E. Dunbar, M. S. Stem, T. S. Blachley, L. Kwark, S. Farsiu, G. R. Jackson, and T. W.Gardner, "The effects of diabetic retinopathy and pan-retinal photocoagulation on photoreceptor cell function as assessed by dark adaptometry DR and PRP effects on photoreceptor cell function," Invest. Ophthalmol. Vis. Sci. 57(1), 208-217 (2016). 7. C. A. Puliafito, M. R. Hee, C. P. Lin, E. Reichel, J. S. Schuman, J. S. Duker, J. A. Izatt, E. A. Swanson, and J. G.Fujimoto, "Imaging of macular diseases with optical coherence tomography," Ophthalmology 102(2), 217-229 (1995

show abstract

Section: Introductionmentioning

confidence: 99%

Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search

Fang¹,

Cunefare²,

Wang³

et al. 2017

Biomed. Opt. Express

426

275

View full text Add to dashboard Cite

show abstract

“…Another deep learning style transfer approach is demonstrated in figure 5g. The CNNMRF [16] approach gives good results for some styles, but it can be hard to control. Again, some aspects of the watercolour style have been well captured, but mismatches and inappropriate transfer have caused undesirable artifacts in the face.…”

Section: Methodsmentioning

confidence: 99%

“…Gabor rather than Perlin noise has been used to perform the wobbling, which can be seen most clearly as the oriented streaking of the background. Two results from the commercial app "DeepArt", which is based on the deep learning [12], (g) CNNMRF [16], (h) Image Analogies [13], (i) Waterlogue, (j) BeCasso.…”

Section: Methodsmentioning

confidence: 99%

Watercolour Rendering of Portraits

Rosin

Lai

2018

Image and Video Technology

View full text Add to dashboard Cite

Abstract. Applying non-photorealistic rendering techniques to stylise portraits needs to be done with care, as facial artifacts are particularly disagreeable. This paper describes a technique for watercolour rendering that uses a facial model to preserve distinctive facial characteristics and reduce unpleasing distortions of the face, while maintaining abstraction and stylisation of the overall image, employing stylistic elements of watercolour such as edge darkening, wobbling, glazing and diffusion.

show abstract

“…The works of Vincent Dumoulin and collaborators have recently further extended the control over the deep-learning stylistic transfer by allowing multiple styles to be selected and combined at once, opening the way to new forms of visual creation, in which a graphical artist could "paint with styles" just as a musician combines sound textures to produce an original mix (Dumoulin et al, 2016). Other approaches to the problem of "non-photorealistic rendering" relied on a recurrent neural network (RNN) architecture to separate a given style from an image content and transform new images accordingly (Zhao and Xu, 2016) or combined convolutional networks with random Markov fields priors to better match local feature patches in both images (Li and Wand, 2016). The latter technique has been recently adapted to produce strikingly convincing paintings based on rough doodles or sketches (Champandard, 2016).…”

Section: Graphic Artmentioning

confidence: 99%

Deep Creations: Intellectual Property and the Automata

Deltorn

2017

Front. Digit. Humanit.

View full text Add to dashboard Cite

The rapid progress of deep neural network architectures is allowing both to automate the production of artworks and to extend the domain of creative expression. As such, it is opening new ground for professional and amateur artists alike. A major asset of these new computer processes is their capacity to derive, from a training phase, a generative model from which new artifacts can be produced. This attribute allows for a wide range of novel applications. New music or paintings in the style of famous artists can be produced at the click of a button, or combined to form new artworks. New graphical compositions can be "hallucinated" by the deep algorithmic models to produce striking, unexpected, visual forms. By the same token, the dependence on preexisting, protected, artworks lays the ground for potential zones of friction with the rights holders of the source data that helped shape the generative model. This articulation, between the popular creative movement initiated by the deep neural architectures and the preexisting rights of the authors, leads to a confrontation between the present legal framework for the protection of artistic creations and the new modalities offered by these new technological objects. The present work will address the conditions of protection of creations generated by deep neural networks under the main copyright regimes.

show abstract

Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

Cited by 619 publications

References 20 publications

Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search

Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search

Watercolour Rendering of Portraits

Deep Creations: Intellectual Property and the Automata

Contact Info

Product

Resources

About