Discriminative Alignment of Projected Belief Networks

Using the Projected Belief Network at High Dimensions

“…A number of variations of this concept have been proposed. The PBN or D-PBN cost functions can be used as a regularization for discriminative neural networks [12], or the opposite: discriminative cost function can be used to "align" a PBN to decision boundaries to create betterperforming generative models [17]. We will use this approach in this paper to test the PBN and D-PBN at high dimensions.…”

“…It was shown in [17] that a generative classifier (a PBN) can compete with state of the art discriminative classifiers. This seems to contradict the widely-held belief that the generative task is much harder, and unnecessary for classifying [18].…”

“…( 16) in [20]). We note that the term "approximation" is misleading because the error can be neglected for all practical purposes [17].…”

“…The goal of our first experiment was to show that discriminative alignment, which was demonstrated previously on input data data dimensions of 196 and 900 [17], can be used on a different data set with much higher dimension. Discriminative alignment is explained in Section I-B and further details are given in Section IV-F.…”

“…Data was classified by evaluating the PBN log-likelihood function for each class assumption, and choosing the largest. A class-dependent prior density was used for the output distribution, as explained in [17]. A benchmark DNN was also trained using the same data partitioning.…”