Heckerthoughts

Abstract

In 1987, Eric Horvitz, Greg Cooper, and I visited I.J. Good at Virginia Polytechnic and State University. The three of us were at a conference in Washington DC and made the short drive to see him. The primary reason we wanted to see him was not because he worked with Alan Turing to help win WWII by decoding encrypted messages from the Germans, although that certainly intrigued us. Rather, we wanted to see him because we had just finished reading his book "Good Thinking" [24], which summarized his life's work in Probability and its Applications. We were delighted that he was willing to have lunch with us. We were young graduate students at Stanford working in Artificial Intelligence (AI), and were amazed that his thinking was so similar to ours, having worked decades before us and coming from such a seemingly different perspective not involving AI. We had lunch and talked about many topics of shared interest including Bayesian probability, graphical models, decision theory, AI (he was interested by then), how the brain works, and the nature of the universe. Before we knew it, it was dinner time. We took a brief stroll around the beautiful campus and sat down again for dinner and more discussion. Figure 1 is a photo taken just before we reluctantly departed.This story is a fitting introduction this manuscript. Now having years to look back on my work, to boil it down to its essence, and to better appreciate its significance (if any) in the evolution of AI and Machine Learning (ML), I realized it was time to put my work in perspective, providing a roadmap to any who would like to explore it in more detail. After I had this realization, it occurred to me that this is what I.J. Good did in his book.This manuscript is for those who want to understand basic concepts central to ML and AI, and to learn about early applications of these concepts. Ironically, after I finished writing this manuscript, I realized that a lot of the concepts that I included are missing in modern courses on ML. I hope this work will help to make up for these omissions. The presentation gets somewhat technical in parts, but I've tried to keep the math to the bare minimum to convey the fundamental concepts.In addition to the technical presentations, I include stories about how the ideas came to be and the effects they have had. When I was a student in physics, I was given dry texts to read. In class, however, several of my physics professors would tell stories around the work, such as Einstein's thinking that led to his general theory of relativity. Those stories fascinated me and really made the theory stick. So here, I do my best to present both the fundamental ideas and the stories behind them.As for the title, "Heckerman Thinking" doesn't have the same ring to it as that of Good's book. I chose "Heckerthoughts," because my rather odd last name has been humorous fodder for friends and colleagues for naming things related to me such as "Heckerperson," "Heckertalk," and "Heckerpaper"-you get the idea. Ironically, a distant relative of mine who co...