This paper describes several approaches for utilizing machine learning technologies towards improving the capabilities of autonomous, simulation-based agents.For an autonomous agent to be robust, it must be able t o plan its activities, react quickly t o unforeseen events, and execute planned or modified behaviors to achieve goals. W e have begun t o develop autonomous agents which exhibit appropriate behaviors for simulated air combat, providing intelligent, realistic adversaries and cooperative allies. Building such agents is not trivial, and the techniqves of machine learning hold great promise for extending the capabilities of hand-coded systems. In this paper we describe the application of some of these techniques, past successes, and current research directions.