Artificial neural networks based on crossbar arrays of analog programmable resistors can address the high energy challenge of conventional hardware in artificial intelligence applications. However, state‐of‐the‐art two‐terminal resistive switching devices based on conductive filament formation suffer from high variability and poor controllability. Electrochemical ionic synapses are three‐terminal devices that operate by electrochemical and dynamic insertion/extraction of ions that control the electronic conductivity of a channel in a single solid‐solution phase. They are promising candidates for programmable resistors in crossbar arrays because they have shown uniform and deterministic control of electronic conductivity based on ion doping, with very low energy consumption. Here, the desirable specifications of these programmable resistors are presented. Then, an overview of the current progress of devices based on Li+, O2−, and H+ ions and material systems is provided. Achieving nanosecond speed, low operation voltage (≈1 V), low energy consumption, with complementary metal–oxide–semiconductor compatibility all simultaneously remains a challenge. Toward this goal, a physical model of the device is constructed to provide guidelines for the desired material properties to overcome the remaining challenges. Finally, an outlook is provided, including strategies to advance materials toward the desirable properties and the future opportunities for electrochemical ionic synapses.