Multifarious artificial synaptic devices are extensively proposed in the field of neuromorphic hardware systems for their applicability in promising parallel computer architecture, which is preferred to classical Von Neumann architecture in numerous and complex information processing. Besides the ability to mimic typical biological synaptic behaviors, low power consumption is critical for the synaptic devices in the neuromorphic hardware system.In this paper, ultralow‐power Ta2O5/Al2O3 bilayer‐gate‐dielectric synaptic transistors (TABSTs) with low‐temperature atomic layer deposited dielectric are proposed. The TABSTs show power consumption as low as 19.9 aJ per synaptic event successfully at a low drain voltage of 0.001 V and a short pulse width of 1 ms. Essential synaptic behaviors including excitatory postsynaptic current, inhibitory postsynaptic current, spike‐amplitude‐dependent plasticity, spike‐duration‐dependent plasticity, paired pulse facilitation, long‐term potentiation, long‐term depression, the transition from short‐term memory to long‐term memory as well as learning and forgetting abilities are well mimicked by the TABSTs. Moreover, algebraic arithmetic operations such as addition, subtraction, multiplication, and division are also implemented by the TABSTs. This work provides a promising approach to emerging neuromorphic systems.