Nowadays, mutual interference among automotive radars has become a problem of wide concern. In this paper, a decentralized spectrum allocation approach is presented to avoid mutual interference among automotive radars. To realize decentralized spectrum allocation in automotive scenarios meet with two challenges: 1) the allocation approach should be dynamic due to the changing positions of all radars; 2) each radar has no communication with others so it has quite limited information. A machine learning technique, reinforcement learning (RL), is utilized because RL can learn a decision making policy in an unknown dynamic environment. As radar's single observation is incomplete, a long short-term memory (LSTM) recurrent network is used to aggregate radar's observations through time so that a subband is chosen by combining both the present and past observations. Simulation experiments are conducted to compare the proposed approach with other common spectrum allocation methods such as the random and myopic policy, showing that our approach outperforms the others.