“…TTD relies on applying, at time t, the following update operation to the state and action from time t -m + h A~rn update ~ (Q, Xt--m+l; at-m+1, Zt_m+ I --Q(xt-m+l, at-m+1)), (2) where zt ~'m is the TTD()~, m) return for time t, defined as . This can be performed either iteratively, based directly on the definition of TTD returns, or in an incremental manner, which is particularly efficient.…”