We propose an entropy current for dynamical black holes in a theory with arbitrary four derivative corrections to Einstein's gravity, linearized around a stationary black hole. The Einstein-Gauss-Bonnet theory is a special case of the class of theories that we consider. Within our approximation, our construction allows us to write down a completely local version of the second law of black hole thermodynamics, in the presence of the higher derivative corrections considered here. This ultra-local, stronger form of the second law is a generalization of a weaker form, applicable to the total entropy, integrated over a compact 'time-slice' of the horizon, a proof of which has been recently presented in [1]. We also provide a general algorithm to construct the entropy current for the four derivative theories, which may be straightforwardly generalized to arbitrary higher derivative corrections to Einstein's gravity. This algorithm highlights the possible ambiguities in defining the entropy current.