Due to low-cost and its practical solution, the integration of RFID tag to the sensor node called smart RFID has become prominent solution in various fields including industrial applications. Nevertheless, the constrained nature of smart RFID system introduces tremendous security and privacy problem. One of them is the problem in key management system. Indeed, it is not feasible to recall all RFID tags in order to update their security properties (e.g. update their private keys). On the other hand, using common key management solution like standard TLS/SSL is too heavy-weight that can drain and overload the limited resources. Furthermore, most of existing solutions are highly susceptible to various threats reaching from privacy threats, physical attacks to various technics of Man-in-the-Middle attacks. This paper introduces novel key management system, tailored to the limited resources of smart RFID system. It proposes light-weight mutual authentication and identity protection to mitigate the existing threats.