In the era of emerging Smart Built Environments (SBEs), a smart house, unlike regular houses with static "components", consists of numerous interconnected and often actuated devices, capable of executing tasks independent of user supervision. Living in such a SBE, where for example, the furniture can rearrange itself, and the doors open and close of their own volition, may be difficult and unpredictable. Furthermore, cybersecurity attacks and intrusion could allow attackers to assume control of the SBE, damage its components and to potentially harm its inhabitants. Such novel characteristics of SBEs present developers with several unique challenges with regards to implementing the needed safety and security measures and protocols that go along with them. With such environments, therefore, there is a need for a system that is capable of monitoring user activities in real-time, identifying the safety and security hazards to users in their immediate local context, warning users of these hazards, and perhaps even taking preventative and mitigative action against the hazards that it identified. In this paper, we survey some of these challenges and explore the design and implementation of a system designed around the safety and security of SBE inhabitants. We propose an approach to modeling SBE safety that combines the three laws of robotics and the swarm behavior model. We also present a preliminary prototype and discuss a case study.