In the Social Web, the users are invited to publish a lot of personal information. These data can be easily retrieved, and sometimes reused, without providing the users with fine-grained access control mechanisms able to restrict the access to their profiles, and data. In this paper, we present an access control model for the Social Semantic Web. Our model is grounded on the Social Semantic SPARQL Security for Access Control vocabulary (S4AC). This vocabulary can be used by the users to define their own terms of access to the data. We define an algorithm, implemented in our Access Control Manager, which allows to check, after a client query, to which extent the data are available, depending on the user's profile. The evaluation of the access conditions is related to different features, such as the social tags associated with the data, and the user's contextual information, such as being part of a group, being located in a specific place. We provide an evaluation of the overhead introduced by our Access Control Manager, and we show that access control in the Social Semantic Web comes with a cost, but this is acceptable given the benefits of data protection.