This paper presents in a simple and very concise manner, and accessible to a broad audience, the concepts of Artificial Intelligence (AI), Agents and Multi-Agent Systems (MAS), Coordination in MAS, Intelligent Robotics (IR), Multi-Robot Systems (MRS), Machine Learning (ML), Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL). Emphasis is given on and how AI and ML, with special focus on DRL, may be efficiently used to create efficient robot skills and coordinated robotic teams, capable of performing very complex actions and tasks, as a team, such as playing a game of soccer.The paper also presents the concept of robotic soccer and the vision and structure of the RoboCup initiative and the world robotic soccer championship with emphasis on the Humanoid Simulation 3D league and the new challenges for the AI community that RoboCup, and in particular this competition, poses. The final topics presented at the paper are based on the research developed or coordinated by the author throughout the last 22 years on the areas of multi-agent systems and intelligent robotics with emphasis for its applications in the domain of robotic soccer in the context of the FC Portugal project.Proxy challenge and the Free/Scientific Challenge at RoboCup 2022. In what concerns scientific publications, the FC Portugal project and the works briefly described in this paper enabled us to publish more than 100 papers in indexed international conferences and journals.