This paper introduces HetGNN, a novel machine learning framework for wireless network resource allocation, blending graph neural networks with long short-term memory networks to adeptly manage graph-structured and temporal data. An adaptive resource allocation algorithm, rooted in Actor-Critic reinforcement learning, dynamically refines strategies, ensuring efficient interaction between agents and their environment. Comparative analyses with benchmark algorithms on a simulation platform highlight HetGNN's superiority in enhancing spectrum and energy efficiency, alongside user experience. The study paves the way for advanced applications in 5G/6G networks, emphasizing the integration of federated and transfer learning for future intelligent wireless network developments.