Despite decades of intensive search for compounds that modulate the activity of particular proteins, there are currently small-molecule probes available only for a small proportion of the human proteome. Effective approaches are therefore required to map the massive space of unexplored compound-target interactions for novel and potent activities. Here, we carried out a crowdsourced benchmarking of the accuracy of machine learning (ML) algorithms at predicting kinase inhibitor potencies across multiple kinase families. A total of 268 ML predictions were scored in unpublished bioactivity data sets. Top-performing algorithms used kernel learning, gradient boosting and deep learning, with predictive accuracy exceeding that of target activity assays. Subsequent experiments carried out based on the the top-performing model predictions demonstrated that these models and their ensemble can improve the accuracy of experimental mapping efforts, especially for so far under-studied kinases. The open-source ML algorithms together with the novel dose-response data for 905 bioactivities between 95 compounds and 295 kinases provide a unique resource for extending the druggable kinome.AI approaches, that can leverage the information extracted from similar kinases and compounds to predict the activity of so far unexplored interactions.The Challenge was implemented in a screening-based, pre-competitive drug discovery project in collaboration with the NIH-supported Illuminating the Druggable Genome (IDG) program ( https://commonfund.nih.gov/idg ), with the common aim to establish kinome-wide target profiles of small-molecule agents, and thereby to extend the druggability of the human kinome space by providing activity information on under-studied proteins. The specific questions this Challenge sought to address were: (i) What are the best computational modelling approaches for predicting quantitative compound-target activity profiles?; (ii) What are the optimal molecular and chemical descriptors for maximal prediction accuracy?; and (iii) What are the most predictive bioactivity assays and publicly available datasets? The Challenge attracted 212 active participants, and a total of 268 predictions were scored, covering a wide range of ML approaches, including deep and kernel learning and gradient boosting decision trees. Here, we describe the benchmarking results from the Challenge, and the use of top-performing models for identifying novel kinase inhibitor activities.