The reverse engineering of gene regulatory networks based on gene expression data is a challenging inference task. A related problem in computational systems biology lies in identifying signalling networks that perform particular functions, such as adaptation. Indeed, for many research questions, there is an ongoing search for efficient inference algorithms that can identify the simplest model among a larger set of related models. To this end, in this paper, we introduce SLInG, a Bayesian sparse likelihood-free inference method using Gibbs sampling. We demonstrate that SLInG can reverse engineer stochastic gene regulatory networks from single-cell data with high accuracy, outperforming state-of-the-art correlation-based methods. Furthermore, we show that SLInG can successfully identify signalling networks that execute adaptation. Sparse hierarchical Bayesian inference thus provides a versatile tool for model discovery in systems biology and beyond.