Organometallic complexes are ubiquitous in homogeneous catalysis and other technological applications. Optimization of such complexes for specific applications is challenging due to the large variety of possible metal-ligand combinations and ligand-ligand interactions. Here we present OM-Diff, an inverse design framework based on a diffusion generative model for in-silico design of such complexes from scratch. Given the importance of the spatial structure of a catalyst, the model directly operates on all-atom (including hydrogen) representations in 3D space. To handle the symmetries inherent to that data representation, OM-Diff combines an equivariant diffusion model and an equivariant property predictor to drive sampling at inference time. The model can conditionally generate novel ligands beyond those in the training dataset. We demonstrate the potential of the proposed approach by designing catalysts for a family of cross-coupling reactions, and validating a selection of novel proposed compounds with DFT calculations.