From new observations and literature data we investigate the presence of H i, dust, and optical cores in the central kiloparsec of low-power radio galaxies. The goal of this pilot study is to identify physical relations between these components, which can help us to study kinematics and feeding mechanisms in future samples of active galaxies. Our results are consistent with neutral gas being associated with dust on sub-kiloparsec scales. Objects that have H i absorption always have significant amounts of dust in their host galaxy. If there is no visible dust in the host galaxy, there is also no H i absorption. The presence of an unresolved optical core correlates with the H i column density, with the core being absent in high-column density sources. This work opens a path for studying the kinematics of cold material in the central regions of active galaxies by combining information of H i absorption and molecular lines. Consistent with previous work, we find no evidence for a compact, parsec-scale obscuring torus in low-power radio galaxies.