We introduce a notion generalizing Calabi-Yau structures on Ainfinity algebras and categories, which we call pre-Calabi-Yau structures. This notion does not need either one of the finiteness conditions (smoothness or compactness) which are required for Calabi-Yau structures to exist. In terms of noncommutative geometry, a pre-CY structure is as a polyvector field satisfying an integrability condition with respect to a noncommutative analogue of the Schouten-Nijenhuis bracket. We show that a pre-CY structure defines an action of a certain PROP of chains on decorated Riemann surfaces. In the language of the cobordism perspective on TQFTs, this gives a partially defined extended 2-dimensional TQFT, whose 2-dimensional cobordisms are generated only by handles of index one. We present some examples of pre-CY structures appearing naturally in geometric and topological contexts.