We investigate the role of a warm sea-surface temperature (SST) anomaly (hot-spot of typically 3 K to 5 K) on the aggregation of convection using cloud resolving simulations in a non-rotating framework. It is well known that SST gradients can spatially organize convection. Even with uniform SST, the spontaneous self-aggregation of convection is possible above a critical SST (here 295 K), arising mainly from radiative feedbacks. We investigate how a circular hot-spot helps organize convection, and how self-aggregation feedbacks modulate this organization. The hot-spot significantly accelerates aggregation, particularly for warmer/larger hot-spots, and extends the range of SSTs for which aggregation occurs, however at cold SST (290 K) the aggregated cluster disaggregates if we remove the hot-spot. Large convective instability over the hot-spot leads to stronger convection and generates a large-scale circulation which forces the subsidence drying outside the hot-spot. Indeed, convection over the hot-spot brings the atmosphere towards a warmer temperature. The warmer temperatures are imprinted over the whole domain by gravity waves and subsidence warming. The initial transient warming and concomitant subsidence drying suppress convection outside the hot-spot, thus driving the aggregation. The hot-spot induced large-scale circulation can enforce the aggregation even without radiative feedbacks for hot-spots sufficiently large/warm. The strength of the large-scale circulation, which defines the speed of aggregation, is a function of the hot-spot fractional area. At equilibrium, once the aggregation is well established, the moist convective region with upward mid-tropospheric motion, centered over the hot-spot, has an area surprisingly independent of the hot-spot size.