Complex systems can be defined by “sloppy” dimensions, meaning that their behavior is unmodified by large changes to specific parameter combinations, and “stiff” dimensions whose changes result in considerable modifications. In the case of the neocortex, sloppiness in synaptic architectures would be crucial to allow for the maintenance of spiking dynamics in the normal range despite a diversity of inputs and both short- and long-term changes to connectivity. Using simulations on neural networks with spiking matched to murine visual cortex, we determined the stiff and sloppy parameters of synaptic architectures across three classes of input (brief, continuous, and cyclical). Large-scale algorithmically-generated connectivity parameter values revealed that specific combinations of excitatory and inhibitory connectivity are stiff and that all other architectural details are sloppy. Stiff dimensions are consistent across a range of different input classes with self-sustaining synaptic architectures occupying a smaller subspace as compared to the other input classes. We also find that experimentally estimated connectivity probabilities from mouse visual cortex are similarly stiff and sloppy when compared to the architectures that we identified algorithmically. This suggests that simple statistical descriptions of spiking dynamics are a sufficient and parsimonious description of neocortical activity when examining structure-function relationships at the mesoscopic scale. Moreover, this study provides further evidence of the importance of the interrelationship of excitatory and inhibitory connectivity to establish and maintain stable spiking dynamical regimes in neocortex.Significance StatementConnections between neurons are continuously changing to allow learning and adaptation to new stimuli. However, the ability of neural networks to vary these connections while avoiding excessively high- or low-activity states is still not well understood. We tackled this question by studying how changes in the parameters of connectivity within and between different neuronal populations impacted network activity in computational models. We identified specific combinations of parameters, deemed “stiff”, that must be maintained to observe activity consistent with recordings from murine visual cortex, while the rest of the parameters can be varied freely with minimal effects on activity. Our results agree with experimentally measured connectivity statistics demonstrating the importance of balancing opposing forces to maintain activity in a natural regime.