A common method for performing multi-scale person detection is a sliding window classification. For every window location and scale a binary classification is done. Many state-of-the-art person detectors follow this sliding window paradigm. Not only this exhaustive search space strategy is computationally expensive, it usually produces large number of false positives. In order to estimate an optimal reduced search space, we derive a maximum a posteriori probability (MAP) solution given only the person annotations of a dataset. The proposed MAP solution considers the naturally height distribution of persons, deviations from a flat world assumption, and annotation uncertainty. The effectiveness compared to the traditional uniform sliding window selection strategy is shown on different realistic monocular pedestrian detection datasets. Moreover the MAP search space estimation provides design parameters for modeling the tradeoff between detection performance and runtime constraints