Pod and seed size are important characteristics for the peanut industry and have been under strong selection pressure since peanut domestication. In order to dissect the genetic control of peanut pod and seed size, a recombinant inbred mapping population from a cross of Florida-07 by GP-NC WS 16 was used to determine the genomic regions associated with traits including 100 pod weight, 100 seed weight, pod weight of double-seeded pods, seed weight of double-seeded pods, and area of double-seeded pods. Nine QTL on linkage groups (LGs) A05, A06, A09, B10, B04, A03, B05, and B08 were associated with pod and seed size. A majority of the QTL have small effects except the locus on LG A05 (93 to 102 Mbp) which explained up to 66% phenotypic variation for all measured pod and seed traits. A comparison of QTL previously reported for yield component traits showed a common QTL on LG A05 was detected in two genetic populations whose parentage is distinct from those used in this study. The markers tightly linked to this major QTL were informative in distinguishing large versus small-seeded germplasm lines in the mini core collection originating from thirty-one countries, suggesting selection for this seed size QTL in large-seeded ecotypes. However, the large seed size allele appeared to co-segregate with a late leaf spot disease susceptibility allele inherited from the Florida-07 parent. Therefore, peanut breeders need to weigh the pros and cons before integrating the large seed size QTL from Florida-07 in their breeding program.