Background: There are large individual differences in physical activity (PA) behavior as well as trainability of physical capacity. Heritability studies have shown that genes may have as much impact on exercise participation behavior as environmental factors. Genes that favor both trainability and participation may increase the levels of PA. The present study aimed to assess the allele frequencies in genes associated with PA and/or physical capacity, and to see if there is any association between these polymorphisms and self-reported PA levels in a cohort of middleaged Norwegians of Scandinavian descent (n = 831; mean age mean age (± SD) 55.5 ± 3.8 years). Results: The genotype distributions of the ACTN3 R577X, ACE I/D and MAOA uVNTR polymorphisms were similar to other populations of European descent. When comparing the genotype distribution between the low/medium level PA group (LMPA) and high level PA groups (HPA), a significant difference in ACTN3 577X allele distribution was found. The X allele frequency was 10% lower in the HPA level group (P = 0.006). There were no differences in the genotype distribution of the ACE I/D or MAOA uVNTR polymorphism. Education and previous participation in sports or outdoor activities was positively associated with the self-reported PA levels (P ≤ 0.001). Conclusions: To the best of our knowledge, this is the first study to report association between ACTN3 R577X genotype and PA level in middle-aged Scandinavians. Nevertheless, the contribution of a single polymorphism to a complex trait, like PA level, is likely small. Socioeconomic variables, as education and previous participation in sports or outdoor activities, are positively associated with the self-reported PA levels.