The membrane sculpting ability of BAR domains has been attributed to the intrinsic curvature of their banana-shaped dimeric structure. However, there is often a mismatch between this intrinsic curvature and the diameter of the membrane tubules generated. For example, I-BAR domains are almost flat but generate much higher membrane curvature. Furthermore, F-BAR and I-BAR domains are quite similar, but the former generates positive curvature and the latter negative curvature. Here, we use implicit-solvent computer modeling to show that the membrane bending of the IRSP53 I-BAR domain is dictated by its oligomeric structure, whose curvature is completely unrelated to the intrinsic curvature of the dimer. We find that I-BAR dimers undergo lateral and end-to-end interactions to form higher oligomers. The lateral interactions, which can occur with varying register, are stronger and give a curved shape that has the membrane binding interface on its convex surface. Two other I-BARs (MIM and I-BARa) gave similar results, whereas a flat F-BAR sheet curved to make its membrane binding interface concave, consistent with positive membrane curvature generation. Individual dimers orient parallel to the tube axis when the tube radius is small, whereas perpendicular orientation is preferred for larger diameter tubes. This orientational preference exists even in the higher oligomeric form. Helical spirals on tube interiors are stable at a density comparable to experimental measurements. The sorting ratio computed from our membrane binding energies is also consistent with experimental measurements.