In this work, a novel DOPO-based spiroorthocarbonate (DSOC), 6,6ʹ-((3,9-diethyl-1,5,7,11-tetraoxaspiro[5.5]undecane-3,9-diyl)bis(hydroxymethylene))bis(6H-dibenzo[c,e][1,2]oxaphosphinine 6-oxide) was synthesized and confirmed with 1 H, 31 P, 13 C NMR, and FT-IR spectra. It was used as anti-shrinkage additive and flame retardant in the cationic polymerization of epoxy resin. The shrinkage of the cured epoxy resin containing DSOC was investigated by measuring the density of the system before and after curing. It was found that the shrinkage of epoxy resin changed from −1.49 to 1.43% with the content of DSOC increased to 20 wt%, not only the shrinkage of epoxy resin was eliminated, but also a slight expansion was achieved. The flame retardancy and thermal stability of the cured epoxy resins were tested by limiting oxygen index (LOI), vertical burning test (UL-94), and thermogravimetric analysis (TGA). LOI values increased from 19.9% for pure E-51 to 30.9% for modified epoxy resins, UL-94 V-0 rating can be obtained with 15 wt% DSOC. The mechanical properties of the modified epoxy resin were also investigated. The results showed that DSOC was a promising anti-shrinkage additive and flame retardant for epoxy resins.