In this paper, we propose a polarization-independent optoelectronic modulator based on the electrical absorption effect of graphene. Firstly, we use the simulation software COMSOL Multiphysics to design the structure, and find via changing the applied voltage on both ends of the graphene that the equivalent refractive index of graphene can be changed, thus changing the light absorption capacity of the modulator. The waveguides in the transverse magnetic (TM) and transverse electric (TE) modes have almost the same extinction coefficient by making a double-layer graphene ridge structure in the center of the silicon-based waveguide, which can achieve approaching modulation depth in the TM and TE modes. At 1550 nm wavelength, the two-dimensional cross-section of the structure is analyzed by the FEM method using COMSOL Multiphysics to obtain the effective refractive index of the structure. The simulation results show that when the distance between the double-layer graphene isolation layer is d = 20 nm, the TE and TM modes can achieve extinction ratios up to 110 dB over the wide communication band by selecting appropriate “ON” and “OFF” switching points. The bandwidth is 173.78 GHz and the insertion loss is only 0.0338 dB.