The Galactic magnetar SGR 1935+2154 has been reported to produce the first known example of a bright millisecond duration radio burst (FRB 200428) similar to the cosmological population of fast radio bursts (FRBs), bolstering the association of FRBs to active magnetars. The detection of a coincident bright X-ray burst has revealed the first observed multi-wavelength counterpart of a FRB. However, the search for similar emission at optical wavelengths has been hampered by the high inferred extinction on the line of sight. Here, we present results from the first search for second-timescale emission from the source at near-infrared wavelengths using the Palomar Gattini-IR observing system in J-band, made possible by a recently implemented detector read-out mode that allowed for short exposure times of ≈ 0.84 s with 99.9% observing efficiency. With a total observing time of ≈ 12 hours (≈ 47728 images) on source during its 2020 outburst, we place median 3 σ limits on the second-timescale emission of 20 mJy (13.1 AB mag). We present non-detection limits from epochs of four simultaneous X-ray bursts detected by the Insight-HXMT and NuSTAR telescopes during our observing campaign. The limits translate to an observed fluence limit of 18 Jy ms, while the corresponding extinction corrected limit is 125 Jy ms for an estimated extinction of A J = 2.0 mag. These limits provide the most stringent constraints (energy 3×10 36 erg at 9 kpc) to date on the fluence of flares at frequencies of ∼ 10 14 Hz, and constrain the ratio of the near-infrared (NIR) fluence to that of coincident X-ray bursts to R NIR 2.5 × 10 −2 . Our observations were sensitive enough to easily detect a near-infrared counterpart of FRB 200428 if the NIR emission falls on the same power law as that observed across its radio to X-ray spectrum. The non-detection of NIR emission around the coincident X-ray bursts constrains the fluence index of the brightest burst to be steeper than ≈ 0.35.