Measurement of fluctuations in diffuse Hi 21 cm background radiation from the post-reionization epoch (z ≤ 6) is a promising avenue to probe the large scale structure of the Universe and understand the evolution of galaxies. We observe the European Large-Area ISO Survey-North 1 (ELAIS-N1) field at 300-500 MHz using the upgraded Giant Meterwave Radio Telescope (uGMRT) and employ the 'foreground avoidance' technique to estimate the Hi 21 cm power spectrum in the redshift range z = 1.96 − 3.58. Given the possible systematics that may remain in the data, we find the most stringent upper limits on the spherically averaged 21 cm power spectra at k ∼ 1.0 Mpc −1 are (58.87 mK) 2 , (61.49 mK) 2 , (60.89 mK) 2 , (105.85 mK) 2 at z = 1.96, 2.19, 2.62 and 3.58, respectively. We use this to constrain the product of neutral Hi mass density (Ω HI ) and Hi bias (b HI ) to the underlying dark matter density field, [Ω HI b HI ], as 0.09,0.11,0.12,0.24 at z = 1. 96, 2.19, 2.62, 3.58, respectively. To the best of our knowledge these are the first limits on the Hi 21 cm power spectra at the redshift range z = 1.96 − 3.58 and would play a significant role to constrain the models of galaxy formation and evolution.