This paper investigates modeling the conditional volatility of the Bangladesh equity market, namely the Dhaka Stock Exchange benchmark index (DSEX) and the Shariah Index (DSES), to explore the presence of leverage effects and long memory behavior covering the period from July 01, 2004 to December 31, 2020. We employ a family of Fractionally Integrated GARCH models FIFARCH BBM, FIGARCH CHUNG, FIEGARCH, FIAPARCH BBM, FIAPARCH CHUNG, and HYGARCH to capture both asymmetric effects and long memory behavior in conditional variance, a unique study in volatility literature. We detect strong evidence in favor of asymmetric effects and long memory behavior in the conditional volatility of DSEX and DSES returns, which repudiates the weak-form efficient market hypothesis. The study reveals FIEGARCH and FIAPRACH CHUNG outperform the other fractionally integrated GARCH specifications in modeling conditional volatility of equity returns. The paper further examines the diagnostic test of misspecification of the conditional variance equation based on the news impact curve, and results ensure that all models are fairly specified. This study also looks into the risk-return tradeoff in time-varying volatility and finds no evidence of the positive relationship between equity returns and volatility dynamics. The findings have pragmatic implications for retail and wholesale investors and other market players to initiate the investments and hedging strategies before investing in an emerging equity market like Bangladesh.