Laser material processing applications often require the flat-top beam profile within focal spots ranged below 100 μm. For high power applications, volume phase masks recorded in photo- thermo-refractive glass (PTR) are promising. The problem is how to achieve simultaneously a high-quality shape and a small size of the beam. The commercial phase masks usually show large power losses in the beam wings, only about 40% of the energy was concentrated under the 95% level. By applying a gray phase mask instead of binary mask, one can reduce losses in the wings of the beam. In this work, a spatial light modulator (SLM) with designed computer generated holograms (CGHs) was used as a beam converter. Using the SLM with programmed gray mask allows obtaining flexible laser beam shaping, but beam quality is limited by imposed parameters of the SLM. The requirements for obtaining a square flat-top beam with energy lost in wings less than 10% is described. It was found that for sharp edges of the square flat-top beam, it is necessary that the size of the output beam contains at least 16 pixels of SLM. This fact is a consequence of the Fourier transform, where high spatial frequencies are responsible for the shape. The concept design of the scanning progressive mechanism of the master volume phase mask recording to exclude the influence of SLM work area dimensions is discussed.