We present a family of attacks on lightweight encryption schemes for visual data that rely on wavelet filter parameterizations to provide security. All of the attacks construct a symbolic representation of the inverse wavelet transform. We show that this representation can be used in ciphertext-only attacks, known-plaintext attacks and in attacks in which some information on the plaintext is available. We investigate the success and feasibility of each of these attacks, and conclude that the presented type of attacks poses a principal problem for lightweight encryption schemes that rely on the parameterization of a (linear) transform.