Let K ⊂ R d be a bounded set with positive Lebesgue measure. Let Λ = M (Z 2d ) be a lattice in R 2d with density dens(Λ) = 1. It is well-known that if M is a diagonal block matrix with diagonal matrices A and B, then G(|K| −1/2 χK , Λ) is an orthonormal basis for L 2 (R d ) if and only if K tiles both by A(Z d ) and B −t (Z d ). However, there has not been any intensive study when M is not a diagonal matrix. We investigate this problem for a large class of important cases of M . In particular, if M is any lower block triangular matrix with diagonal matrices A and B, we prove that if G(|K| −1/2 χK , Λ) is an orthonormal basis, then K can be written as a finite union of fundamental domains of A(Z d ) and at the same time, as a finite union of fundamental domains of B −t (Z d ). If A t B is an integer matrix, then there is only one common fundamental domain, which means K tiles by a lattice and is spectral. However, surprisingly, we will also illustrate by an example that a union of more than one fundamental domain is also possible. We also provide a constructive way for forming a Gabor window function for a given upper triangular lattice. Our study is related to a Fuglede's type problem in Gabor setting and we give a partial answer to this problem in the case of lattices.