Solving large systems of equations is a problem often encountered in engineering disciplines. However, as such systems grow, the effort required for finding a solution to them increases as well. In order to be able to cope with ever larger systems of equations, some form of decomposition is needed. By decomposing a large system into smaller subsystems, the total effort required for finding a solution may decrease. However, whether this is really the case of course depends on the additional effort required for obtaining the decomposition itself. In this thesis several aspects of the difficulty of obtaining such decompositions are explored.The first problem discussed in this thesis is that of the decomposition of a system of non-linear equations. After describing the well-known conditions for consistency in systems of equations, the decomposition problem for under-specified systems is analyzed. This analysis, based on the existing notion of free square blocks, leads to several W [1]-hardness results. The most important of which is the proof that finding a decomposition into subsystems where the largest subsystem is as small as possible is W [1]-hard. This implies that even if the size of such a largest subsystem is bounded by a constant, the problem is still not solvable in polynomial time under the current working hypothesis that W [1] FP T . As a by-product of these results two open problems regarding crown structures for the vertex cover problem are settled.Having investigated the non-linear case, attention is then shifted to the case of systems of linear equations. Such systems are commonly represented as matrices upon which pivot operations are performed. In such operations preserving sparsity of the input matrix is often beneficial to limit both storage and processing requirements. The choice of pivot elements can strongly influence the level of sparsity that is preserved. Changing a zero value in a matrix to a non-zero value is called fill-in. An important role in preserving sparsity is played by bisimplicial edges in the bipartite graph that corresponds naturally to a matrix. Such bisimplicial edges correspond to pivots in the matrix that completely avoid fill-in. In this thesis a new O (nm) algorithm for finding such bisimplicial edges for an n×n matrix with m non-zero values is described and analyzed on a common class vii of random matrices. It is shown that the expected running-time of the algorithm on matrices corresponding to bipartite graphs from the G n,n,p model is O n 2 , which is linear in the input size.After analyzing single pivots that avoid turning zero elements into nonzero elements, the class of matrices that allow Gaussian elimination using only such pivots is discussed. Such matrices correspond to prefect elimination bipartite graphs. Several algorithms are known for the recognition of this class of graphs or matrices, but most of them are based on matrix multiplication implying sparse input matrices may still result in dense matrices along the way. In this thesis two new algorith...