This thesis contains material from 5 paper(s) published or accepted in the following peer-reviewed journal and conferences in which I am listed as an author. Chapter 3 is partially published as Shiqing Li, Di Liu and Weichen Liu, "Optimized Data Reuse via Reordering for Sparse Matrix-Vector Multiplication on FP-GAs" in International Conference on Computer-Aided Design (ICCAD) 2021 and is partially published as Shiqing Li, Di Liu and Weichen Liu, "Efficient FPGAbased Sparse Matrix-Vector Multiplication with Data Reuse-aware Compression" in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD) 2023. The contributions of the co-authors are as follows: • Prof. Weichen Liu provided the initial idea of accelerating sparse-matrix dense-vector multiplication on embedded FPGAs. • I proposed the data reordering algorithm and the data reuse-aware compression. Meanwhile, I conduct experiments on FPGAs. Dr. Di Liu and Prof. Weichen Liu provided insightful comments about the idea and experiments. • I drafted the manuscript. Dr. Di Liu and Prof. Weichen Liu helped polish the manuscript. Chapter 4 is partially published as Shiqing Li and Weichen Liu, "Accelerating Gustavson-based SpMM on Embedded FPGAs with Element-wise Parallelism and Access Pattern-aware Caches", in Design, Automation and Test in Europe (DATE) 2023 and is partially published as Shiqing Li, Shuo Huai and Weichen Liu, "An Efficient Gustavson-based Sparse Matrix-matrix Multiplication Accelerator on Embedded FPGAs", in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD) 2023.The contributions of the co-authors are as follows:• Prof. Weichen Liu provided the initial idea of accelerating sparse-matrix sparse-matrix multiplication on embedded FPGAs. • I proposed to perform the Gustavson's algorithm with element-wise parallelism, access pattern-aware caches and optimized mergers. I further conduct experiments on FPGAs. Shuo Huai helped conduct experiments. • I drafted the manuscript. Prof. Weichen Liu and Shuo Huai helped polish the manuscript.