Dynamic and non-linear systems are emerging as potential candidates for random bit generation. In this context, chaotic systems, which are both dynamic and stochastic, are particularly suitable. This paper introduces a new continuous chaotic system along with its corresponding implementation, which targets fieldprogrammable gate arrays (FPGAs). This chaotic system has five dimensions, which exhibit complex chaotic dynamics, thus enabling the utilization of chaotic signals in cryptography. A mathematical analysis is presented to demonstrate the dynamic characteristics of the proposed hyperchaotic system. A novel digital implementation of the proposed system is presented. Moreover, a data scrambling circuit is implemented to eliminate the bias effect and increase the randomness of the bitstream generated from the chaotic signals. We show that the proposed random bit generator has high randomness. The generated bits successfully pass well-known statistical randomness testsuites, i.e., NIST SP800-22, Diehard and TestU01. The readyto-use random bit generator is deployed on a Xilinx Zynq-7000 SoC ZC702 Evaluation Kit. Experimental results show that the proposed random bit generator can achieve a maximum throughput of 6.78 Gbps, which is over 3.6 times greater than state-of-the-art designs, while requiring under 4% of the resources available on the targeted FPGA.