Considering the current era of mobile Internet, the prevalence of user access, such as accessing multimedia via high-speed Internet and, in particular, video contents, such as those from YouTube and Netflix, has become the norm. However, drastically increased video access can lead to several issues, e.g., long delays and low throughput, unless efficient Internet traffic management schemes are applied. One promising approach is based on caching efficacy built into the proxy server or content delivery networks. Several traditional caching policies derived from classic CPU caching are available; various policies provide simplicity gains but at a cost of low precision. Advances in storage technologies and CPU speedup have drawn attention to the latter aspect. This research thus focuses on a two-level caching scheme. The first level combines two classic caching models, Least Frequency Used (LFU) and Least Recently Used (LRU), to enhance the hit rate (HR) and byte hit rate (BHR) for real-time or (fast) online communication with small cache size constraints. Fuzzy logic (FL) is applied to derive a proper weight for this hybrid model. A Support Vector Machine (SVM) is also used for the second level to mainly focus on improving the replacement precision with more knowledge given the larger cache size. Some key features are selected, geographical distance and similarity hashing in particular; in addition, a proper period of training knowledge is selected for the SVM based on human behavior. The proposed scheme is evaluated and compared to two well-known caching schemes, LRU and LFU, in addition to other state-of-the-art intelligent hybrid caching models, e.g., SVM-LRU and SVM-LFU, using a well-known dataset from IRCache. The comparison is performed in terms of performance gain; the proposal has the highest HRs and BHRs, i.e., higher HRs and BHRs than LRU, LFU, SVM-LRU, and SVM-LFU.