A distributed recommender system architecture

Recommender systems over distributed architectures

“…We have achieved the following contributions in FPM: a. iFP-Growth (Giannikopoulos et al, 2012), an incremental algorithm, not making use of the taxonomy information, which is about 90% faster than the state of the art, while requiring about 88% less memory, resulting in pattern loss less than 2% b. distributed FGP (Giannikopoulos & Vassilakis, 2012a), a distributed, taxonomy-aware frequent-pattern mining algorithm. This contribution has been straightly motivated by the performance shortcomings of the (centralized) FGP algorithm and considerably outperforms existing distributed frequentpattern mining algorithms, both in terms of speedup gain and frequentpattern quality (the latter stemming from the incorporation of the taxonomy information).…”

“…A workaround to the aforementioned problem might be the broadcasting of each entry to all processors, in order to determine the support values locally and subsequently send them to the arbitrator (algorithm SPA). Moreover, a common way to partition the original database is to apply hashing (Giannikopoulos & Vassilakis, 2012a), in order to determine the processor, which will be responsible for the corresponding transaction; nevertheless, the so-called HPA algorithm results in underutilization of the available memory of the entire system, in case the candidate itemset size is smaller than the aggregate memory of all the processors. This major drawback, along with the difficulty in achieving a fair load balance, is addressed in HPA-ELD, where, if the support of a candidate frequent itemset exceeds a predefined threshold, it is copied in all processors' hash tables and processed like in NPA.…”

“…Another distributed FP-Growth extension is presented in (Giannikopoulos et al, 2012a), which does not require the entire dataset to be collected in one computer, prior to its application. It operates by making each machine specialize in the time periods it has more entries, based on the histogram information exchanged among them, running FGP (Gianikopoulos et al, 2010) in the time entries (stemming from all machines) belonging to these periods only.…”

“…All potential appearances of each rule are finally collected in the computer determined by hashing on its antecedent, in order to sum up all of its potential occurrences in many machines. We shall present distributed FGP (Giannikopoulos et al, 2012a) in more detail in section 5.…”

“…It is worth mentioning that distributed FGP (Giannikopoulos et al, 2012a) can also be applied in sequential rule mining. This can be achieved by preserving the order the frequent items have been requested in each session during the construction of the corresponding tree structure in each machine, instead of inserting the items in each session in descending global support order.…”