System S is a large-scale distributed streaming data analysis environment. Ultimately, we envision that there will be multiple sites running the System S software, each with their own administration and goals. However, cooperation between these sites can frequently be of mutual benefit. We are designing the framework to support numerous sites that can work both independently and in cooperative fashions, with a variety of interaction models such as peer-to-peer or federated. Depending on the degree of site autonomy and the relationships between any given pair of sites, the sites would be able to share data, perform processing on behalf of one another, or even take over tasks of a failed site. Interoperability is complicated by the degree of site autonomy as well as differences in execution environments and security policies. This paper surveys System S, describes its models for site interoperability, and discusses particular design issues such as site failover and heterogeneity.
The crux of large scale Application Layer Multicast or Peer-to-Peer streaming systems is how to cope with the inherent dynamics, the reason is that the participating users may join and leave at will. It is even worse for single-tree-based multicast systems, which are preferred due to their efficiency. In these single multicast tree based schemes, user's departure may cause serious service disruption for all the downstream users. The solution stems from the characteristics of the problem itself, and it exploits the property that the participating users' lifetime follow a Pareto distribution, which has the used better than new (UBTN) feature. The participating nodes are dynamically organized into a hierarchy in such a way that it reflects the relative stabilities among the nodes. The proposed algorithm is distributed in the sense that no a prior knowledge about users' lifetime is needed. A maximum of 50% improvement can be achieved in terms of peers' perceived QoS. Detailed mathematical analysis and simulation results are presented to validate the proposed algorithm. Simulation results show that the algorithm is valid for other lifetime distributions as well.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.