Scaling Up Machine Learning: Introduction

Bekkerman, Ron; Bilenko, Mikhail; Langford, John

doi:10.1017/cbo9781139042918.002

Cited by 35 publications

(40 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Developing optimally distributed and parallel implementations of existing algorithms (e.g., in the Hadoop or Spark frameworks) is important. Nevertheless, the demand for genuinely new algorithms is growing, too (Bekkerman et al, 2011).…”

Section: New Requirements For Data Mining Algorithmsmentioning

confidence: 99%

Exploring complex and big data

Stefanowski

Krawiec

Wrembel

2017

International Journal of Applied Mathematics and Computer Science

View full text Add to dashboard Cite

This paper shows how big data analysis opens a range of research and technological problems and calls for new approaches. We start with defining the essential properties of big data and discussing the main types of data involved. We then survey the dedicated solutions for storing and processing big data, including a data lake, virtual integration, and a polystore architecture. Difficulties in managing data quality and provenance are also highlighted. The characteristics of big data imply also specific requirements and challenges for data mining algorithms, which we address as well. The links with related areas, including data streams and deep learning, are discussed. The common theme that naturally emerges from this characterization is complexity. All in all, we consider it to be the truly defining feature of big data (posing particular research and technological challenges), which ultimately seems to be of greater importance than the sheer data volume.

show abstract

Section: New Requirements For Data Mining Algorithmsmentioning

confidence: 99%

Exploring complex and big data

Stefanowski

Krawiec

Wrembel

2017

International Journal of Applied Mathematics and Computer Science

View full text Add to dashboard Cite

show abstract

“…One of the standard problems is to scale up classical learning algorithms to huge problems in presence of distributed information (Bekkerman et al, 2012). It is beyond the scope of the paper to give an exhaustive overview about such cooperative ML approaches, only some of them, which were already applied to production and logistics problems, are highlighted.…”

Section: Cooperative Learningmentioning

confidence: 99%

Cooperative Control in Production and Logistics

Monostori

Valckenaers²,

Dolgui

et al. 2014

IFAC Proceedings Volumes

View full text Add to dashboard Cite

This is a milestone paper of the Coordinating Committee on Manufacturing and Logistics Systems (CC5) of the International Federation of Automatic Control (IFAC).Classical applications of control engineering and information and communication technology (ICT) in production and logistics are often done in a rigid, centralized and hierarchical way. These inflexible approaches are typically not able to cope with the complexities of the manufacturing environment, such as the instabilities, uncertainties and abrupt changes caused by internal and external disturbances, or a large number and variety of interacting, interdependent elements. A paradigm shift, e.g., novel organizing principles and methods, is needed for supporting the interoperability of dynamic alliances of agile and networked systems. Several solution proposals argue that the future of manufacturing and logistics lies in network-like, dynamic, open and reconfigurable systems of cooperative autonomous entities.The paper overviews various distributed approaches and technologies of control engineering and ICT that can support the realization of cooperative structures from the resource level to the level of networked enterprises. Standard results as well as recent advances from control theory, through cooperative game theory, distributed machine learning to holonic systems, cooperative enterprise modelling, system integration, and autonomous logistics processes are surveyed. A special emphasis is put on the theoretical developments and industrial applications of Robustly Feasible Model Predictive Control (RFMPC). Two case studies are also discussed: i) a holonic, PROSA-based approach to generate short-term forecasts for an additive manufacturing system by means of a delegate multi-agent system (D-MAS); and ii) an application of distributed RFMPC to a drinking water distribution system.

show abstract

“…Using the HDFS client [17], user applications access the file system. Similar to most conventional file systems, HDFS is having provisions of reading, writing and deleting files, and provides operations for creating and deleting directories.…”

Section: Hdfs Clientmentioning

confidence: 99%

“…MAP-REDUCE programming model [17] is defined by Dean et al MAP-REDUCE computing model comprises of two functions, Map ( ) and Reduce ( ) functions. The Map and Reduce functions are both defined with data structure of (key1; value1) pairs.…”

Section: Mr Programming Modelmentioning

confidence: 99%

Verification and Validation of MapReduce Program model for Parallel K-Means algorithm on Hadoop Cluster

Kumar¹,

Kiran²,

Mukherjee³

et al. 2013

IJCA

View full text Add to dashboard Cite

With the development of information technology, a large volume of data is growing and getting stored electronically. Thus, the data volumes processing by many applications will routinely cross the petabyte threshold range, in that case it would increase the computational requirements. Efficient processing algorithms and implementation techniques are the key in meeting the scalability and performance requirements in such scientific data analyses. So for the same here, it has been analyzed with the various MapReduce Programs and a parallel clustering algorithm (PKMeans) on Hadoop cluster, using the Concept of MapReduce. Here, in this experiment we have verified and validated various MapReduce applications like wordcount, grep, terasort and parallel K-Means Clustering Algorithm. It has been found that as the number of nodes increases the execution time decreases, but also some of the interesting cases has been found during the experiment and recorded the various performance change and drawn different performance graphs. This experiment is basically a research study of above MapReduce applications and also to verify and validate the MapReduce Program model for Parallel KMeans algorithm on Hadoop Cluster having four nodes.

show abstract

Scaling Up Machine Learning: Introduction

Cited by 35 publications

References 10 publications

Exploring complex and big data

Exploring complex and big data

Cooperative Control in Production and Logistics

Verification and Validation of MapReduce Program model for Parallel K-Means algorithm on Hadoop Cluster

Contact Info

Product

Resources

About