Integrating legacy system into big data solutions: Time to make the change

Abstract

Storing, analyzing and accessing data is a growing problem for organizations. Competitive pressures and new regulations are requiring organizations to efficiently handle increasing volumes and varieties of data, but this doesn't come cheap. And as the demands of Big Data exceed the constraints of traditional relational databases, evaluating legacy infrastructure and assessing new technology has become a necessity for most organizations, not only to gain competitive advantage, but also for compliance purposes. The challenge is how well the organization's legacy infrastructure integrates Big Data. It is without a doubt that one way or another Big Data must be accommodated by legacy systems. Legacy systems contain significant and invaluable business logic of the organization. Organizations cannot afford to throw away or replace this business logic. These legacy systems are assets of the organization. These invaluable assets of encoded "business logic" represent many years of coding, development, real-life experiences, enhancements, modifications, debugging etc. Most of the legacy systems were developed without the process models or data models-now needed to support and integrate Big Data. To integrate Big Data into legacy system, modernization of legacy system is required. There are many approaches for modernization of legacy systems but none of them are focused on integrating Big Data into legacy systems. Legacy systems hold valuable data too important to be lost in the process of modernization. However, addressing the issues and scope related to incorporating Big Data with legacy systems allows mature legacy systems to become part of groundswell changes. There are many areas unaddressed about integration of Big Data into legacy systems. Incorporating data from new sources, specifically "live" sources, into existing legacy systems is a technical challenge. Moreover, the sheer volume of Big Data can be daunting. Our paper presents the scope of integrating Big Data into modernization of legacy systems.

A Guidance to Legacy Systems Modernization

“…Several issues, methods, approaches, and guidelines on legacy systems have been discussed and developed [17], [18], [19], [20], [21]. Similarly, studies that are focusing in legacy systems modernization are rapidly attracting researchers [4], [16], [17], [22] to make sure the systems are aligned to the latest technology development.…”

“…Even though legacy systems cause technical problems, these systems are important assets of organizations. They cannot easily eliminate or avoid the use of legacy systems because the systems contain essential business information and data since implemented [4], [5], [6]. Any failure caused by the systems will have serious consequences in running daily business tasks.…”

Current Issues in Software Re-Usability: A Critical Review of the Methodological & Legal Issues

“…Reuse is still possible without documentation as long as domain expertise within the team remains adequate, though without documentation there are legacy issues if key members of the development team leave the business, taking with them their knowledge and expertise. This is an important unsolved issue with legacy systems where experts have left and business and the documentation is out of sync, leaving a significant gap within the team over how to maintain and evolve the system [40] [41] [42]. The stability of the business and its software must be considered here, with stability vital to ensure business continuity [43] (Table 2).…”

Big data analytics architecture design—An application in manufacturing systems

“…Once chosen, such techniques can supply inputs to the second step of our approach as decision alternatives for the goal operationalization or obstacle resolution (see third column of Table 4 for example) where their impact on quality goals is investigated for the optimum selection of solution architecture. Jha et al define both forward and backward reengineering activities through which legacy system functionalities are reused, and their data can be accessed and processed by big data analytics platforms (Jha, Jha, O'Brien et al 2014). They suggest a framework to construct an architectural view of big data solution including business, data, and application architecture (Jha, Jha and O'Brien 2015).…”

“…Thence, reluctance of manufacturing organisations in moving to these platforms is unsurprising. Some are also still figuring out what kind of data is worthy for advanced data analytics and which stage of product lifecycle management is suitable to utilize big data analytics platforms (Govindarajan, Ferrer, Xu et al 2016), (Jha, Jha, O'Brien et al 2014), (Bi and Cochran 2014). It has been a long-standing acknowledgement that a poor system upgrade with a new technology can have far reaching consequences in later stages that are costly to rectify.…”