Galapagos: Model-driven discovery of end-to-end application-storage relationships in distributed systems

Abstract

Modern business information systems are typically multi-tiered distributed systems comprising Web services, application services, databases, enterprise information systems, file systems, storage controllers, and other storage systems. In such environments, data is stored in different forms at multiple tiers, with each tier associated with some level of data abstraction. An information entity owned by an application generally maps to several data entities, logically associated across tiers and related to the application. Discovery of such relationships in a distributed system is a challenging problem, complicated by the widespread adoption of virtualization technologies and by the traditional tendency to manage each tier as an independent domain. In this paper, we present a system and methodology for model-driven discovery of end-to-end application-data relationships spanning multiple tiers, from the applications to the lowest levels of the storage hierarchy. The key to our methodology involves modeling how data is used and transformed by distributed software components. An important benefit of our system, which we call Galapagos, is the ability to reflect business decisions expressed at the application level to the level of storage.

Automated Discovery and Maintenance of Enterprise Topology Graphs

“…Other approaches exist for specific domains of enterprise IT, for example, by Joukov et al for Java EE applications [12] and storage [10]. Galapagos [11] is a template-based approach focusing on mapping the data dependencies and storage locations of application data. There exists a multitude of tools to extract information about a certain operating system or machine, we are naming only a few here: Httprint and Nmap have already been integrated in our framework to enable identification of Web servers and operating systems, find open ports of running services, and so on.…”

“…Such models can be discovered manually, but this is a time-consuming, costly, and error-prone task [7]. Existing solutions are either restricted to a particular scope (e. g., discovery on the level of network [8], [9], storage [10], [11], application configuration [7], or application [12]), focus on other aspects than analyzing, optimizing, and adapting IT (e. g., monitoring, license management, vulnerability scanner), address a different level of granularity (e. g., enterprise architecture documentation [3], [13], [14]), or do not discover instance information at all (e. g., software architecture reconstruction [15] or application topologies in deployment automation solutions [16], [17]). An approach covering enterprise IT from business process to infrastructure offering an extensible framework for existing solutions that enables developers to provide plugins has not been created yet.…”

A Configuration Crawler for Virtual Appliances in Compute Clouds

“…The approach uses a repository of appliance configurations that has been build based on the Galapagos system [15] that only detects few well-known enterprise software packages [16]. The notion of agents and automated crawling with the purpose to provide a configuration model database for a variety of applications is missing in the approach.…”

Managing business health in the presence of malicious attacks

“…The second step is to identify the dependencies that each IT entry point has on other IT system components. Automated discovery systems such as Galapagos [5] can be used to discover fine-grained dependency information between applications, middleware, and data. In the case of the catalog browsing activity, such a discovery system will indicate that the web server forwards requests to an application server, which in turn accesses data from a database server.…”

“…Such documentation is not only rare in practice but, even if present, also difficult to keep up-to-date. With the advent of automated discovery tools for fine-grained application dependencies, like Galapagos [3], [5], the ability to perform this mapping for largescale IT infrastructures is only now starting to become a reality. Second, it requires understanding and quantification of the impact of violation of dependability and security (i.e., business health) supported by the IT system components underlying these business processes.…”