A visual analytics paradigm enabling trillion-edge graph exploration

Wong, Pak Chung; Haglin, David J.; Gillen, David; Chavarria, Daniel; Castellana, Vito Giovanni; Joslyn, Cliff; Chappell, Alan R.; Zhang, Song

doi:10.1109/ldav.2015.7348072

Cited by 9 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Today, researchers use computer clusters with as much as 1 TB of memory (or more) per computer node for high dimensional, in-memory database queries in interactive response time. For example, T-Rex (Wong et al, 2015) is able to query billions of data records in interactive response time using a Resource Description Framework 1 RDF 2014 database and the SPARQL (2008) query language running on a Linux cluster with 32 nodes of Intel Xeon processors and ∼24.5 TB of memory installed across the 32 nodes. Because such a large amount of information can be queued from a database in interactive time, the role of data warehouses continues to diminish in the big data era and as cloud computing becomes the norm.…”

Section: In-memory Computationmentioning

confidence: 99%

The Argument for a “Data Cube” for Large-Scale Psychometric Data

et al. 2019

Self Cite

View full text Add to dashboard Cite

In recent years, work with educational testing data has changed due to the affordances provided by technology, the availability of large data sets, and by the advances made in data mining and machine learning. Consequently, data analysis has moved from traditional psychometrics to computational psychometrics. Despite advances in the methodology and the availability of the large data sets collected at each administration, the way assessment data is collected, stored, and analyzed by testing organizations is not conducive to these real-time, data intensive computational methods that can reveal new patterns and information about students. In this paper, we propose a new way to label, collect, and store data from large scale educational learning and assessment systems (LAS) using the concept of the "data cube." This paradigm will make the application of machine-learning, learning analytics, and complex analyses possible. It will also allow for storing the content for tests (items) and instruction (videos, simulations, items with scaffolds) as data, which opens up new avenues for personalized learning. This data paradigm will allow us to innovate at a scale far beyond the hypothesis-driven, small-scale research that has characterized educational research in the past.

show abstract

Section: In-memory Computationmentioning

confidence: 99%

The Argument for a “Data Cube” for Large-Scale Psychometric Data

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…These are used for multiple tasks such as network performance monitoring and used as a means of security evaluation when an incident has been detected. Visualization of the NetFlow has also proven to be of tangible benefit [23]. Using this aggregated data for anomaly detection has numerous benefits, such as data size being reduced for processing purposes and storage.…”

Section: Cisco Netflowmentioning

confidence: 99%

NetFlow Anomaly Detection Though Parallel Cluster Density Analysis in Continuous Time-Series

Flanagan

Fallon

Connolly

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The increase in malicious network based attacks has resulted in a growing interest in network anomaly detection. The ability to detect unauthorized or malicious activity on a network is of importance to any organization. With the increase in novel attacks, anomaly detection techniques can be more successful in detecting unknown malicious activity in comparison to traditional signature based methods. However, in a real-world environment, there are many variables that cannot be simulated. This paper proposes an architecture where parallel clustering algorithms work concurrently in order to detect abnormalities that may be lost while traversing over time-series windows. The presented results describe the NetFlow activity of the NPD Group, Inc. over a 24-hour period. The presented results contain real-world anomalies that were detected.

show abstract

“…Moreover, if any u, w neighbors, to be merged into a vertex v, having degrees at most ffiffiffiffi m p , were instead tested for fu, wg 2 E, which is possible in O(1) time, the runtime would be O(m 3 2 ), which is optimal. 24 Recent development is now encroaching on trillionedge scale graph exploration, 27 which could admit the navigation of trillions of triangles in a graph. The multiscale layout algorithm of Wong et al 27 is reported to be similar to the merging algorithm in GreenHornet and could therefore be extended in the aforementioned merging proposal of neighboring vertices to identify triangles.…”

Section: Visualizing Trillions Of Trianglesmentioning

confidence: 99%

Graphing trillions of triangles

Burkhardt

2016

Information Visualization

View full text Add to dashboard Cite

The increasing size of Big Data is often heralded but how data are transformed and represented is also profoundly important to knowledge discovery, and this is exemplified in Big Graph analytics. Much attention has been placed on the scale of the input graph but the product of a graph algorithm can be many times larger than the input. This is true for many graph problems, such as listing all triangles in a graph. Enabling scalable graph exploration for Big Graphs requires new approaches to algorithms, architectures, and visual analytics. A brief tutorial is given to aid the argument for thoughtful representation of data in the context of graph analysis. Then a new algebraic method to reduce the arithmetic operations in counting and listing triangles in graphs is introduced. Additionally, a scalable triangle listing algorithm in the MapReduce model will be presented followed by a description of the experiments with that algorithm that led to the current largest and fastest triangle listing benchmarks to date. Finally, a method for identifying triangles in new visual graph exploration technologies is proposed.

show abstract

A visual analytics paradigm enabling trillion-edge graph exploration

Cited by 9 publications

References 14 publications

The Argument for a “Data Cube” for Large-Scale Psychometric Data

The Argument for a “Data Cube” for Large-Scale Psychometric Data

NetFlow Anomaly Detection Though Parallel Cluster Density Analysis in Continuous Time-Series

Graphing trillions of triangles

Contact Info

Product

Resources

About