Analysts are being asked to assimilate more and more data into meaningful and actionable intelligence that can be acted upon quickly. This is a daunting task as the volume of data that comes into play is staggering and crippling to most analytic tools.
First generation data mining and analysis tools are typically difficult to use and involve extensive programming. Lacking an interactive analytical approach, they often limit the depth and breadth of an investigation by constraining the analysis to a predefined set of data and static operations. Further, most of these programs were designed as stovepipes. Disconnected from important data sources needed to conduct a comprehensive investigation, they support isolated investigations and prevent analysts from sharing insights and relevant findings.
Over the past few years, most of the innovation in analytics has been in the area of automated information analysis, a technique that removes the analyst from the equation and attempts to reveal relevant insights automatically. What we have found, however, is that in most investigative processes the single most important component is human judgment, which begs the question: “Where is the analyst-centric innovation?”
Next-generation tools must address the lack of dynamic integration, exploration, and collaboration while allowing analysts to apply their expertise and empowering them to quickly detect hidden relationships, collaborate with others, and act on the results. Three emerging technologies that address these issues are:
- Interactive visualization
- Unified data views
- Collaborative analysis
These three innovations comprise the pillars of a new approach to analysis called interactive analytics, a human-centric approach to analyzing data in support of better analysis and decision making. Interactive analytics holds great promise for quickly and effectively detecting hidden meanings, and is based on highly interactive visualizations that allow users to rapidly comprehend and identify unobvious patterns within the data. These visualizations include relationship or network graphs for link analysis and social networking, summary charts and heat maps for quantitative analysis, timelines for temporal analysis, and maps for geospatial context. This remarkable approach empowers analysts to apply their domain knowledge and experience while exploring all relevant data.
Interactive Visualization
Most analysis tools require one to know what one is looking for in advance and constrain the ability to differentiate noise from meaning. Should one want to explore the expanse of data, the tool falls short. Yet this is precisely what businesses must do – discover the unknown. Information visualization begins to address this issue by making use of visual metaphors to enhance the ability to detect patterns in data. Interactive visualization builds on this by freeing the analyst to interact directly with the visualizations to ask open-ended questions and pursue a line of inquiry. This has proven to be effective at allowing analysts to navigate, explore, and understand massive amounts of data. When something relevant is found, inferences are drawn almost instantly, allowing the analyst to work at the speed of the human brain.
Unified Data Views
Access to all relevant data pertaining to an investigation is crucial to accurately identify actionable intelligence. Because important facts often exist in unrelated systems, third-party data sources, including social networking sites, blogs, news wires, network traffic, etc., are increasingly important to analysts. The ability to access these data sources without extensive integration and programming is critical, as the absence of this capability often yields incomplete conclusions. A common complaint, however, is that the analyst needs to employ multiple tools, many of which require the time-consuming construction of complex ETL processes and data warehouses. Because this can be tedious and highly disruptive to a particular line of reasoning, providing the analyst with the ability to easily reach out to these sources from within the analysis framework to create a unified view is extremely powerful. Further, the ability to switch between multiple, integrated views of the same data is a powerful paradigm for visual analysis. Visualizations allow one to detect relevant patterns almost instantly. Integrated views allow one to “shift the lens,” such as moving from a quantitative to a relational to a temporal view of the same data. With this capability analysts can quickly validate findings and eliminate false positives.
Collaborative Analysis
Since analysts are often working on interrelated problems, it makes sense that if individuals can collaborate, alert each other to important findings, and make the results available, they can more quickly take action while advancing the analytic process. The ability to document the results of the investigation for audit purposes is also helpful in validating specific findings.
With interactive analytics the analyst’s brain serves as the ultimate pattern recognition machine. The technology allows for unconstrained investigative analysis across disparate data sets. Analysts are empowered to take control of the analytic process, apply their training, experience, and judgment to visualize and detect hidden relationships within the data, and collaborate with others. The result is a more rapid reporting of actionable intelligence. The approach also drastically improves the user experience, which has been too complicated. It is easily adoptable. It is consistent with the way analysts have been trained and think. Most important, it allows them to apply their knowledge and experience to the problem.
Interactive analytics has been put to the test in some of the most demanding cyber crime analysis, counter terrorism, and homeland security applications worldwide and has proven to be highly effective. It can also be applied to many other applications such as sales, service, life sciences, marketing analysis and advertising.
Increasing amounts of data coupled with shrinking windows of time to understand and act on the information it contains are driving businesses to seek new approaches to analytics. The ability to express relationships in a visual way is extremely powerful. Because of the enormous information challenges organizations face today, interactive analytics is taking on new meaning worldwide as analysts leverage technology to efficiently and effectively uncover hidden meanings within their data.
Author: Anthony Agresta is a Vice President with Centrifuge Systems, a provider of next-generation business intelligence software that helps organizations discover patterns and relationships hidden in data. Centrifuge’s blog on how businesses can use information visualization to improve decision making can be found at www.centrifugesystems.com/blogs/.