Situational Awareness Interface (SAI)

Laboratory for Analytical Sciences - Machine Learning Interface for Data Analysts

How might the design of an interface use the affordances of Machine Learning (ML) to provide a personalized user experience so that the analyst might quickly and knowledgeably enter the day’s workflow?

This was a class project at NCSU that I completed with graphic design students Amanda Williams and Riley Walman. *Please note that the data and details depicted in these personas and use cases are based on unclassified, fictitious scenarios.

User Journey Map

The Project

Persona

Nyah, a Senior Search + Discovery Analyst

Use Case

As Surge Team member, Nyah has received instructions to investigate a crisis in the fictional country of Kronos where GASTech employees have been kidnapped.
The early stages of a surge are often very chaotic, so Nyah needs to efficiently gain contextual awareness of the event in order to assess the situation and discover the location of the missing employees and how to get them home.

Identified Pain Points

  • Difficult to get a case overview with the amount of initial available information in multiple windows or tabs

  • Strenuous to immediately identify anomalous activity in the case

  • Copious amounts and iterations of queries need to run in short period of time

Scenario Video: Situational Awareness Interface

AI-Prioritized Content

AI customizes tree map display of patterns based on key prioritization and also uses scale as a secondary identifier of priority.

AI-Prioritized Key

Priority is established by ML based on what the analyst needs to know to efficiently gain contextual awareness.

Accessibility

Interface colors are accessible for analysts with visual impairments.

Textual Summaries

AI generates summaries from all case data to provide an overview of the situation. It also sifts through customer interest analyses to incorporate customer interest trends. The supervisor summary provides the human perspective from the analyst’s supervisor.

Customized Display

Analyst can adjust AI-selected features using the menu.

Interface Features

Each square is an event relating to the crisis. Analysts can scroll though the timeline to see major events in the case.

AI highlights high priority events important for the analyst to gain situational awareness based on their preferences/feedback. ‘Predicted behaviors and events’ are available to analysts if they scroll past the present and into the future.

Interactive Timeline

Points of Interest

Feature shows high priority locations in the case and AI-predicted routes between locations.

Image vs. Text View

Offers an alternative view for analysts who prefer textually-presented data.

Text View

Image View

Wider Implications

  • Summaries provide top-level overview while other features equip analyst to
    drill down into specific data points

  • Enables the analyst to intuitively understand the prioritization system via color and hierarchy

  • Using ML provides rich opportunities for identifying and responding to anomalous behavior in the case

Reflection

  • How could SAI incorporate cross-collaboration between analysts?

  • In addition to using the watch’s proximity sensing to enhance data security (monitor locking), how else might the watch be leveraged to support security and awareness?

  • In addition to the 24-hour updating cycle, how might SAI streamline the reporting process?

Interface Still