The journey of a policy re-design
The problem
Problem: The current policy system in Stealthwatch consists of over 20 pages and widgets in which policy parameters can be created, read, updated, and deleted (CRUD). These policy elements will conflict with each other, and having full visibility into your system and what it is actually configured to do is near impossible. The biggest issue among customers was noise. The ability to use policy to tune out the noise is difficult at best. While it can be used with surgical precision, it is often used by trial and error.
Task: Redesign the policy system in order for user to gain greater insight into their system and provide them with the ability to easily and intuitively tune out the noise in the system. Allow the users to see what they want to see, and hide the rest.
The solution
The solution utilized deep examination of the current policy system, a lot of contextual inquiry (in-person and remote) to uncover not only the obvious problems, but the smaller ones as well. We had to understand each users goals when it came to creating policy, as they did differ across customers. We had to get at the root problem of not only the current policy UI, but at what users' intentions for policies were, their mental models, what worked and made sense, and what did not.
From there we had to figure out how to solve that problem, including the obvious problem of over 20 different types of policy that could be applied, each with potentially contradicting rules. We uncovered users' frustrations with unintended consequences and blind spots. For example if there was a denial rule in place in a policy, the user could remove that rule and there would still be a denial because one of the OTHER applied policies had that same denial rule in place, and the user had no idea!
We needed to break this down, and reassemble the pieces, getting rid of the right parts, finding overlaps, in order to build a much more efficient and intuitive policy model. Almost immediately we 20 policy types down to four, just based on removing redundancies and a more intuitive policy framework. Then we needed to design a way to turn intricate, complex functions into something that our users could use and build with right out of the box with little to no prior product training, and iterate on that many times over.
The steps are outlined below, and a larger image of all of these steps in available at the bottom of the page.
Step 1
Click image to enlarge
Examining the current policy model
Policies were confusing. There were upwards of 4 per host group. Some users had 20,000 groups. This was unsustainable. Customer were confused and could not accomplish their goals with the product. FOr some it was infinitely flexible, for other it was completely unusable. Either way it was a mess of screens and contradictions and overlaps.
Step 2
Click image to enlarge
Deconstructing the current policy model
Research started with untangling the twisted web that was policy and developing a real understanding of the underlying architecture and its failings. We had to diagram out typical workflows and policy realtionships such as how do hosts relate to policies and rules, and how do policies relate to hosts and groups and rules and how do alarms relate to all of those components
Step 3
Click image to enlarge
Initial prototype (retaining the current policy engine)
The first stab involved a tabbed interface that allowed the user, from one page and not 20+, to see the current policies from a host or group point of view, a policy rule POV, and see them across all 4 policy types.
In addition, based upon user input, it proposed a new method for policy deployment
Step 4
Click image to enlarge
Revamping and simplifying the user workflows
It quickly became clear that the approach of simply building a new UI around the current, broken system was not the right solution for the long- term. We developed a new policy system from scratch that would accomplish the same goals, but in an intuitive, simple, and friendly manner.
It was determined that to move forward, we needed to design 2 approaches: an approach using a brand new architecture that was forward-looking, flexible, and simple, and an approach utilizing the current architecture as a fallback plan for the realities of the release and development cycles and to account for shifting priorities and other time constraints.
Step 5
Click image to enlarge
More iterations, and a new UI prototype
The fallback approach was designed and tested with user input and resulted in a much improved interface built to handle the unweildy policy engine.
20+ screens were distilled down to ~5.
It was still the same faulty policy model, but it was also a vast improvement over the previous UI scheme. The feedback was mixed. Some were glad to see any improvement, and some were upset that it was still the old policy model underneath it all.
Step 6
Click image to enlarge
Iterate prototypes based on a policy re-architecture
The second approach was deisnged which required a new policy model in order to execute. However, it tool all of the previous models functionality and distilled it down to 3 screens: a master rule list page, only 1 type of rule instead of 4 or 5, and a unified visual rule editor that was an order of magnitude improvement over the current policy model. It was future looking, flexible, and removed the confusing and conflicting hierarchical model and utilized tagging instead. It was far more flexible than the old system as well.
Step 7a
Click image to enlarge
Many iterations later: Re-architecting approach
The fallback approach was designed and tested with user input and resulted in a much improved interface built to handle the unweildy policy engine.
20+ screens were distilled down to ~5.
It was still the same faulty policy model, but it was also a vast improvement over the previous UI scheme. The feedback was mixed. Some were glad to see any improvement, and some were upset that it was still the old policy model underneath it all.
Step 7b
Click image to enlarge
Many iterations later: current policy model approach
The second approach was deisnged which required a new policy model in order to execute. However, it tool all of the previous models functionality and distilled it down to 3 screens: a master rule list page, only 1 type of rule instead of 4 or 5, and a unified visual rule editor that was an order of magnitude improvement over the current policy model. It was future looking, flexible, and removed the confusing and conflicting hierarchical model and utilized tagging instead. It was far more flexible than the old system as well.
The full journey. Click to see the full-sized image (opens in new tab)