Visual SQL
Visual SQL is the chart builder interface in Chartio, a SaaS dashboarding tool.
Over the course of one year, I worked with a team of engineers and a designer to completely rethink how people build charts.
My roles
Product management
Project management
User research
Team
4 engineers during prototyping, 8+ during final build
1 designer
Duration
1 year: 2019–early 2020
Research
10+ customer interviews
10+ rounds of unmoderated usability studies
15+ moderated usability studies
2 React prototypes
The Challenge
Chartio’s chart builder had existed in more or less the same form since its inception in 2010.
Though we had won awards for usability, engagement among new users was low. In 2018, the percentage of new users who created a chart was about 10%. Usage of the Data Pipeline (post-processing step)—arguably our most powerful feature—was even lower.
Goals
More chart builders
Increase the percent of new users who create a chart
Faster chart building
For users who do create a chart, lower the average time-to-first-chart
Increase Data Pipeline usage
Increase the percentage of charts with at least one Data Pipeline step
Original editor, before the redesign
User Feedback
Getting baseline data
I had previously worked in support, so I had good foundational knowledge of new user pain points. To verify what we knew, I ran unmoderated usability tests on our current application, and interviewed several customers who were Chartio trainers within their organization.
Findings
Selecting columns
Users didn’t fully understand the concept of “Measures and Dimensions” and how they related to their chart.
Users struggle to select the correct data in a complex database.
Data Pipeline
The Data Pipeline button does not indicate what it actually does
The lack of prominence (underneath the chart builder) makes the feature easy to miss.
Ideation
Participants: 2 engineers, the CEO, the VP of Product, and myself
We explored a variety of chart building concepts, with an emphasis on methods that produce a chart the fastest. We created a few simple prototypes, but they all failed for one major reason: users struggled with column selection order (measures vs. dimensions). If it wasn’t exactly right, the chart would not build.
Measures and Dimensions drop zones
Getting rid of Measures and Dimensions
Seeing the abysmal test results, I wondered if we could remove the need to choose “Measure” or “Dimension” when selecting columns. Based on the data type, we should be able to predict which category the column falls into. When I checked the data, customers used the columns the way we predicted about 85% of the time.
That number was more than good enough for default settings.
Prototype
We quickly cycled from an Invision prototype to a simple React prototype to a more complex React prototype with real querying support. We found that the test results weren’t usable unless the prototype simulated fetching data; the interactions we needed to test were too complex.
Early prototype without querying support
Later prototype with querying support
Usability testing
Once we had a solid prototype with querying capabilities, we were able to run tests and iterate within days. I enforced a one-change-per-test rule to keep the process scientific.
We completed dozens of iterations over the course of several months.
Iteration process
Run 5-10 unmoderated tests on UserTesting.com
Take notes on the patterns observed
Discuss patterns with engineers and designer, propose changes—ideally the smallest change possible to resolve the issue
Implement change, test, compare results
Decision making
We relied on our testing process to help solve decisions as well. If the team was split on an approach, and neither was much effort to code, we would test both and compare results. It allowed us to remove a lot of bias from the decision making process.
Introducing Visual SQL
By the end of 2019, the majority of testers (8 out of 10) were able to make a chart and complete a post-processing step. Tests with longtime customers yielded positive results as well; trainers were confident that we had addressed the major stumbling blocks.
An improved way to browse
Select columns right from the table data itself, so you can be sure you’re pulling the right data.
One-step chart building
Users simply select the columns they want to include in the chart, and we’ll use smart defaults for the column grouping and aggregations.
Seamless transition to post-processing
Apply updates to your table directly from the table itself, in a familiar Excel-like format.
Learnings
Asking the right questions
In both moderated and unmoderated testing, I learned the importance of phrasing a task correctly. Subtle changes of phrasing altered the results. Tasks should be succinct and direct, with no hints of the action required.
I also learned that asking “Is there anything else you’d like to add?” at the end often prompted interesting feedback.
The importance of guiding principles
Early in the process, we created a set of seven principles we wanted the app to embody. We chose characteristics like Intuitive, Responsive, etc. Ultimately, they were too vague to help guide our decisions. Going forward, I would choose principles that are more specific and research-based. Something like: “Users should be able to build a chart without understanding aggregations.”
The emotional impact of video
Throughout the project, I pulled video clips from usability tests to share with my team of engineers and designer. The visual aid not only helped make our discussions more productive, but they encouraged empathy among my team and got them excited about resolving the pain points they observed.