MQTT Cloud Connector
Launching InfluxData’s first cloud connection.
The Team
Amy, Product manager
Me, Product designer
Kristina, Engineering manager
along with a team of 4 engineers
3 month project in 2022
What is MQTT?
MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol that is used to send and receive messages over a network. It is often used in Internet of Things (IoT) applications, where devices need to send and receive data with minimal overhead.
A client connects to the broker and sends the message along with a topic. The broker then receives the message and stores it until a subscriber is ready to receive it.
InfluxData’s MQTT connection communicates with an MQTT broker to receive data on a specified topic and stores it in an InfluxDB time series database.
Getting started
When I joined the MQTT product team, the basic connection form was in place and mostly working. I was tasked with refining the user experience in preparation for the promotional launch in 2 months’ time.
For the launch, I focused on answering one specific research question:
Can users successfully create an MQTT connection, and if not, why?
Heuristic Review
Typographic Hierarchy
The text styles used for the header sections did not follow our usual patterns, and did not make it clear how the sections were organized/grouped.
Error Handling
I identified several cases where the user would be unable to submit the form but did not receive feedback as to the cause.
Free Account User Experience
Only paid accounts could use the MQTT connector, but free accounts needed to have a good experience too if we hoped to encourage an upgrade. As it was, the free account experience was confusing and didn’t provide a clear path to upgrade.
Education
The connection form required some knowledge that I wasn’t yet convinced all users would have. For example, understanding JSON parsing, regex wildcard syntax, as well as Influx-specific terms like measurement, tag, and field.
After learning the basics of MQTT and setting up a connection myself, my first task towards understanding the current state of the connection form was to perform a heuristic review. I identified several issues, some that could be addressed quickly and some that would need further investigation.
Findings
Lack of troubleshooting ability / error feedback
Both troubleshooting connection failures as well as feedback when there are errors completing the connection form.
Users need flexibility with data formatting
The MQTT connector supported one measurement (metric) per one topic per connection. Several users indicated that they would expect to have multiple measurements per connection.
There also seems to be a major use case that’s currently unsupported, which is defining tags and defaulting the remaining items to fields. This saves users a lot of effort if they have many fields, and is flexible if the number of fields changes.
Form usability issues
For example:
a field appearing to be filled out when it isn’t
a label whose meaning is unclear
lack of clarity as to whether specific form fields are optional
lack of clarity around what format is required for specific text inputs
User Interviews
I spoke with several MQTT users to understand their use case, feature needs, and their experience with the connection form.
“Everyone uses trial and error. You give it your best shot, then you look at the logs and try to get to the bottom of the error tree.”
Triaging updates
Out of the heuristic review and user interviews I produced a wishlist of updates. To make the list actionable, I divided these items into “Quick wins” and “Needs work”.
The PM and I worked together to decide which to prioritize, with a primary focus on which quick wins would give us the biggest quality improvements for the fast-approaching promotional launch.
We knew that the promotional launch could bring in new users, so we focused particularly closely on updates to enhance the new user experience.
Form UX issues
Baseline UX improvements to remove stumbling blocks in the connection form design, like rewording a label to be clearer, providing clear form error feedback, etc.
Improve free account experience
For a free account, the initial UI allowed them to fill out the connection form but not submit it. We knew this would be a frustrating experience, so I updated the flow to allow users to preview the form along with a CTA to upgrade.
Education
Particularly for users who are new to InfluxData’s platform, we’d need to provide some help around Influx-specific and other technical terminology. If we want to get them to convert, we needed to educate them on the platform and connector requirements.
If their JSON parsing rules were incorrect, for example, the parsing would silently fail and you wouldn’t realize until you retrieved the data and noticed some of it missing.
Free Account Before (left) and After
Make it clear that an upgrade is required to use MQTT, and provide a clear path to do so.
Help resources
Help text added for both JSON parsing and Measurement/Field.
Measuring success
Our primary metric for measuring success is: Are people using MQTT and are they regularly ingesting data?
Within a month of launch, we hit 100 unique subscriptions across over 30 paying accounts. In some regions, MQTT made up a little over 1% of all data ingestion.
Learnings
Data structures are varied and unique
As a database tool, it’s so so important to talk to our customers and deeply understand how they are storing data. The kinds of data they store, the way they organize it, and the things they want to do with it can vary quite a lot.
Start user research early
Because the team didn’t talk to very many users early in the process, we didn’t realize that lack of certificate support was a blocker for nearly all of the customers who had requested MQTT. We didn’t delay the launch, but it meant we couldn’t get as much usage data until several weeks later when we had certificate support.
Users expect feature consistency
The customers who had used our platform previously were surprised that MQTT didn’t have the same kind of error logging that Telegraf (another kind of connector) provided. While the PMs were unsure those logs were the best user experience, it’s something we should have given users because it’s what they were familiar with.
Next steps
Development on the MQTT connection was paused to focus on other projects. If I had dedicated time to continue with the project, here are the things I would focus on:
Gather feature requests
Develop a feedback loop with sales and support for MQTT feature requests.
Troubleshooting
I’d like to dig in to logs as a project to investigate how we can provide users more visibility into connection issues. Building on top of that, Influx should be monitoring the types of connection errors and preventing them whenever possible.
Investigate parsing improvements
For users, creating parsing rules is tedious and error-prone. Are there ways we could automate it or make the process easier? At a minimum, we should provide visual confirmation as to whether the data was parsed as expected.