Improving IoT Inspector (Part 1)

I’m Danny Y. Huang, an Assistant Professor at the Center for Cyber Security at New York University Tandon School of Engineering. Recently, I was awarded the Consumer Reports Digital Labs Fellowship.

I’m the main developer of IoT Inspector, an open-source desktop app that allows users to identify potential security and privacy vulnerabilities in their smart homes. My team and I launched a prototype of this desktop app in April 2019. Since then, more than 5,500 users from around the world have managed to run the tool and inspect more than 55,000 devices, at the time of writing. However, user churn remains a problem.

Together with CR, I will co-create a strategy to improve the usability of IoT Inspector, scale up the user-contributor community, and attract more users and researchers toward this open data science. In this post, we describe our next steps to make IoT Inspector better.

Problem 1: How to communicate security risks without spooking users?

Problem. It is an open challenge to communicate security risks and non-risks to everyday users. Here is a real example. One of the users of IoT Inspector emailed me to ask: They have a Belkin Wemo Smart Plug, and it seems to be communicating with a navy.mil — a military domain. Has the military hacked into their network?

It turns out that indeed the smart plug, at the time of writing, was communicating with the said domain over Port 123, which is the Network Time Protocol (NTP). In other words, the smart plug was asking the server what time it was. The server happened to be hosted by the military. As with Network Time Protocol (or NTP pools), the time server could have been hosted by anyone in the world. Although it is true that the smart plug was communicating with the server that the military owns, it is unlikely that the military had hacked into their network.

Possible solutions. So in retrospect how should IoT Inspect have told the user about this communication? Options include:

- Showing the raw information (as above)
- Indicating that the smart plug was communicating over NTP with the military
- Indicating that the plug was asking the military for the time
- Indicating that the plug was asking for the time
- Indicating that the plug was behaving normally

Next steps. We don’t really know what’s the best option, but we’ll find out through A/B testing. In our next internal release, we will ask users to voluntarily provide us with demographic information, along with their technical know-how. We will then randomly show each of the options above and ask for their opinions.

Our hypothesis is that certain demographic groups would prefer certain options. In our actual deployment, we can potentially infer these groups based on the number of IoT devices in the network, along with the types of devices. For example, our guess is that a high-level of technical expertise is correlated with a large number of IoT devices, and/or the presence of certain devices (e.g., Ubiquiti and Raspberry Pi) that allow for customization. In short, we don’t know what the best options are, and we will find out from experiments.