Testing Methodology

How a device earns its spot.

The criteria, the process, the timeline, and what gets something removed. Every device on this list went through all of it.

Updated April 2026

189+
devices currently in my home
23
categories covered
5+
years building this setup
0
paid placements, ever

What I look for before buying

Before anything makes the shortlist, I run it through a set of criteria that have held up over five years and 100+ devices. The About page has the backstory — this page is about the process itself.

My bar before anything makes the list:

  • Works without an internet connection. Local control is a hard requirement for anything I depend on — locks, lights, thermostats. Cloud-dependent devices get flagged.
  • No mandatory subscription for core functionality. A subscription for premium features is fine. A subscription to unlock what should ship in the box is a red flag.
  • Ecosystem fit. Apple Home is my primary platform. I note anything that works better on Google or Alexa and say so. Matter and Thread get weighted heavily — open standards beat walled gardens when the product is otherwise equal.
  • Long-term manufacturer reliability. I'm wary of companies with a history of killing their cloud services or abandoning older hardware.

What the testing actually looks like

There's no lab here — just a real house. Installation happens the way it would for anyone: with the stock instructions, the same app, the same Wi-Fi network everyone else would use. I don't get pre-configured demo units or pre-loaded accounts.

Minimum time on the list: three months of daily use before a device gets a permanent listing. That's long enough to catch firmware regressions, see how the manufacturer handles a real bug, and find out whether I'm still actually using the thing or if it quietly became a dust collector.

For smart lighting and plugs, I run dozens of automations. For cameras, I check false positive rates over weeks. For locks, I test with every family member, including people who have no idea what HomeKit is. For thermostats and sensors, I track whether the schedule held up through a full heating and cooling season.

What makes something fail

Devices get removed from the list when:

  • I stopped using it — the most honest signal there is
  • The manufacturer pushed an update that broke something and didn't fix it within a reasonable window
  • A new product replaced it and I switched
  • The company changed their business model in a way that affects existing owners
  • I found out the product requires more cloud dependency than I originally understood

When a device comes off the list, its page stays up with a note explaining why. I don't bury mistakes.

How the catalog stays current

Prices, software, and firmware all change. I update listings when something material changes: price drops or spikes, ecosystem support added or removed, app redesigns that affect usability. The "Updated" timestamp on each product page reflects the last time I actively revisited it — not just an automated date stamp.

On affiliate links

Most product links go to Amazon via the Amazon Associates program. If you buy something through one of my links, I earn a small commission at no extra cost to you. That revenue covers hosting and, occasionally, the next thing I add to the catalog.

Affiliate commissions don't influence which products are listed or recommended. I've left off products with higher commission rates because I don't use them or don't like them. I've kept products with lower rates because they're genuinely the best option I've found. The test is always the same: is it earning its place in my home?

See what made the cut

189+ devices across 23 categories — every one bought, tested, and still in use.

Browse the setup About this site