AI agents for usability testing – thoughts?

Hey all!

I've been thinking about how AI could potentially handle usability testing. The idea would be AI agents that can actually navigate live websites while thinking out loud, kind of like an unmoderated usability test.

The interesting part is they could theoretically be "recruited" similar to real participants – you'd input your screener questions and demographic preferences, and the AI would form a persona from that (including stuff like mood and environmental factors) before running through the test.

These AI testers would understand typical research prompts like "You're on REI and need hiking boots – find a pair you like and add them to cart" and could do most basic actions (clicking, scrolling, typing, etc) while voicing their thoughts.

Curious what you all think about this direction: 1. This sounds awesome, I'd definitely want to try it out 2. Skeptical but interested if it can actually capture human nuance 3. Not interested even if it works as described (would love to hear why!)

What's your take on this? Could AI testing actually be useful or is it missing something fundamental?