You are asking about user research. There is an entire industry of user researchers out there, and lots of great books and articles. It fascinates me that HN has such a blind spot for this topic.

The responses on here are reasonable but a bit short sighted. What works in one context will not work in another.