Drinking makes me suicidally depressed, so I stopped many years ago and have witnessed as a result this exact phenomenon. The new neighbor, coworker, potential partner or aquaintence are all super friendly and welcoming until the first time there's an opportunity to get drunk with them, and you don't (no matter how politely or subtly you demure). After that, the relationship fades. I've seen it time and again. It's annoying, but I've accepted it as reality now. I do get frustrated at how it inhibits my career - it's basically impossible to build a strong network without drinking (or at least is to me. Maybe you're some super gregarious teetotaler, but I'm not) and without a strong network you naturally lose opportunities both personally and professionally. If you can drink without issues, I'd highly recommend it - you'll have a happier, more successful life.
Actually as a drinker, I always envied the way smokers got to network in the workplace. I started working at the same time as a smoker. I was generally more extroverted than him, but he ended up knowing way more people in the company than me, because of his cigarette breaks.