I think people think I'm talking about "how well people can learn new things" when I talk about this. That's potentially a part of it, but it's not what I interviewed for. I interviewed for, literally, simply, "ability to do the job we had Matasano consultants perform".
Microcorruption and the Crypto Challenges were outreach for Matasano, but they were not part of the hiring process. Our hiring challenges were slightly more boring, and explicitly tuned to generate the exact signal we wanted.
I don't know whether there's any valid signal to recover from candidate psychology. I don't have to reach that argument, because in my universe, there's a giant, very accurate signal available that makes looking at other signals a poor use of my time. The fact that the big signal also keeps me from using dubious signals and making bad decisions is just a knock-on benefit. :)
That's great that you had a very specific job req and set of things you were looking for. Not all of us are that lucky ;)
Startups sometimes look for generalists (i.e. people who can learn new things) specifically because nobody knows what the stack will look like, how the product will change, etc. a year from now.
Microcorruption and the Crypto Challenges were outreach for Matasano, but they were not part of the hiring process. Our hiring challenges were slightly more boring, and explicitly tuned to generate the exact signal we wanted.
I don't know whether there's any valid signal to recover from candidate psychology. I don't have to reach that argument, because in my universe, there's a giant, very accurate signal available that makes looking at other signals a poor use of my time. The fact that the big signal also keeps me from using dubious signals and making bad decisions is just a knock-on benefit. :)