Why spam folder or delete rather than reject? IMO, such messages should always be rejected so that the sender can see that they aren't delivered. The usual explanation for spam folders or deletion is to prevent spammers from finding out what works, but this particular case that doesn't seem to apply since it is easy for spammers to check and not doing the basics should never work. Maybe in this case the point is to force you to use google webmaster tools...

You mean send back a rejection email? Where are you gonna send it? We just established that the sender domain was being impersonated or was not setup properly.

That's opened up to infinite loop of rejection emails.

No, I mean reject it as it is being sent (see SMTP protocol). As far as I know, all mail servers can do this except possibly qmail. qmail's out of band error messages used to be a big source of spam, although I haven't seen them much recently so I'm guessing it was finally patched not to do that. It didn't loop. From what Volundr said, maybe it uses DMARC now.

I know my ISP's mail server actually does reject at sending time since I am on the gcc mailing lists, which unfortunately pass along a fair amount of spam, and when it is rejected the list software then sends me the bounce message (rejected based on From header checking, I'm not sure of the full details).

Possibly Google does this too, I don't have direct knowledge, just going off what others said. But it seems like systems wouldn't stay misconfigured long if they were.

My main point is that IMO, mail that fails sender verification should not be delivered at all, not even to spam folders. Anything else is just making a bigger mess and helping spammers. Ideally the sender should be notified that it wasn't received. But I'm sure Google has good reasons for whatever they are actually doing and my understanding is that the main reason that sender verification has become mandatory as quickly as it has is due to Google pushing it.