Block based on cookies (i.e., set a cookie on the browser and check on the serve... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		kjok 74 days ago \| parent \| context \| favorite \| on: Meta's crawler made 11M requests to my site in 30 ... Block based on cookies (i.e., set a cookie on the browser and check on the server whether it exists).

direwolf20 74 days ago | [–]

This project implements a variety of similar JSless checks, such as image loading

https://github.com/WeebDataHoarder/go-away

bigbadfeline 73 days ago | [–]

That helps, but the big bot farms use clients that support cookies, we need to add more defenses on top of them.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact