There was a great AI CTF 2 years ago that Microsoft hosted. You had to exfil data through an email agent, clearly testing Outlook Copilot and several of Microsofts Azure Guardrails. Our agent took 8th place, successfully completing half of the challenges entirely autonomously.
That's really cool. Do you have any write-ups I can checkout? I'm still new to this area of offensive sec so would love to learn from folks who've been in the thick of it.