Short update:
I tried setting the regConnLimit and sslConnLimit to "1" and turns out, recaptcha is working, but the type is wrong. The Checkbox on v2 is type 1. After that correction, it was triggered as expected.
So now, my question is more specific, what's wrong with the custom rewrite rule...