It seems more like WHMCS with recpatcha issue to me, but you mentioned it works with Apache, is there any special config or rewrite for recaptcha inside of the .htaccess file?
I still have no clue about this, is there any error shown in the error log while the issue happening?
Maybe you can submit the issue to support@litespeedtech.com with the forum post link appended for a further look.