It supports the virtual host level only, see https://docs.openlitespeed.org/security/headers/
If you want to reduce the duplicate set up, you could consider the virtual host template or include conf feature(https://docs.openlitespeed.org/config/advanced/includes/).