If you want to remove the virtual host manually, yes, you can delete the folder, remove the virtual host by clicking the trash, and remove the domain mapping from the listener.
You can also use the script to delete the virtual host, and remove the document root folder manually.