Hi le_florin,
First, it looks like you want a public cache enabled globally. For that, your configuration should work at the server level.
Second, if you want /wp-admin to have a different set-up, you should create a vhost for /wp-admin specifically. Then, in the vhost config, you can...