normally as a reverse proxy, OLS will only proxy the traffic to the backend and it should not impact much on static file loading.
maybe you have setup some browser cache on Nginx(hence accesssing it directly will be quicker) but did not set it up on OLS?
By the way, any benefit to setup OLS + Nginx instead of just OLS itself?