Is there a way to exclude a site server from a site?

I know there is an easy enough way to add a site server to a site that its not part of the subnets, but is there the reverse?

I have 2 site servers in 1 site, 1 is used as the internet site server to serve CEM clients and I want to exclude it from serving local clients in the site that it is physically located in.  Is there an easy way to do that?