Ability to tie site locations to user groups

We have a customer who's current site only allowed for apparel orders that all shipped to the same location. Now, they want to add the ability for a single user to order business cards. I have added the card, but now the user also wants to be able to ship to any business location.

It would be nice if I could add all of the locations to the site, but limit the visibility of the other locations to just the group that this user belongs to. In other words, people in the Everyone group would only see a single location, but anybody belonging to the Stationery group would see all of the locations.