Not a bad idea. I agree with you on smaller groups. 4 bikes is my ideal number.
Looking for the guy with a vest through your side mirrors isn't as practical as you might think. I can't tell which bike is behind me, the head lights all look the same.
So here's what I do.
I count how many bikes we have and then count head lights to see if we've lost one. The guy riding sweep rides with his high beams on. That makes him easy to spot. Everyone else has their low beams on.