If the whole society is not part of the military, then you don't have the backing of that whole society. You have a backing of a fraction of that society. The public recognizes its diverse military has a difficult job and unifies behind it. That moral authority is crucially important to the military's ability to operate.