Is it bad if someone sees your drivers license?

Some drivers license numbers are actually your social security number (some states do this). Only in that case would I be concerned.