This is, in fact, the major reason capital gains are taxed at a lower rate. Taxes on earned income are higher than those on investment income. If you are someone who earned income (taxed) and then invested it (taxed again) you are getting double taxed. That's why so many people want the capital gains rate to go to zero.
On the other hand, one can argue that some people inherit their wealth and never work, so they never work (no tax) and all they have is investment income (taxed at a low rate). It's this type of thing that creates resentment and the desire to "tax the rich."