Check out this interactive star map of the 32 nearest stars [32 Nearby Stars].
Imagine that humans could establish colonies on several of these stars in the next 10,000 years. This means that we would have reached out 10 light years in that time. Continuing at that pace, in one million years we would have colonies that are 1000 light years away. In one hundred million years we will have covered more than half the galaxy.
If there are other civilizations like ours, they would have to be less than one hundred million years older than us or they would likely be here by now. Maybe we are alone in the galaxy/universe?
UPDATE: I've been reminded that this argument against the high probability of life is known as the Fermi Paradox.
[Hat Tip: Bad Astronomy]