"Fermi realized that any civilization with a modest amount of rocket technology and an immodest amount of imperial incentive could rapidly colonize the entire Galaxy. Within ten million years, every star system could be brought under the wing of empire. Ten million years may sound long, but in fact it's quite short compared with the age of the Galaxy, which is roughly ten thousand million years. Colonization of the Milky Way should be a quick exercise.
So what Fermi immediately realized was that the aliens have had more than enough time to pepper the Galaxy with their presence. But looking around, he didn't see any clear indication that they're out and about. This prompted Fermi to ask what was (to him) an obvious question: "where is everybody?" -seti.org
See this short article and video: