Silicon Valley is a place located just south of San Francisco, California. Even though I feel it should include San Francisco (I know lots of programmers who live there). The reason why it has been named Silicon Valley is because silica is used to make computer chips (to make metals non corrosive) and the region is the most recognized to be the birthplace of computer technology.
While it is known as such it is still the place where large amounts of programmers stem from and/or live/work.
So to make it simple: Silicon = Computer chips + Northern California + Computer nerds = Silicon Valley