Write what you know? Really?

I was talking to a friend today, an aspiring writer (aren't we all), and he quoted the hoary old adage "write what you know". Teachers trot this out to their students without really expanding upon it.  

If we all only wrote what we know, the world of literature would be a much poorer place. There wouldn't be any sci-fi, no fantasy, a limited range of extreme pornography and so on. 

But we can use what we know, to fill in the blanks. After all that's all any of do. Fit the sound of a scraping chair into a new environment, or move the sound of silence into space or a desert. 

We can't experience everything, so add what you know into your imagined spaces.