I had too much time on my hands so I got out my Stone edition Chumash. Which is not the last word in Torah commentary by any means, but apparently some commentators say it doesn't actually mean "sons of G-d" (though that is what it means literally) but sons of powerful people, who took wives from the lower classes who had no ability to say no to them. Others say it has to do with righteous descendants of Seth marrying people who weren't righteous.

However, Torah commentary being what it is, it's a pretty safe bet that there are more... supernatural (hee!) explanations from other commentators.

Also if the lack of angels is disappointing, commentators still seem to agree that the Nephilim were huge giants.

(I have no idea what the implications are for Castiel and pretty girls.)