game crashes when extremely high ping enters lobby

I host a lot of games, and whenever I have someone come into the lobby with a 1000+ ping, the chances of a crash become almost 100%. I've noticed something about it as well, usually one of the cores of my CPU is maxxed out in terms of processor resources, the other's will be down at normal values, but the single core will be maxxed out. So my Ethernet is using so much of a resource that the comp freezes, and can't defer the process to another core for whatever reason. So I have to ctrl-alt-del->task manager->end process. Fix? Workaround?