Next time it happens can someone do a Traceroute and tell me the IP address where it fails?

This will confirm which step in the chain is causing the problem. It is likely to be ISP / backbone specific.