it happens. there are some people you just cant connect to online. see if your party privacy is set to open and that maximum players allowed in isnt set at zero. if that doesnt help, try joining him. if he cant join you then your not a good host for him. if he has a better connection it will work out better as him host anyway.