Online multiplayer breaks when entering goat castle door (Xbox)

Hello, I’ve beaten the game several times and the first time I was playing I was playing couch co-op with two others and it worked perfectly with no bugs, but later when I was playing online we opened the door and it kicked me (I was not party owner) out of the game. This happens no matter who I’m playing with online and it has happened to other people not playing with me. A fix would be amazing.