Dialogue-based relation extraction (DiaRE)aims to detect the structural information from unstructured utterances in dialogues. Existing relation extraction models may be unsatisfactory under such a conversational setting, due to the entangled logic and information sparsity issues in utterances involving multiple speakers. To this end, we introduce SOLS, a novel model which can explicitly induce speaker-oriented latent structures for better DiaRE. Specifically, we learn latent structures to capture the relationships among tokens beyond the utterance boundaries, alleviating the entangled logic issue. During the learning process, our speakerspecific regularization method progressively highlights speaker-related key clues and erases the irrelevant ones, alleviating the information sparsity issue. Experiments on three public datasets demonstrate the effectiveness of our proposed approach. * * Equal contribution, work done during Guoqing Luo and Yao Xiao's internships at SUTD.Accepted as a long paper in the main conference of EMNLP 2021 (Conference on Empirical Methods in Natural Language Processing). S1: Jack. Could you come in here for a moment? Now! S2: Found it. S3: I'll take that dad. S1: It seems your daughter and Richard are something of an item. S2: That's impossible, he's got a twinkie in the city. S4: Dad, I'm the twinkie. S2: You're the twinkie? S3: Yes, that is impossible S5: She's not a twinkie.S2: Am I supposed to stand here and listen to this on my birthday? S4: Dad, dad this is a good thing for me. Ya know, and you even said yourself, you've never seen Richard happier.