In this paper, we study algorithmic problems for automaton semigroups and automaton groups related to freeness and finiteness. In the course of this study, we also exhibit some connections between the algebraic structure of automaton (semi)groups and their dynamics on the boundary.First, we show that it is undecidable to check whether the group generated by a given invertible automaton has a positive relation, i. e. a relation * The first author was supported by the Austrian Science Fund projects FWF P24028-N18 and FWF P29355-N35. † The second author thanks the project INDAM-GNSAGA.
IntroductionAutomaton groups, i. e. groups generated by synchronous Mealy automata, are a quite intriguing class of groups. They have deep connections with many areas of mathematics, from the theory of profinite groups to complex dynamics and theoretical computer science, and they serve as a source of examples or counterexamples for many important group theoretic problems (see e. g. [40] for an introduction). Despite these connections and the many surprising and interesting consequences, knowledge about the class of automaton groups from the algebraic, algorithmic and dynamical perspective is still rather limited. From the algorithmic point of view, the word problem for automaton groups is decidable 1 while many other problems are suspected (and sometimes proven) to be undecidable. In this regard, the most studied problems in the literature are the finiteness problem, the freeness problem and the conjugacy problem. The latter has been proven to be undecidable by Šunić and Ventura [38]. However, if this problem is restricted to the contracting case, then it turns out to be decidable, see [5]. Decidability of the finiteness problem for automaton groups is still open. However, some partial results are known when certain properties of the generating automaton are relaxed: Belk and Bleak showed undecidability of the finiteness problem for groups generated by asynchronous automata [3] and Gillibert showed undecidability of the finiteness problem for automaton semigroups [20], which are generated by synchronous but not necessarily invertible Mealy automata. Other results worth mentioning in this respect are the recent proof by Gillibert showing undecidability of the order problem of automaton groups [21] and the result by Bartholdi and Mitrofanov that the order problem is already undecidable for contracting automaton groups [2]. On the other side, Klimann showed that the finiteness problem is solvable for reversible, invertible automata with two states or two letters and that so is the freeness problem for semigroups generated by two state invertible, reversible automata [28]. Obviously, it is usually easier to show undecidability results for automaton semigroups than it is for automaton groups. This might be one of the reasons why the less studied class of automaton semigroups seems to stir up more interest lately. There is, for example, the semigroup theoretic work of Cain [9], and Brough and Cain [6,7], approaches to semigroups via duals ...