We construct Graev ultrametrics on free products of groups with two-sided invariant ultrametrics and HNN extensions of such groups. We also introduce a notion of a free product of general Polish groups and prove, in particular, that two Polish groups G and H can be embedded into a Polish group T in such a way that the subgroup of T generated by G and H is isomorphic to the free product G * H.Theorem (Folklore, see Theorem 2.11 in [Kec94]). The group F (N N ) is surjectively universal in the class of Polish groups that admit compatible two-sided invariant metrics. An important question raised in [Kec94] and further advertised in [BK96] is whether there is a universal Polish group. Motivated by this question, L. Ding and S. Gao [DG07b] constructed generalized Graev metrics, and based on this construction Ding [Din12] answered the question of Kechris in the affirmative. In a recent paper Gao [Gao13] addressed the question of the existence of surjectively universal Polish ultrametric groups and gave yet another modification of Graev's original definition. The latter paper of Gao motivates our study of the Graev ultrametrics on free products of ultrametric groups.1.1. Main results. The main results of this work are twofold. In Section 2 we give the constructions of Graev ultrametrics for free products and HNN extensions of groups with two-sided invariant ultrametrics. In particular we prove Theorem (see Theorem 2.13). Let (G, d G ) and (H, d H ) be groups with two-sided invariant ultrametrics, and let A = G ∩ H be a common closed subgroup. There exists a two-sided invariant ultrametric on the free product with amalgamation G * A H that extends ultrametrics d G and d H .Theorem (see Theorem 2.21). Let (G, d ) be a group with a two-sided invariant ultrametric d, A and B be closed subgroups of G and φ : A → B be an isometric isomorphism. If diam(A) ≤ K, then there exists a twosided invariant ultrametric δ on the HNN extension H of (G, φ) which extends d and such that δ(t, e) = K, where t is the stable letter of H.While we follow closely the methods of [Slu12], the formalism for trivial words used in this paper is different. We introduce a new notion of a maximal evaluation forest and argue that it provides a more unified tool for studying Graev metrics on free products than the notion of an evaluation tree.