We study the coupled surface and grain boundary motion in bi-and tricrystals in three space dimensions, building on previous work by the authors on the simplified two dimensional case. The motion of the interfaces, which in this paper are presented by two-dimensional hypersurfaces, is described by two types of normal velocities: motion by mean curvature and motion by surface diffusion. Three hypersurfaces meet at triple junction lines, where junction conditions need to hold. Similarly, boundary conditions are prescribed where an interface meets an external boundary, and these conditions naturally give rise to contact angles. We present a variational formulation of the flows, which leads to a fully practical finite element approximation that exhibits excellent mesh properties, with no mesh smoothing or remeshing required in practice. For the introduced parametric finite element approximation we show well-posedness and, in general, unconditional stability, i.e. there is no restriction on the chosen time step size. Moreover, the induced discrete equations are linear and easy to solve. A generalization to anisotropic surface energies is straightforward. Several numerical results in two and three space dimensions are presented, including simulations for thermal grooving and sintering. Three dimensional simulations featuring quadruple junction points, nonstandard boundary contact angles and fully anisotropic surface energies are also presented.