A knot invariant ordered by filtered finite dimensional vector spaces is called finite type. It has been conjectured that every finite type invariant of classical knots could be extended to a finite type invariant of long virtual knots (Goussarov-Polyak-Viro conjecture). Goussarov, Polyak, and Viro also showed that this conjecture is strongly related to the Vassiliev conjecture that the knots could be classified by Vassiliev invariants. In this paper, for the order-three case of the Goussarov-Polyak-Viro conjecture, we obtain an answer and give a new viewpoint of the conjecture by introducing a reduced Polyak algebra, which is a simple version of the original one, derived from the first Gauss diagram formula that is the linking number for classical knots. This gives a nontrivial example that Gauss diagram formulae of knots of higher degree are systematically obtained by those of lower degree.