The Language of Innovation

Abstract

Predicting innovation is a peculiar problem in data science. Following its definition, an innovation is always a never-seen-before event, leaving no room for traditional supervised learning approaches. Here we propose a strategy to address the problem in the context of innovative patents, by defining innovations as never-seen-before associations of technologies and exploiting self-supervised learning techniques. We think of technological codes present in patents as a vocabulary and the whole technological corpus as written in a specific, evolving language. We leverage such structure with techniques borrowed from Natural Language Processing by embedding technologies in a high dimensional euclidean space where relative positions are representative of learned semantics. Proximity in this space is an effective predictor of specific innovation events, that outperforms a wide range of standard link-prediction metrics. The success of patented innovations follows a complex dynamics characterized by different patterns which we analyze in details with specific examples. The methods proposed in this paper provide a completely new way of understanding and forecasting innovation, by tackling it from a revealing perspective and opening interesting scenarios for a number of applications and further analytic approaches.

A Context Similarity-Based Analysis of Countries’ Technological Performance

“…The dynamic in the space of technologies provides enough information to successfully forecast radical innovations (see [ 17 , 18 ]). Here, we took this a step further and addressed the problem of predicting which country is more likely to introduce them.…”

“…On each training set, we trained 30 different copies of the same neural network, and we define the context similarity of codes and as the scalar product averaged over 30 runs. For more details, see [ 17 , 18 , 23 , 24 ].…”

“…In this section, we discuss how to estimate the capability of a country to patent a potential innovation through context similarity . In principle, context similarity ranges between , but we have empirically observed a distribution of values in the range , with 0 meaning that the two codes are as different as possible, and 1 meaning that they practically can be considered synonyms [ 17 , 18 ]. Recall that the higher the context similarity , the higher the probability of the potential innovation to be patented.…”

“…In particular, a radical innovation is a true never-seen-before event in the moment when it happens, a novelty is an innovation only in a restricted context, e.g., the discovery of a new song written 5 years ago is an innovation only for the person who hears it for the first time now, but not a true innovation because it already exists. In a recent work, we introduced an effective method to predict forthcoming radical innovations based on two simple assumptions (see [ 17 , 18 ]). First, to predict radical innovations we start by defining a framework, which, in our case, is the technological progress documented by the PATSTAT database [ 19 ] through technological codes [ 20 ].…”

“…In such a space, a similarity metric between codes, which we refer to as context similarity , quantifies the contextual distance between codes, see Section Methods for a brief introduction on the subject. We have already shown in detail that looking at the dynamics in this space allows for the prediction of radical innovations with high accuracy (see [ 17 , 18 ]). In this work, we take this a step further and discuss how to predict where a radical innovation is more likely to be patented.…”