DaiMoN: A Decentralized Artificial Intelligence Model Network

Abstract

We introduce DaiMoN, a decentralized artificial intelligence model network, which incentivizes peer collaboration in improving the accuracy of machine learning models for a given classification problem. It is an autonomous network where peers may submit models with improved accuracy and other peers may verify the accuracy improvement. The system maintains an append-only decentralized ledger to keep the log of critical information, including who has trained the model and improved its accuracy, when it has been improved, by how much it has improved, and where to find the newly updated model. DaiMoN rewards these contributing peers with cryptographic tokens. A main feature of DaiMoN is that it allows peers to verify the accuracy improvement of submitted models without knowing the test labels. This is an essential component in order to mitigate intentional model overfitting by model-improving peers. To enable this model accuracy evaluation with hidden test labels, DaiMoN uses a novel learnable Distance Embedding for Labels (DEL) function proposed in this paper. Specific to each test dataset, DEL scrambles the test label vector by embedding it in a low-dimension space while approximately preserving the distance between the dataset's test label vector and a label vector inferred by the classifier. It therefore allows proof-of-improvement (PoI) by peers without providing them access to true test labels. We provide analysis and empirical evidence that under DEL, peers can accurately assess model accuracy. We also argue that it is hard to invert the embedding function and thus, DEL is resilient against attacks aiming to recover test labels in order to cheat. Our prototype implementation of DaiMoN is available at https://github.com/steerapi/daimon.

A Brief Survey of Data Pricing for Machine Learning

“…Mechanism design is a field in economics that deals with setting incentives and interaction rules among self-interested agent samples to achieve desired objects for the group as a whole. In [33], Teerapittayanon et al proposed a deep learning-based approach to automatically design auctions in a wide variety of domains, shifting the design work from human to machine. On the other hand, considering the traditional markets, it is extremely difficult for e-commerce companies to adjust prices when they receive more information from consumers.…”

“…In traditional blockchain [19], proof-of-work (PoW) incentivizes people to participate in the consensus protocol for a reward. Teerapittayanon et al [33] introduced DaiMoN, a decentralized artificial intelligence model Network, which incentivizes peer collaboration in improving the accuracy of machine learning models. It is an autonomous Network where peers may submit models with improved accuracy and other peers may verify the accuracy improvement.…”

Adaptive Ensemble Prediction for Deep Neural Networks based on Confidence Level

“…The basic idea of using the confidence level for the early-exit condition does not depend on this specific order of local predictions. Some existing classifiers with a deep neural network (e.g., [Bolukbasi et al, 2017], [Teerapittayanon et al, 2017], [Huang et al, 2018]) take an early exit approach in ensembling similar to ours or take an early exit from one neural network.…”