Applications and Potential of Zero-Knowledge Machine Learning (ZKML)

Author: Callum@Web3CN.Pro

ZK has continued to be hot since 2022, and its technology has made great progress, and the projects of the ZK series have also continued to make efforts. At the same time, with the popularization of Machine Learning (ML) and its widespread application in production and life, many companies have begun to build, train, and deploy machine learning models. But a major problem currently facing machine learning is how to ensure trustworthiness and dependence on opaque data. This is the significance of **ZKML: to let people who use machine learning fully understand the model without revealing the information of the model itself. **

1. What is ZKML

What is ZKML, let's look at it separately. ZK (Zero-Knowledge Proof) is a cryptographic protocol where the prover can prove to the verifier that a given statement is true without revealing any other information, that is, the result can be known without the process.

**ZK has two major characteristics: first, it proves what it wants to prove without revealing too much information to the verifier; second, it is difficult to generate a proof, and it is easy to verify the proof. **

Based on these two characteristics, **ZK has developed several major use cases: Layer 2 expansion, private public chain, decentralized storage, identity verification, and machine learning. **The research focus of this article will focus on ZKML (Zero-Knowledge Machine Learning).

What is ML (Machine Learning), Machine Learning is a science of artificial intelligence that involves the development and application of algorithms that enable computers to learn and adapt to data autonomously, optimizing their performance through an iterative process without the need for a programming process. It uses algorithms and models to identify data to get model parameters and finally make predictions/decisions.

**At present, machine learning has been successfully applied in various fields. With the improvement of these models, machine learning needs to perform more and more tasks. In order to ensure a high-accuracy model, ZK technology is required: using public Model validates private data or validates private model with public data. **

The ZKML we're talking about so far is creating zero-knowledge proofs of the inference steps of ML models, not ML model training.

2. Why ZKML is needed

As artificial intelligence technology advances, it becomes more difficult to distinguish between artificial intelligence and human intelligence and human generation. Zero-knowledge proofs have the ability to solve this problem. It allows us to determine whether a certain content is generated by applying a certain model generated without revealing any other information about the model or the input.

Traditional machine learning platforms often require developers to submit their model architectures to the host for performance verification. This can cause several problems:

• Intellectual Property Loss: Disclosing the full model architecture may expose valuable trade secrets or innovations that developers wish to keep private.
• ** Lack of transparency: ** The evaluation process may not be transparent and participants may not be able to verify how their model ranks against other models.
• Data Privacy Concerns: Shared models trained on sensitive data may inadvertently reveal information about the underlying data, violating privacy norms and regulations.

These challenges have created a need for solutions that can protect the privacy of machine learning models and their training data.

ZK proposes a promising approach to address the challenges faced by traditional ML platforms. By leveraging the power of ZK, ZKML provides a privacy-preserving solution with the following advantages:

• Model Privacy: Developers can participate in verification without disclosing the entire model architecture, thereby protecting their intellectual property.
• Transparent Validation: ZK can verify model performance without revealing model internals, thereby facilitating a transparent and trustless evaluation process.
• **Data Privacy: **ZK can be used to verify private data using public models or verify private models using public data to ensure that sensitive information is not leaked.

Integrating ZK into the ML process provides a safe and privacy-preserving platform that addresses the limitations of traditional ML. This not only promotes the adoption of machine learning in the privacy industry, but also attracts experienced Web2 developers to explore the possibilities within the Web3 ecosystem.

3. ZKML Applications and Opportunities

With the increasing improvement of cryptography, zero-knowledge proof technology and hardware facilities, more and more projects have begun to explore the use of ZKML. The ecosystem of ZKML can be roughly divided into the following four categories:

• **Model Verification Compiler: ** An infrastructure for compiling models from existing formats (eg Pytorch, ONNX, etc.) into verifiable computational circuits.
• Generalized Proof System: A proof system built to verify arbitrary computational trajectories.
• **ZKML-specific proof system: **A proof system specially built to verify the computational trajectory of ML models.
• Applications: Projects dealing with ZKML use cases.

According to the ecological category of these ZKML applications, we can classify some current ZKML-applied projects:

Image credit: @bastian_wetzel

ZKML is still an emerging technology, its market is still very early, and many applications are only experimented at hackathons, but ZKML still opens up a new design space for smart contracts:

DeFi

** Defi applications parameterized using ML can be more automated. ** For example, lending protocols can use ML models to update parameters in real time. Currently, lending protocols primarily trust off-chain models run by organizations to determine collateral, LTV, liquidation thresholds, etc., but a better alternative might be a community-trained open-source model that anyone can run and verify. Using a verifiable off-chain ML oracle, ML models can process signed data off-chain for prediction and classification. These off-chain ML oracles can trustlessly solve real-world prediction markets, lending protocols, etc. by verifying reasoning and publishing proofs on-chain.

Web3 Social

** Filter Web3 social media. **The decentralized nature of Web3 social applications will lead to more spam and malicious content. Ideally, social media platforms could use community-agreed open-source ML models and publish proofs of model reasoning when they choose to filter posts. As a social media user, you may be willing to view personalized advertisements, but wish to keep your user's preferences and interests private from advertisers. So users can choose to run a model locally if they prefer, which can be fed into media applications to provide content for them.

GameFi

**ZKML can be applied to new types of on-chain games, and can create cooperative human and artificial intelligence games and other innovative on-chain games. **In which the artificial intelligence model can act as an NPC, and every action taken by the NPC will be published on the chain. Accompanied by a proof that anyone can verify to determine the correct model is running. At the same time, ML models can be used to dynamically adjust token issuance, supply, burning, voting thresholds, etc. An incentive contract model can be designed that will rebalance the in-game economy if a certain rebalancing threshold is reached and the proof of reasoning verified.

Authentication

** Replace private keys with privacy-preserving biometric authentication. **Private key management remains one of the biggest pain points in Web3. Extracting private keys via facial recognition or other unique factors might be a possible solution for ZKML.

4. ZKML Challenge

Although ZKML is constantly being improved and optimized, the field is still in the early stages of development, and there are still some challenges from technology to practice:

• Quantization with minimal loss of precision
• the size of the circuit, especially when a network consists of multiple layers
• Efficient proof of matrix multiplication
• Adversarial attack

**These challenges firstly affect the accuracy of the machine learning model, secondly affect its cost and proof speed, and thirdly affect the risk of model theft attacks. **

Improvements to these problems are currently underway, @0xPARC's ZK-MNIST demo in 2021 showed how to implement a small-scale MNIST image classification model in a verifiable circuit; Daniel Kang did the same for the ImageNet scale model, currently ImageNet scale The accuracy of the model has been improved to 92% and is expected to be reached soon with further hardware acceleration of the wider ML space.

ZKML is still in the early development stage, but it has begun to show a lot of results, and we can expect to see more innovative applications of ZKML on the chain. As ZKML continues to develop, we can foresee a future where privacy-preserving machine learning will become the norm.