a16z leads $43 million investment in Gensyn, an AGI computing power market, to understand Gensyn

On June 12, 2023, Gensyn, a blockchain-based AGI computing power market protocol, announced the completion of a $43 million Series A round of financing, led by a16z, with participation from Eden Block, CoinFund, Galaxy, and Protocol Labs.

What kind of project is Gensyn? Why can you get huge investment from top VC? Golden Finance will take you to understand it in one article.

a16z: Why Leading Gensyn’s $43M Series A Funding

a16z posted an article explaining why it led Gensyn’s $43 million Series A round of financing. a16z said that the recent progress of artificial intelligence is incredible and has the power to save the world (see Jinse Finance's previous report "a16z founder's long article: Why AI will save the world"). But building AI systems requires deploying far greater computing power to train and reason about the largest, most powerful models available today. That means big tech companies have an edge over startups in the race to extract value from AI, thanks to privileged access to computing power and the economies of scale of large data centers. To compete on a level playing field, startups need to be able to affordably use their own massive computing power.

Blockchains are unique as a new type of computer in that developers can write code that makes firm promises about how the code will behave in the future. This permissionless component of the blockchain could create a marketplace for buyers and sellers of computing power — or any other type of digital resource such as data or algorithms — to trade globally without intermediaries.

**Gensyn, a blockchain-based AGI computing market protocol, connects developers (anyone who can train machine learning models) with solvers (Solver, anyone who wants to use their own machines to train machine learning models) stand up. Gensyn can increase the available computing power for machine learning by 10-100x by leveraging the long tail of machine learning capable computing devices left unused around the world such as small data centers, gaming PCs, M1 and M2 Macs, and even smartphones. **

Problems facing AGI (General Artificial Intelligence): Highly Centralized

After nearly half a year of development, the market generally recognizes that AGI is the future. But the AGI industry currently looks highly monopolized**, between countries is a trade and talent war between China and the United States, and between companies is the game of large technology companies (Microsoft, Google, Meta). **Because the three key resources of AGI (computing power, knowledge and data) are currently highly centralized. **

Computing power: Increasingly large and complex models require high-computing processors for training. Between countries: The chip war between China and the United States, the United States has been actively preventing China from obtaining high-power chips. Between companies: Insufficient production capacity, Nvidia's latest AI chips are all purchased by some big customers, and other companies cannot buy them at all. On the tech stack: Some companies even create their own dedicated hardware for deep learning, such as Google's TPU clusters. These outperform standard GPUs for deep learning and are not for sale, only for rental.

Knowledge: Many public breakthroughs stem from new large-scale model architectures developed by researchers, but there is a battle over the underlying intellectual property and talent. For example, the United States has attracted more than 50% of China's AI talents, and large companies that use these talents to develop large-scale models are increasingly reducing the accessibility of this technology; OpenAI's GPT-3.5 or 4 is nominally publicly available , but it sits behind an API and only Microsoft has access to its source code.

Data: AGI deep learning models require large amounts of data—both labeled and unlabeled—and typically improve with more data. GPT-3 was trained on 300 billion words. Labeled data is particularly important, and the data sets needed to train AGI are concentrated in the hands of some large companies. For example, a little bit of good knowledge: every time you visit a website solving reCaptcha, you are labeling training data to improve Google Maps.

Difficulties in decentralized AGI computing

Decentralized computing can create a cheaper and freer basis to research and develop artificial intelligence. But there is a work verification problem in decentralized AGI. How do you know that the third party has completed the calculation you requested?

The proof-of-work puzzle has two factors: state dependence, and high computational cost.

State Dependency: Each layer in a neural network is connected to all nodes in the layer before it. This means it needs the state of the previous layer. To make matters worse, all the weights of each layer are determined by the previous time step. So if you want to verify that someone trained a model -- say, by picking a random point in the network and seeing if you get the same state -- you need to keep training the model until that point, which is computationally expensive.

High Computational Expenses: The cost of a single training session of GPT-3 in 2020 is about $12 million, which is more than 270 times higher than the estimated value of about $43,000 for GPT-2 training in 2019. In general, the model complexity (size) of the best neural networks currently doubles every three months. Validation overhead, possibly from state dependencies, is acceptable if the neural network is cheaper, and/or if training represents less of a model development process.

If you want to make deep learning training cheap and decentralize control, you need a system that trustlessly manages state-related validation while being cheap in terms of overhead and rewarding those who contribute computation.

How Gensyn Decentralizes AGI Computing

The Gensyn protocol unites all the computing in the world into a global machine learning supercluster that is readily available to anyone. It enables trustless training of neural networks at very large scale and at low cost by combining two things:

1. Innovative verification system

A verification system that efficiently solves the state-dependence problem in neural network training of arbitrary scale. The system combines model training checkpoints with probabilistic checks that terminate on-chain. It does all of this in a trustless manner with an overhead that scales linearly with model size (keeping validation costs constant).

According to Gensyn Litepaper, Gensyn mainly solves the verification problem through three concepts: probabilistic proof-of-learning (using metadata in the gradient-based optimization process to build a certificate of the work performed and quickly verify it through the replication of certain stages), Graph-based pinpointing protocol (uses multi-grained, graph-based pinpointing protocol and cross-evaluator consensus execution to allow validation work to be re-run and compared for consistency, and ultimately confirmed by the chain itself), Truebit-style incentivized games (using staking and slashing to build an incentive game that ensures each financially rational player acts honestly and performs their intended tasks)

**The system mainly consists of four main participants: submitters, solvers, verifiers and whistleblowers. **Submitter: the end user of the system, who provides the task to be calculated and pays for the completed unit of work; solver: the main working part of the system, performs model training and generates proofs for verification by the verifier; verifier: the non-deterministic The training process is linked to a deterministic linear computation, replicating part of the solver's proof and comparing the distance to an expected threshold; the whistleblower: the last line of defense, checking the work of the verifier and challenging for a jackpot.

2. New supply

Take advantage of underutilized and underutilized/underoptimized computing device resources. These devices range from currently unused gaming GPUs to GPU miners from the pre-Ethereum PoW era. And the protocol's decentralization means that it will ultimately be governed by a community majority and cannot be "shut down" without community consent; unlike its web2 counterpart, this makes it censorship-resistant.

Scale + low cost: Gensyn protocol offers a similar cost to data center owned GPUs, which can scale beyond AWS