Thesis: Digital abstraction simplifies coordination by dramatically reducing the complexity of verifiability and enforcement. It is primarily the progress of computation, not game theory, that fuels modern progress in coordination. Since AI happens to be digitally abstracted, it will enable a further revolution in coordination technology.
Suppose we want to cooperate in order to avoid some kind of prisoner’s-dilemma-esque tragedy of the commons. A normal pattern here is to agree to cooperate optimistically, and also punish anyone who doesn’t cooperate. That is, we take a game which does not have a strong Nash equilibrium for cooperation, and introduce an exogenous structure which cements one.
A problem with such mechanisms is that of information asymmetry: I might be able to defect without anyone else noticing, and thereby avoid the punishment. In the general case, protecting against information asymmetries is costly. Each separate cooperation rule requires resources for detecting and proving defections.
Consider a government official who has been charged with avoiding conflicts of interest in making an acquisition decision. Or an admissions committee responsible for not making use of certain forms of information in making admissions decisions. Or a doctor who is expected to adhere to certain standards of diligence in performing a diagnosis.
Each of these instances represent a coordination problem with opportunities for defection. Policing against defections in these situations is difficult because the action spaces, information channels, and flavors of strategic behavior in each case are all essentially unboundedly large. Enforcement is an open-ended problem that is extremely difficult and costly. While institutional agency exist which keep defection in check, they are never completely effective.
The power of digital abstraction is essentially to take a cooperation problem such as one of the above, and embed it into a digital sandbox which provides an extreme level of environmental insulation. The outcome of an operations in a digital abstraction layer can be completely traced back to a finite collection of known inputs and the known details of the operation itself. Thus, for coordination problems which have been entirely embedded in this way, the problem of enforcement transforms into much simpler, more closed-ended problem. One way of thinking about this transformation achieved by digital abstraction is that it allows enforcement to largely take place at the level of the elemental digital logic instead of at the higher level of emergent action spaces. Thus, I only need one simple and generic coordination engine for the enforcement of digital logic instead of N complex and error-prone engines–one for each problem.
It could be argued that pulling coordination problems into a digitally abstracted setting is actually the key contribution of modern blockchain technology. It’s true that blockchains often involve complex and innovative consensus mechanisms that push the frontier of Byzantine Fault Tolerance (BFT) performance. But, taking a step back, these mechanisms are really only useful because of the revolution of computation over the past century which has provided a substrate for substantive economic activities to occur in a fully digital manner. While the founding fathers of democracy might have understood the essential considerations underwriting the construction of a BFT consensus mechanism, the idea that much of governmental process could be executed by a deterministic, digital device would have been difficult to comprehend. It is the progress of computation, not game theory, that makes such systems viable.
Indeed, this matter of the expressivity of the digital domain remains the key bottleneck in the conversion of coordination processes to the digital domain. Suppose that we could coordinate to select a policy which we wanted to be executed by some kind of digital governance process. Except in the case of very simple policies, existing computational paradigms have been inadequate to address the nuanced requirements of judgement and reasoning required by such a task.
The modern revolution in AI technology stands to change this. It happens that when humans figured out how to create intelligence, they did it within a digital environment. This means that modern AI is exactly the tool needed to take complex reasoning processes and allow them to be verified end-to-end via simple reductive verification of their elemental operations–the key benefit of digital abstraction.
(There remains an important question, the so-called AI alignment problem, of whether it is possible to specify an AI that implements a policy specified by humans in a manner keeping with general human expectations. While this is certainly an important and non-trivial question, when it comes to many types of tasks there is room for optimism. In a word, while AI alignment is hard and indeed perhaps critical to the survival of our species, aspects of AI such as its digital nature may mean that it is not as hard as, say, human alignment.)