Can You Trust a Criminal?

Yossi Sheriff

Trust Emergence in Large Enough Systems

A few days ago, I ran the thoughts leading to this article with a good friend, a police officer. He said, "No trust. Also, the life of a gangster boss is hell, always worried about betrayal, always looking over his shoulder." I thought, how can you then explain that in prisons around the world, complex networks of trust emerge among the most violent gangs? More surprising still, these trust networks are not only robust—they run sophisticated economies, rules, and are attuned to the members' psychological misgivings. This paradox offers a window into a broader phenomenon: trust emergence. By examining trust as it operates in extreme environments like prisons, we can gain insights that apply to large-scale human systems and even human-AI relationships.

Insights from Prison Governance

I am looking at The Puzzle of Prison Order: Why Life Behind Bars Varies Around the World by David Skarbek, but I will explore his conclusions in other research papers as well. Skarbek writes about California's prison system. Inmates have developed intricate governance structures that maintain 'better than chaos' order (e.g., fewer violent crimes), which is not governed by official authority. His findings are not isolated—criminal organizations operate across continents with what I call "trust matrices." These aren't anomalies—they're clues to a deeper truth about how trust works in human systems and distributed systems. While the resilience of trust matrices everywhere is surprising, it's important to avoid romanticizing these environments. These systems emerge out of chaos in contexts of violence and exploitation, and I want to acknowledge that.

Trust Beyond Truth and Reliability

The traditional view of trust focuses heavily on truth-telling and reliability. When I say I trust someone, I usually mean I believe they will keep their word. But this view misses something crucial about trust matrices: demonstrated capabilities and interdependence. Think about how we trust in everyday life—I might trust my barista to make coffee but not to perform surgery. Trust isn't just about truth, and it isn’t binary; it's about specific capabilities in specific contexts.

Diego Gambetta's research on the Sicilian mafia (Codes of the Underworld, 2009) finds that what mattered wasn't just whether someone would tell the truth, but whether they could consistently deliver on their commitments. A criminal might lie to the police but be absolutely reliable to their associates. Why? It's not only fear as a motivator.

Survival as a Binding Element

Let's look deeper at how survival works as a binding element for trust emergence. Peter Leeson shows that pirates, despite their violent reputation, created elaborate democratic constitutions and systems of shared governance (The Invisible Hook, 2009). The binding element wasn't just fear of death—it was the recognition that their specific survival style required sophisticated cooperation.

We see similar patterns in Peter Kollock's study of exchange structures emerging in ungoverned spaces. In liminal spaces where formal institutions can't enforce agreements, people don't default to 'all-fight-all' pure destructive self-interest. Squid Game is just a TV series, not a depiction of social reality. Often, though not always, communities develop intricate trust systems that often surpass official mechanisms in effectiveness and reliability.

Trust Emergence in Extreme Contexts

The binding element of survival creates interesting paradoxes. In California's prisons, as Skarbek documents, inmates developed systems for resolving disputes that were more nuanced and efficient than the official justice system, even between gangs. Why? Because when survival (e.g., inside-prison economics) depends on maintaining order, systems naturally evolve toward interdependent complexity and reliability.

These insights about trust emergence reach far beyond criminal contexts. Consider long-duration space missions—another contained system where survival depends on sophisticated cooperation. The success of a Mars colony will depend not on individual trustworthiness but on emergent trust structures that transcend national and cultural boundaries. (Just a thought: what is the minimal number of people/nodes needed for trust matrix emergence? How many should go to Mars before trust emerges?)

Applications to AI and Beyond

Perhaps the most intriguing applications lie ahead. As we develop more sophisticated AI systems, understanding trust emergence becomes crucial. The question isn't just "Can we trust individual AIs?" but "How do we create conditions for robust trust emergence in human-AI systems?" The lessons from prisons, pirate ships, and possible space colonies might suggest that binding pressures are one key element in fostering trust emergence in large enough groups. Trust is fragile, but trust resilience—how the system recovers from mistakes and bad participants—is a skill that can be improved.

Conclusion

Looking back at my original question—"Can you trust a criminal?"—I can deduce, reluctantly, yes. In specific settings described here, I must. What matters is better understanding how trust emerges as a systemic property under the right conditions. This insight challenges intuitive views of trust and opens new possibilities for designing trustworthy systems, whether they're made up of criminals, astronauts, or androids.

The law of the jungle isn't endless competition—it's the emergence of sophisticated cooperation under specific settings. Sometimes, there's a need to look in dark corners to see truth clearly.

Footnotes

  1. Trust emerges in two forms: interpersonal and systemic. Interpersonal trust assesses truthfulness and capabilities between individuals, influenced by cognitive biases against betrayal. It’s fragile and requires constant maintenance. Systemic trust emerges in large systems under binding pressures, transcending individual relationships pitfalls to create “trust matrices” of interconnected systems. It’s more sophisticated and resilient than interpersonal trust, sustaining complex interactions even among untrustworthy individuals. Systemic trust doesn’t directly scale up from interpersonal trust; it emerges as a distinct phenomenon when systems reach sufficient scale and have strong binding elements. Good interpersonal trust can help trust emergence.
  2. Trust matrix: An emergent system of interconnected relationships, rules, and practices develops in large groups under binding pressures. Unlike simple networks of interpersonal trust, a trust matrix transcends individual relationships and cognitive biases about betrayal. Trust matrices emerge most clearly in contained environments with strong binding elements, where survival and economics create interdependencies. Even individuals who might be untrustworthy, participate reliably in the larger system.
  3. Binding elements: The forces that create interdependence and necessitate cooperation within a system, enabling trust emergence. Unlike simple enforcement mechanisms, binding elements are fundamental pressures that make cooperation necessary for survival or success. In prisons, these include physical survival, economic necessities, and protection needs. In other systems, binding elements might be resource dependencies, shared risks, or mutual economic interests. The key characteristic of effective binding elements is that they create situations where betrayal or non-cooperation becomes more costly than compliance with the trust matrix. Importantly, while fear can be one component, binding elements work best when they create positive interdependencies rather than just negative consequences. The strength and nature of binding elements influence how trust matrices develop and maintain themselves.
  4. Trust emergence in AI-human systems: While current discussions often focus on individual AI trustworthiness (like truthfulness or capability), trust emergence suggests we should consider systemic trust. Binding elements for humans-AI trust emergence should include shared data environments, interdependent decision-making processes, and resource optimization requirements.
    Just as prison systems develop sophisticated governance without central control, Distributed AI systems might develop emergent trust structures when properly engineered by shared constraints and goals. This has particular relevance for AI alignment - instead of trying to make each AI individually trustworthy, we should focus on creating conditions where trustworthy behavior emerges as a systemic property. However, we must be cautious about directly applying simplistic lessons from human systems, as future AI trust matrices might operate under fundamentally different dynamics. I think that the key question here: what are the appropriate binding elements for human-AI and AI-AI trust emergence, and how many nodes (individual AIs or AI-human interactions) are needed for stable trust emergence?
  5. Comparative analysis: Skarbek investigates a wide array of prisons, including facilities in Brazil, Bolivia, Norway, England and Wales, as well as a prisoner of war camp, women's prisons in California, and a gay and transgender housing unit in the Los Angeles County Jail https://academic.oup.com/book/33625?login=false
  6. Governance theory: The book develops a framework to understand how social order evolves and takes root behind bars, focusing on the relationship between official and informal institutions https://mises.org/quarterly-journal-austrian-economics/puzzle-prison-order-why-life-behind-bars-varies-around-world
  7. Governance regimes: Skarbek categorizes prison governance into four ideal types: official governance, co-governance, self-governance, and minimal governance https://mises.org/quarterly-journal-austrian-economics/puzzle-prison-order-why-life-behind-bars-varies-around-world
  8. Diego Gambetta's work stands out, particularly his 1988 book "Trust: Making and Breaking Cooperative Relations" and especially his later work "Codes of the Underworld" (2009) which specifically examines how criminals solve problems of trust. His research on Sicilian mafia provides insights into trust-building mechanisms in high-stakes, illegal environments.
    There's also interesting research from a different high-risk context - Jeffery Stevenson Murer's work on trust development among combat units and paramilitary organizations. While not about criminals per se, it examines trust formation in life-or-death situations.