What is the difference between 100 kilograms of human being and 100 kilograms of algae? One answer to this question is the veins and arteries that carry nutrients throughout the human body, allowing for the intricate coordination needed in a complex organism. Energy requirements determine how the evolutionary process settles on the body plans appropriate to an environment — one way to tell the story of life’s major innovations is in terms of how a living system solves the problems of increasing body size with internal transport networks and more extensive regulation. And the same is true in our invented information systems, every bit as subject to the laws of physics as we are. Computers, just like living tissue, seek effective tradeoffs between their scale and energy efficiency. A physics of metabolic scaling — one that finds deep commonalities and crucial differences between ant hives and robot swarms, between the physiology of elephants and server farms — can help explain some of the biggest puzzles of the fossil record and sketch out the likely future evolution of technology. It is already revolutionizing how we understand search algorithms and the genius of eusocial organisms. And just maybe, it can also help us solve the challenge of sustainability for planetary culture.
This week’s guest is Melanie Moses, External Professor at the Santa Fe Institute, Professor of Computer Science and Biology at the University of New Mexico, and Principal Investigator for the NASA Swarmathon. In this episode, we talk about her highly interdisciplinary work on metabolic scaling in biology and computer information-processing, and how complex systems made and born alike have found ingenious ways to balance the demands of growth and maintenance — with implications for space exploration, economics, computer chip design, and more.
If you enjoy this podcast, please help us reach a wider audience by leaving a five-star review at Apple Podcasts, or by sharing the show on social media. Thank you for listening!
Visit our website for more information or to support our science and communication efforts.
Join our Facebook discussion group to meet like minds and talk about each episode.
Melanie’s UNM Webpage & full list of publications.
“Beyond pheromones: evolving error-tolerant, flexible, and scalable ant-inspired robot swarms” by Joshua Hecker & Melanie Moses.
“Energy and time determine scaling in biological and computer designs” by Moses, et al.
“Shifts in metabolic scaling, production, and efficiency across major evolutionary transitions of life” by DeLong, Moses, et al.
“Distributed adaptive search in T cells: lessons from ants” by Melanie Moses, et al.
“Curvature in metabolic scaling” by Kolokotrones, et al.
The NASA Swarmathon.
Podcast Theme Music by Mitch Mignano.
Follow us on social media:
Twitter • YouTube • Facebook • Instagram • LinkedIn
Michael: Melanie Moses, it's a pleasure to join you in the complexity.
Melanie: Thanks. It's great to be here.
Michael: You have sent me some very interesting papers on what I think is preliminary research to where you spend most of your time these days. But this work is so interesting, and speaks directly to my abiding curiosity in major evolutionary transitions. So I'd like to start by talking about the PNAS paper you co-authored on shifts in metabolic scaling. But actually, before we get into that, I'd like to know a little bit about your background as a scientist and how you got into science?
Melanie: Excellent. Well, I have — as many people at the Santa Fe Institute — have probably a rather meandering path to my current place in science. I started with an interest in physics, computer science, and in fact, I think probably my first introduction to science that I was excited about was reading the book Complexity, which I read on the airplane to go to my freshman year in college. So I was sort of influenced…before I chose my major, I had this amazing book describing this interdisciplinary center where people brought together ideas from physics and economics and…I was really excited by that and so I majored in something called symbolic systems, which is an interdisciplinary computer science, philosophy, particularly focused on philosophy of mind and philosophy of language as a way to approach artificial intelligence.
Michael: So that's a very different approach — just so I can get this clear in my head – that’s very different from the non-symbolic AI, like machine learning, deep neural networks, right?
Melanie: Yes, this is much more traditional AI, good old fashioned AI that has in many circles been sort of supplanted, surpassed by these machine learning approaches. We didn't learn any machine learning in my undergraduate years, which of course, now that's quite useful. But I started studying actually agent-based modeling and robotics as an undergrad and then I worked in computer security for a while and just was not excited by the work I was doing. So I spent a few months living in the rain forest in Costa Rica and decided that I what I really wanted to study was ecology. And so, I switched entirely. In fact, I had not taken a biology class as an undergrad, my last biology class had been in 10th grade.
Michael: Oh, wow.
Melanie: So I had a bit of work to do. I went to school in the evenings in the summers and things, and picked up some biology and then pursued a pilot biology PhD at UNM, University of New Mexico, right down the road from the Santa Fe Institute, which was on purpose and began working on scaling theory as part of my biology. There was nothing really computational about that original work. It was mathematical biology and theoretical ecology. But that's sort of my entry into the world of complexity research.
Michael: I may be jumping ahead of ourselves by just pointing out it's interesting how, in retrospect, it all seems to come together…that all of these different disciplines are now so intricately woven into a single understanding, in what you're doing. And it's funny because when I spoke to Jen Dunne, she got her start in philosophy also…there's something about the, I don't know, the disposition of the kind of person drawn to this vastly synthetic work — you know, the big questions, the deep ones?
Melanie: Yeah, I think the real key for me was that living in the forest, was watching trails of ants, often army ants, but other leaf-cutter ants and having worked in network security, just having this sort of idea that networks are everywhere and then you go to the rain forest…right? I was trying to escape that world. I didn't want to work in that world anymore and then I go to the rain forest, and it's full of these ants that are building networks and basically dominating this ecosystem by these dynamic networks. You seem them…the forest floor is moving with these ants everywhere. And, yeah, that connection was very... It was sort of a deep connection that just made me feel like, "Oh, I just can't get away from these networks. I'm going to spend my life investigating questions about networks and I think the symbolic systems approach, which is just the idea that you really can draw from really different disciplines to come to some kind of synthetic understanding."
Michael: Well, I want to call the shot for this conversation because your recent work on swarm robotics and ant foraging and T-cells in the immune system…there’s a clear link there, the cyber security and ants and everything. But to get to that, I think it's important, Carl Sagan says, "In order to bake an apple pie from scratch, first you must create the universe." So there is this beautiful portrait of the history of life and its major transitions that comes through in some of your earlier publications. You were just talking about the more straightforward scaling math stuff. And I think that this is where we see the braid, where we see life as a physical process as an energetic process unfolding in time. Yeah, I'd love to dig in, first on this PNAS paper, the DeLong and et al. piece on shifts and metabolic scaling. So could you set the stage a little bit for the thinking about this piece? The history of scaling thought in biology.
Melanie: Sure. I'm glad we're starting with this because I do think this is really the place where the implications of scaling become most readily apparent. So the basic concept with the scaling laws that were developed here at SFI by Geoff West and Jim Brown, Brian Enquist, I had the really great pleasure working with them on my PhD, and as a postdoc, really trying to understand these scaling laws, which sort of at their heart are explaining this non-linear relationship between metabolism and mass in plants and animals. So the basic empirical pattern, known for many decades at this point, is that the metabolic rate of an animal is proportional to its mass to the three quarter power. And basically what that means is if you compare an elephant and a mouse, the elephant is a million times bigger, but its metabolism, I'm going to forget the number, it's something like 10,000 times faster, not a million times larger.
So bigger animals have, obviously larger energetic needs, but not proportionally so. And the scaling papers that came out of SFI, and really this tremendously influential body of theory, had relied on data that suggested that this went across the tree of life down to bacteria, certainly in plants and animals. But John DeLong had some data that suggested that, in fact, if you look at the smallest bacteria, even unicellular eukaryotes, you don't see the same scaling relationship. And so this started from a mindset that this is a universal pattern. We started to speculate maybe there's something else going on here. What I love about this work is it really is in some sense exceptions that prove a rule.
So the explanation for these three quarter power scaling came from the geometry and dynamics of a fractal branching network. And so, you can look outside; there's trees all over; you see fractal branching networks with big trunks that branch into smaller branches. You've got a beautiful fractal branching network in your body, right? Your aorta branching out to capillaries. But that's not true in bacteria. There's no visible fractal branching network. And so really, when we thought about this, we thought, this constraint, this three quarter power constraint, perhaps shouldn't hold in these other cases. And when John went back and he looked at the data that others had collected and his own data, it became apparent that the basic fundamental energetic constraints are different as you go across these evolutionary transitions from bacteria to eukaryotes, things with no nucleus to multicellular plants and animals.
Michael: One of the things that I love about this paper is how it makes clear, or at least proposes how it could be the case, that each of these radical reconfigurations of body plan, each major shift in the size of an organism over evolutionary history has been the answer to a crisis in energy distribution. And I always thought about it in terms of information flow, like cohesion. When I got into complexity science, it was through the work that David Krakauer was doing with Martin Nowak at Princeton on the error catastrophe and the emergence of language and how these networks grow in response to error rates as they scale. But there's this other piece, which I love in your work here, about how this is about physical objects and surfaces and volumes and how energy diffuses across membranes. And I'd love to hear you say a little bit more about actually what you found and what you think is going on in these transitions.
Melanie: Excellent. So yeah, I'll start by talking about energy. I, too am really fascinated by the scaling of these information systems. And obviously, they are not unrelated, right? Biological systems are energetic and information processing systems. But the energy story, in some sense, is fairly simple. The argument here is that we start with bacteria that have a scaling that essentially suggests that it's superlinear, so that as the bacteria are growing, they are getting essentially more energetically efficient as they grow larger. So that's a neat trick! Certainly you want to max that out so you can see bacteria getting larger and larger over evolutionary time…
Michael: And why is that? Why is that happening?
Melanie: Well, so there's essentially two benefits to being large. One, you're able to consume more resources, which means you can then put those into reproduction. In the case of bacteria, you can double your population size by consuming more and out competing, potentially, other species that are smaller. But you're also able to consume energy, not just in a total faster rate, but at a sort of per-volume faster rate. So it's just things are getting better and better, faster and faster, as you get larger as bacteria.
But there's a limit to that, because all of this energy that the bacteria are using, this has to happen by diffusion, and diffusion is a slow process as volume gets larger, diffusion becomes limiting in that it just is sort of the rate-limiting step. To move things around in a large space eventually becomes untenable. And so it sets up a place where things can no longer become more efficient by getting larger unless they switch design. And so the argument in this paper is that chloroplasts and mitochondria are essentially solving a problem of diffusion. These are organelles, which are now changing an essential surface-area-to-volume problem that a bacteria has. Everything that comes into the bacteria has to come across its membrane. And that's a surface area. But its volume is growing faster than its surface area. So it's got more need for stuff on the inside, and the rate at which stuff can go in and out is slowing relative to that volume on the inside.
And so the solution is to create these membrane-bound organelles. So you basically put lots of membrane on the inside. That's a genius answer to the problem. And so it allows cells to get much bigger, but it also has a different scaling constraint. So what we found in the data is that when that transition happens, now the scaling relationship becomes linear rather than superlinear. These membrane-bound organelles are a solution to a problem, but they come with their own constraint. And that's the main message in these evolutionary transitions is each transition is a sort of an evolutionary, almost a technological, solution to a previous problem about how energy can flow. But every technology sort of has its own new set of constraints that it imposes on the organism. And so, it scales up until that really hits a wall and there becomes some other process or technology that gets around that constraint in some way.
So with these membrane bound organelles, you still have a diffusion problem, you still have to get things from the outside then to these organelles that are internal. And so this fractal branching network is sort of the solution to that, you now have, basically, a branching pipe going from where, for example, the oxygen can come from the outside into the organism and be distributed really efficiently to these organelles that are inside the cells. And, again, that's a great solution that allows things to get many, many orders of magnitude larger, plants and animals, but it's got its own constraints that as things get bigger and bigger, they're slightly less efficient. They can't run at the same speed as a smaller things. And so this is just sort of putting these transitions that are already known into this sort of energetic explanation for why we see these transitions in the places that we do and the sizes that we see.
Michael: I'm reminded, to call back to my conversation with Luis Bettencourt, and we were talking about slums, and how do you fix slums? How do you get services into these places, and it was all about mapping them and finding where to cut into these blocks, where there's no infrastructure. There's no streets or piping or anything. And it's the same question. It's the same issue of how do we diffuse resources into these areas and clean out the waste from these spots. That's one thing. And then the other thing that you mention in your paper is that each of these evolutionary transitions apparently coincided with major increases in the concentration of oxygen in the atmosphere and oceans.
When I was talking to Olivia Judson, about her work on major energetic transitions in life, there was this question that she was holding that I feel like you just answered, which is “why endosymbiosis,” like, “why bacteria drawing other bacteria inside themselves,” because they're all of these other instances where there's deep and intimate symbiosis, but not where one organism has literally become a component inside the other. And it seems as though the oxygen as a metabolic byproduct has sort of created this. And then now there's also talk about how it may have been diffused oxygen that prepared the world or enabled the Cambrian Explosion and that radical diversification. So I don't know where I'm going with that, but there's lots of cool stuff that branches off of this understanding.
Melanie: Yeah, and I think this is why this is such a core idea in complexity — it’s most obvious when you think about a circulatory network — it's solving this problem that once you've created this complex system, it's full of lots of parts. To some extent, it's tightly or loosely bounds depending on the system, but once you've got this collection of things that have to work together in some way, you need to allocate some infrastructure to allow that to happen. So in the slums, that’s sort of Luis’ point, is that there's no space for the infrastructure. And that infrastructure is really key. There's resources you have to move in and waste you have to move out and that is a feature of any complex system. Evolved, engineered, it doesn't matter. You've got heat byproducts and waste byproducts, and you've got to have networks to move them out.
And the circulatory system, the branching patterns that we see in computer chips, the branching of trees, those are all examples of sort of how resources can be moved through space. And so I think this is, yeah, you raise this point about the volumes and space being so important here. Because in all of these biological and engineered systems, there really are physical things that have to move through space, and a network that makes that efficient becomes sort of an evolutionary imperative.
Michael: So my question for you is: I look at this, and I think a lot about what comes after. And both in this piece, and the next piece I want to discuss with you, that you wrote about scaling work…in both, you cast a glance forward into the next crisis and its resolution. Or arguably a recent crisis, because maybe we've already hit some metabolic limit as individual organisms and that's where sociality has come from, and that civilization itself is a response to this. And so I can't help but think about how this curve goes from superlinear, to linear, to sublinear as you reach out into these larger visible organisms. And then draw a dotted line between that and reports that we waste 40% of the food that we generate in society and wondering about... This is perhaps an insanely broad analogical leap…
Melanie: That's my favorite. Go for it. [Laughs.]
Michael: But the question is, to what extent are the inefficiencies that we see in energy use in production and resource distribution in society, evidence of some sort of sublinear metabolic scaling, and kind of inescapable?
Melanie: Okay. Ultimately, my interest in this area was driven by two things, one, I told you that I was fascinated by ants. And the other is, like many people, I'm a little concerned about what we're doing to our world. And we are part of this enormous metabolic system where we're extracting largely fossil fuels out of the ground, moving them across these networks and raising the temperature of our planet among other crises we're creating. Maybe that big crisis is a lot to take on with scaling theory. So let's put that aside for a little while, and understand the scaling properties of social animals. And so that's part of what motivated trying to understand the scaling of ant colonies.
That is kind of this next evolutionary to transition is going from bacteria in unicells in plants and animals then to eusocial creatures, ant colonies being one of the best examples of organisms that really are best understood as collective organisms. Actually, when we wrote this paper, there was very little data on the scaling properties of ant colonies. And since then, there have been quite a few papers that have come out. And so in this paper, we kind of speculated…
Michael: We're talking about which one?
Melanie: Sorry, in the paper on evolutionary transitions.
Michael: PNAS?
Melanie: The PNAS paper.
Michael: Okay.
Melanie: So one of the evolutionary transitions in which there's some debate about whether this counts, but most people consider sociality as an evolutionary transition. And so the question there is, well, why do you see that transition? Is there an energetic explanation for it? If you consider a colony, a large colony of ants, which might have 20 million ants in the colony, it has this great advantage that it doesn't have to internally sort of house is it networks. So you can actually have networks that are built of the agents themselves, these ants that are forming paths in the rain forest or out here in the desert. And they don't really have to pay the full cost of sort of maintaining that. They don't maintain all the tissue around that network, and yet they use the network to bring in resources and take out waste.
And so it's a nice sort of new technological answer to the fact that networks are essentially less and less efficient as they get larger. So an elephant, for example, has to build a really big network. If the elephant could be as fast as a mouse, it would need an enormous circulatory system. So I sort of imagine this elephant walking around with a giant network on its back, the thing that would pump enough oxygen to its cells, so an elephant cells could run as fast a mouse's cells. So sort of this impossibility, like the network would have to be larger than the elephant itself. So sociality is an answer to that question. You can have a network that's external to the bodies that are being maintained and have all these energetic requirements, if you are much more loosely coupled. And so that's one of the things that we think ant colonies are doing.
So in this paper, we speculated that maybe when we look at sociality, we might see a different scaling exponent in say, the metabolic rate of an ant colony versus its size, we might see a different slope than we do for individual animals. And maybe there's this trend, this exponent, we've gone from superlinear, to linear, to this three quarter power, maybe we should see a slightly smaller, an exponent, that you can make bigger colonies, but it gets sort of harder and harder to make a bigger one. Interestingly, the data suggests that the exponent is almost the same, really close to this three quarter. So you put big colonies into a metabolic chamber versus small colonies, and the metabolism of the whole colony is going up at about this three quarter scaling, again, maybe slightly steeper.
Michael: So does that mean that given the amount of available oxygen in the atmosphere that the biggest ant colonies have a comparable biomass to the largest elephants or whales? Or…
Melanie: So where do they fit in this?
Michael: Yeah.
Melanie: Okay, one way to look at this is the biomass of ants is about the same as the biomass of humans. So the two most ecologically successful groups on the planet are social. That's not quite the question you asked, I don't know what the mass of the largest ant colony is, I would imagine it's considerably smaller than the mass of an elephant, but I'm not sure. It depends on what you count, because they have a lot of external…their nest is extremely large and heavy. In fact, there's an image of grass cutters that I frequently like to use, grass cutters in Argentina, so these are colonies of millions of ants. And when you look at the colony, actually, its mass must be greater than that of an elephant because there are people who have excavated this ant colony and there are multiple people walking around inside the cavity left from excavating this ant colony. So it's enormous.
Michael: Is that when they're pouring lead down…
Melanie: Yeah, they're making these casts of the network structure, actually inside the colony. So it's a good question. I'll have to look up what is the actual mass of those largest colonies.
Michael: Well, I mean it just gets to these questions about like the 1950s monster movie trope of the giant insects, and why we don't have dragonflies with a six-foot wingspan anymore. Like, why “The Blob” isn't a thing. We have no reason to fear The Blob. And it feels like our science fiction has grown up and matured into, okay, now we fear swarms.,
Melanie: We maybe should fear the swarms.
Michael: It's a little easier to suspend your disbelief.
Melanie: And certainly, you could never have an ant that was the size and power of an entire 20 million ant colony. Its body design is not designed to support anything near that size. So certainly is showing how you can escape that constraint that way.
Michael: I mean, for what it's worth, I do appreciate how when they blow up Ant Man in the Avengers films to the size of an elephant, his movement is comparably slower.
Melanie: Slower, yeah, I appreciated that too.
Michael: Although, I would imagine he would just sort of… Does his vasculature scale?
Melanie: No, he would have a diffusion problem. The insect tracheal system is not designed to scale to that size for sure. [Laughs.]
So, yeah, maybe a few liberties there. So to take this back to the original question, in ant colonies, we can see these networks and we can see sort of the advantages, and even show that at least in theory, the advantages are mathematically describable, what's the fractal branching network for foraging and retrieving resources. There was a great paper by by Joon et al., gosh, 15 years ago now, that really looked at that question that you can think of the colony's cardiovascular network is that foraging trail. And then we see the metabolism of these colonies following this similar three quarter power scaling relationship, and we don't really understand why. There's all kinds of interesting hypotheses. One of them actually, one of the best ones is the “lazy ant hypothesis,” which is the way the colony stays on these on this three quarter scaling line is, the bigger it gets, the more lazy ants there are that are just sitting around doing nothing.
So there's these measurements that show large colonies have lots of workers that are low metabolism because they're just sitting around doing nothing. Whether there's an analogy to your question about inefficiency in human populations, I am not sure. Do we have more people sitting around doing nothing? It almost feels like the opposite. The more energetically connected our society is the more it generates more and more activity.
Michael: Then again, so much of the work now, so much of the activity, at least at the level of an individual human being, is like, “I’m sitting here at my desk,” rather than, “I'm out throwing spears at large animals.” And just the very fact that we're even having a Universal Basic Income discussion in this presidential election cycle is, I think, evidential of something about the decoupling of the world that we're used to, and moving into a cohesive planet-scale framework economically.
Melanie: Yeah. So part of the lesson from the ant colonies is that these ants, which are viable organisms that can move around as independent animals, they can't survive without being tapped into the system of their colony that feeds them. In fact, there's a lot of evolutionary mechanisms that keep ants from just eating food when they're out in the world, they have to have it processed by the colony. So they're very tied to this infrastructure. So that changes their behavior, it makes some of them lazy, it changes all sorts of things about how they interact with each other. And I think really a fundamental societal question is, how does being part of this vast economic system, that you and I and everybody else are part of, how does that change our behavior? Have we sort of become ants in the colony? This is sort of the Borg idea, that we've had to change things about our own life history in order to be participating in this huge energetic system that delivers us our food and gas for our cars.
The first paper I actually wrote was with Jim Brown. It was in some ways, an analogy that I think doesn't directly hold. But this was a paper that showed that as societies become more consumptive birth rates fall. The demographic transition is a well understood…or well documented phenomenon, not well understood. The argument there is that as you become more energetically consumptive, the system essentially has to have all of its components slow down. Just like the cells of an elephant slow down, people in a more consumptive society sort of have to slow down their reproductive rate. I mean, if you could imagine if we were still having eight to 12 children each that are biologically possible, we've got all these resources in the United States, can you imagine how quickly we would have destroyed the planet if continue a growth trajectory like that? It's impossible.
And what's astonishing is that the most important biological imperative is reproduction, right? And we all just voluntarily limit that dramatically when we're in large consumptive societies. And maybe we change a lot of our other behaviors in order to allow the whole system to work.
Michael: Yeah, in a way, it's kind of a hopeful portrait that you're painting here, that as we approach what we're thinking of is overpopulation that we adjust, and that it's more of a sigmoidal curve than it is this...
Melanie: So it's more optimistic than the continued exponential, super-exponential growth of our population and our energy consumption. It's more optimistic than that. It's not terribly optimistic. So if you do the math and you look at the rate at which we've slowed population growth and how much energy we consume individually to slow that growth, the equilibrium point is billions more people, each of which are vastly more energetically consumptive. So if we continued on the current trends, I mean, our energetic consumption would be orders of magnitude greater than it is now as a planet. And that is clearly not viable. We can't do that. And so it's not clear…there’s not some nice asymptotic continuing-of-the-trends. We can't stay on the trend that we're on.
Michael: Certainly, but I mean, we're talking about the membrane of the planet, and the membrane of the carrying capacity…
Melanie: Right. Yeah, yeah.
Michael: There's the question of, if we're going to look at the bacterial transition into eukaryotic cells for advice, then maybe it's like “underground cities.” You know? We’ve got to add that third dimension, create more membrane surface. Orbital colonies.
Melanie: It's interesting. We need ways to pump more of the CO2 out of the Earth's membrane that we currently have. We don't have any way to get it out. Of course, it's not quite the right analogy. This is a system where it's only a problem because this was all locked up and we've discovered it and we pump it back out. But it does point to one other interesting piece of this, which is that, for all of these other transitions, there's been some outside that you can incorporate: multiple cells get together and now they've created some new larger thing. Colonies are great examples of this. Now they've sort of incorporated part of the environment into the colony. Certainly, humans have done that.
Once we're at a planetary scale, “outside” — it's hard to kind of conceive about what that is. I mean, you get into science fiction ideas of colonizing other planets, which energetically is also probably not terribly viable. [Laughs.] It’s a really hard boundary. It's a different kind of boundary to think about crossing. The fact that we're doing all of this on a planetary scale…I don't think we have any answers. Scaling theory gives us some ways to think about it, maybe.
Michael: Yeah! Well, I want to skip ahead now to a paper that you lead authored in Philosophical Transactions B: “Energy and time determines scaling and biological and computer designs.” Because up to this point, in this conversation, we've been talking about scaling out from the meso to the macro, or from the micro to the meso. I want to talk about how you have paired computer architecture here with biological anatomy and how you're looking at it. You talked about how chips really haven't grown all that much individually. Each chip, it’s kind of a same surface it had 40 years ago. The change has been small enough that you and your co-authors just assume it's constant in this paper.
Melanie: A bit of a bold assumption, but yeah.
Michael: But for the sake of simplicity, it makes sense. Whereas, what you're looking at is an increasing "vasculature" into the micro scale, a finer and finer carving up of that surface area. Talk about what the differences are in those two systems. It is very easy to draw analogies, but what did you and your co-authors observe as departures from the way that this is handled by machines versus the way that this is handled by biological systems?
Melanie: Great. I think the difference is probably the way this is handled by information systems, versus the way that it's handled by energetic systems.
Michael: Yeah, not machines.
Melanie: Yeah, so, computer chips. So we started by trying to understand whether the scaling theory that talked about moving things through networks also applied to computer chips, how it might be the same and different. It maybe is helpful to start with one of the big, obvious similarities that we kind of came across was that computer chips have a fractal branching network — or some computer chips, for some iterations of chips over a few decades, had fractal branching networks that followed precisely the West et al. fractal branching geometry. So if you took the equations that were really sort of simplifications and idealized equations to describe a circulatory network, they really describe in two dimensions, the network called an H tree, that connected components of a computer chip. It was the thing that allowed fast synchrony on a computer chip. So this H tree would deliver a timing signal through this branching network, the signal would get everywhere at the same time.
But it ran into exactly the scaling problem, which is to grow that network to a larger and larger size. You both have to make the main arteries longer and you also have to make them thicker. And so suddenly the chip is all network. It's just a network to deliver timing signals. You don't have room to do any computation. So, that's problematic.
Michael: Bureaucracy.
Melanie: Bureaucracy. Yeah, so that's exactly it. [Laughs.] These transportation networks, it’s all just infrastructure to move stuff around. All the action happens out in the leaves. And so this bureaucracy becomes hugely burdensome. Interestingly, one of the differences is that computer chips are largely 2D, it's a flat surface and all of the transistors are on one two dimensional plane. The way that engineers came up with to deal with the fact that they needed more network as they put more transistors on the chip is they built up extra layers. So now, I don't know, up to 10 or 13 layers just designed to hold all the wires, essentially, that are connecting the components.
So when I talked about the elephant walking around with a network on its back, that's exactly what computer chips do. Because they're in 2D, they have another dimension. They can actually carry a a network on their back. So they don't really have to internalize sort of the cost of networks, the way that a three-dimensional animal has to internalize that. So, that's one difference. But maybe the more interesting difference is that we all kind of know in computing technology, transistors are getting smaller and smaller. They've been getting Moore's Law talks about the way that they've been getting systematically smaller over time. And that means you can have things that are not spatially separated, but you can have more of them.
And so again, that changes the geometry of these networks, so that computer chips basically can get bigger and bigger…the power that you can process, the information that you can process on a chip can be linearly related to its size, it doesn't have these diminishing returns of this kind of three quarter scaling relationship that we see in plants and animals. So the elephant-sized computer chip, basically, all the components run as fast as they do on a mouse. And that's kind of part of the explanation for this tremendous increase in computing power that we have. There's a cost you don't have to pay if you're moving bits around on a chip that you do have to pay if you're moving oxygen molecules around in an animal.
Michael: So this is an interesting opportunity to look at the continuity of energy processing on the surface of this planet. Where do you stand…? Just to take a step back into the quasi-philosophical, there are people like technologists, Kevin Kelly, who tried to articulate the internet and all of our technological creation as continuous, that draw Moore's Law back into the prehistoric and say that we're looking at this one process and that the industrial revolutions and the deliberation of available energy from the fossil fuel revolution and so on are like what we were talking about earlier in terms of the Great Oxidation Event. We're living through one of these punctuations, and that we can look at the built world as a continuation of the process that we are. It's not something separate in that regard.
I'm curious…obviously, metazoan arrival…the evolution of complex animals…created new opportunities, but also new challenges for simpler forms of life. And I wonder if you think that's a fair analogy, and what we're looking forward to in that case.
Melanie: Yeah. Yes and no. We're part of a group right now that's writing up something on, essentially whether computing technology is engineered or evolved. And I think you can ask that question about any technology in our energetic system: that's oil pipelines and ships and airplanes and all those things. Are they engineered or are they evolved? And in some sense, clearly, they're engineered. We have specifications. If you look at an airplane, and it's pretty clearly not evolved…
Michael: At the human scale.
Melanie: ...at the human scale. But then you look at these systems, the way we put things together, and it's very much trial and error and things that don't work and don't integrate with the rest of the system or are left behind. The process that's generating this technology, I think is very similar to evolution and in important ways. Your question is maybe is it is there still something fundamentally different about technology and things we've built?
Michael: Or is it simply a matter of scale? Because, especially, I was watching the old Nova documentary on the the X planes, Lockheed versus Boeing coming up with the next fighter. And at that point, which by now is years ago, we were designing our fighter planes with computer assistance. Like you just said, everything we were doing was applying a kind of an evolutionary algorithm to designing the components of something so the process itself was complex, even if the result was merely complicated.
Melanie: Right. Yeah, I think that it's an important point. So I think the idea that we are sort of just this continuation of this previous trend is, it's true in one sense, but I do also think there are some fundamental differences and maybe, one way when we look back at trends, when we look at things like Moore's Law, there is a sense in which it is very optimistic. It is a particular case. There are very few other things where things have just doubled in power every couple of years. Luckily, solar panels seem to be…photovoltaic efficiency and cost and those things seem to be following a Moore-like trend. So that's, I think, the one tiny bit of hope I see for our energetic future.
I wrote a paper that pointed to that, and a lot of people said, "You know what, that's not going to solve the problem, we're not going to have a solar powered society.” It’s not going to solve the problem. But it's a little piece of optimism that some technologies follow this kind of more like trend. You can look back at vacuum tubes back hundreds of years and you'll see Moore-like trends. The vast majority of our technological innovations are actually pretty slow. And there are these moments where we just we figured out how to get oil out of the ground, and that created a huge shift, but the efficiency with which we've gotten oil out of the ground hasn't followed any kind of Moore-like process. It's extremely rare and understanding…Doyne Farmer does some work trying to understand when you get this kind of more like efficiency and when you don't, it seems like figuring out how to get more of that is a really important thing for us to do right now.
So there are particular technologies that seem to really rapidly take off. And I think some of those don't really look like evolutionary processes. And there's the sort of punctuated moments in time where we discover how to leverage some resource that we didn't know how to use before. And so we get a big increase in something because of that. That's different in scale, but similar in process, maybe to these other evolutionary transitions.
Michael: To kind of depart from this particular conversation, one of the things that I admire about this paper, I think it's the 2016 paper, is how you bring time back into the conversation of scaling. And I'd like to speak to that because you draw a lot on West et al., and the way that they've looked at the actual physical structure, but you and your co-authors make a good point, which is that there are flows going through these systems, and that it's not just about whether the branching vasculature in your body minimizes the waste heat, and the waste diffused oxygen through that membrane, but also that it's really trying to get nutrients into the capillaries as quickly as possible. And as these different sections of the vasculature gets smaller, the diameter gets smaller, and the flow rate has to change. This answers some questions about what we were seeing, the departures we were seeing from that that clean sub-linear mammal scaling curve. And I'd love to hear you say more about that.
Melanie: Yeah. So in this paper, we really did try to take two different hypotheses, I guess, about how it is that these fractal branching networks, what they're minimizing. And right, as you said, the West et al. model really said that it's about minimizing energy dissipation, energy waste, essentially the rate. And another argument that it's about maximizing the rate of flow or minimizing time. And so in this paper, we argued that organisms should be doing both. Right? So if you're maximizing your reproductive rate, you need energy to provide to the next generation, and you need to do it quickly. If someone else can do it faster, you're going to lose the evolutionary race.
And so when we sort of married those two things, we put both of those constraints into the model, minimizing energy and time. And we assumed that they were equally important based on no knowledge of how we would make any other assumptions. That is an interesting question. Are there places where minimizing energy or minimizing time is more or less important? We said to first brush, we'll assume they're equally important. And yeah, what that did was set up sort of a slightly different optimization that leads to almost three quarter power scaling, but some systematic deviations from three quarter power scaling, which actually match what's been observed in the data.
There's this curvature, slight curvature in the scaling relationships, great paper by Kolokotrones and Savage pointed that out. And that scaling, the original scaling model predicts some curvature but in the wrong direction. And this model actually predicts curvature in the correct direction. And it wasn't set up to do that. It was set up to model computer chips.
Michael: So you're talking about specifically that the metabolisms of the very smallest and the very largest mammals don't fit the curve?
Melanie: The curve. Yeah. They're very slight deviations. But interestingly, one of the results if that's in fact the empirical truth, the scaling exponent you get depends on where you measure on the line because it's not quite a line. So taking this line through different data sets will get you slightly different scaling exponents. I think that if, in fact, there's a little bit of curvature that helps to explain why when people fit lines, they get different numbers. This has practical importance on how we understand these things as well to be able to come up with a theoretical explanation that accounts for the curvature.
Michael: There is, in the discussion of this paper, when you go into more detail on evolutionary transitions, you compare the transition to sociality we just discussed with this movement to decentralized architecture in computer networks. And I can't help but think…you talk about, although I don't think you say it by name, the Internet of Things. And the way that we seem to be observing a transition into, and again, you're a swarm roboticist…so it's like this movement into...I like Ricard Solé’s description of solid brains and liquid brains and how an ant hive, or a human city is a sort of a liquid brain. And it looks like our computer architectures are starting to move in this direction, out of the block that sits in front of you, and into this flowing thing.
Melanie: Yes, a key question is a question of centralization that a key constraint clearly in these multicellular plants and animals, the network is centralized. There's this big, bureaucratic aorta that everything has to flow through. And that's a problem, right? It causes things to have to slow down. And so you can get around that constraint by, for example, becoming a social organism that doesn't have quite the same centralization, that allows more decentralization and basically, that’s the analogy, right? A thinner aorta, you can function with something without complete centralized flow and we certainly see that in the design of computer chips. There’s not a central place where every bit has to pass through this central location.
And yeah, we now have multiple core computer chips, Internet of Things…there is certainly this trend towards decentralization, but I think we need to be careful in saying that that is what leads to a scalable solution. If you look at, for example, Bitcoin, and these kind of trends, their whole setup is trying to avoid centralization. And they're horribly not-scalable. This is why we're pumping so much CO2 into computers in China that are processing, that are doing these mathematical operations in order to verify things without centralization. So with centralization, it's really I think about a balance and an understanding about what needs to be centralized and what should be decentralized in order to have a scalable system.
In the swarm robotics domain, our most recent work in that domain is we're trying to build a scalable swarm of robots. So these are robots that might search for resources on the surface of Mars. And we want to imagine sending thousands and thousands of robots to search a really large area. And what you soon run into is, no matter how clever you are with your algorithms for how the robots search for things, their problem becomes all transport. Like, if you're going to take stuff back to the base where the humans are, all of the work becomes robots moving over long distances.
And so we were actually able to use scaling theory to say, you know what, we could have independently searching robots that stay in their areas and build a fractal branching network of basically a series of increasingly larger dump trucks. And they can take care of the transportation problem and scaling theory tells us exactly how they can take care of the transportation problem, so the robots out in the hinterlands can keep doing their things as fast as as needed. There's always a depot or a little dump truck for them to put their stuff in connected to the centralized network. So there it's really the balance of how do you keep independent units operating as independently as possible, but connected to a larger system that's taking care of the coordination and transportation piece.
Michael: So that seems to be the real meta lesson here. If we pull back just a little, and ask the unasked question of why these major evolutionary transitions, these metabolic innovations didn't just replace what came before. To speak to the Bitcoin thing in particular, I remember in 2017 so many people were convinced that decentralized money systems were just going to replace centralized money systems and that decentralized systems of governance were just going to radically outcompete. It was, I think, a misfit mammals-scurrying-under-dinosaurs kind of a story, and yet, mammals and dinosaurs lived together rather harmoniously for like 150 million years. So why was that the case? That story doesn't really work.
It sounds to me in this paper, like you're pointing to a robotic ecosystem in which we have... The real answer is to take a look at this from the ecosystem level and look at how these things are occurring, not just within an organism or within an ant-like super organism, but within the entire rain forest, and how those systems all fit together. And that the different sizes of organisms are themselves representative of a kind of modular and mobile vasculature.
Melanie: Yeah, I think that's a question we don't know the answer to: is there an equivalent to a metabolic network in an ecosystem. So if you just think about an animal, you can sort of think about, just like our little robots that are doing their thing and get to work in parallel and in complete ignorance of the rest of them. Each of your cells, in some sense is doing that, right? It's doing its own thing. It doesn't need to get a signal from your brain in order to do its work. It's sort of always doing its work. And every now and then it gets a signal that tells it to do something different. But it's that marrying of this centralized thing that keeps them all sort of under control, but also independently doing their own thing.
And the same with ants in a colony, they're independently going out, they've all got their individual brains, but they're still connected with this system. The system that connects a collection of different animals in a rain forest is a little bit different. It seems much less centralized. In some sense, there is energy flow, obviously through that system, but it's not cohesive in the same sense as a social animal is and maybe that is part of what allows this coexistence of many different niches. It's a good question whether the difference between an ecosystem and a social animal really could be understood as a difference in sort of how much centralized control there is. There's differentiation in both cases, but really far less in something like a social insect colony than in an ecosystem. So yeah, I don't know, that's worth speculating on.
Michael: I think, again, with Luis, I think we got into this about the organism-ecosystem thing. Does it analogize to companies in a city? It seems like companies have this rapid turnover, they die like organisms, and cities persist in a different way that looks more like ecosystems. But anyway, you've given me an hour already. I want to make sure that we get to the NASA Swarmathon before we end this conversation, because here's a really awesome project that you're the Principal Investigator for. Yeah, I mean, you've been bringing the Swarmathon out to InterPlanetary Festival and it's really exciting. And I don't want to talk for you. So yeah.
Melanie: Yeah, so the Swarmathon is a project we ran for the last four years. And the idea was that we wanted to build swarms of robots to revolutionize space exploration. And we also wanted to do this as a way to engage particularly minority underrepresented minority students in a really exciting research project. And I think it was really successful. Our last official competition was last year, although we have sort of some spinoff things happening. It was super fun, first of all. We built about 100 robots total. So these are, I don't know, sort of a-foot-and-a-half cubed robots. We were able to design those. We sent kits out to students. We had about 45 teams total who participated in this. Over the four years, we had about 1500 students involved in building these robots, designing algorithms to have these robots go out and forage like ants.
They had to find little cubes that we scattered around in arenas, eventually hiding them behind vaguely Martian-looking rocks. And the idea was, one, that students would learn about robotics, particularly autonomous robots. So the idea was that students would upload their code to us, we would pretend we were on Mars with their robots, and they didn't get to change it. And they would have to run through days of different competition rounds. So in the rounds, we would actually change the distribution of how these cubes were out in the world so that the students had to develop algorithms that were not just autonomous but also flexible, so they would work well given sort of different kinds of worlds.
We had a fantastic time, it was really exciting to basically crowdsource this question of how should you forage? How should you search for resources in a big space using a swarm of robots? And there was no answer to that question before. We got over the years about 50 different answers to that question. It actually did lead us to some theoretical work that we recently published that showed that of the best algorithms that won this competition, we could demonstrate theoretically which was the most efficient way to search the space in theory, and we could show empirically that theory is often not the right metric to use when you have robots, really — particularly cheap robots built from cheap components running around in a parking lot with variable wind lights and all kinds of other conditions.
And so, weirdly, we were able to kind of engage these students. They had a great time lots of them have gone on to get master's degrees and do internships at NASA, so it was successful in that regard, but they really did contribute to advancing the cutting edge of research in this question of how should you search a large space.
Michael: Some of your more recent publications have been on…like, you have this one out this year, “Distributed adaptive search in T cells: lessons from ants.” Did the Swarmathon stuff make it into that?
Melanie: A little bit. Yeah, the idea that we've really been pursuing here is that search in theory is often different from search in practice. T cells are essentially searching for viruses, pathogens in your body. It's a pretty messy system. They're autonomous agents, and we can think of them just like ants in a colony or robots in a swarm. They're going off, they're searching, they communicate with each other in some cases. Their movement patterns are really important for how quickly did they discover things. But in all three of these cases, the ants, and the T cells, and the robots, noise and error and structural constraints really change what's the right way to search a space.
So yeah, we were actually able to start to draw some lessons. Maybe one of the best ones was early on, our first robots were called iAnts. It was an iPod on a robotic platform and they were really cute. They weren't super functional, but they were fun to work with. We wanted an analogy to ant pheromones. So everybody knows you put the food down in the picnic basket, and a trail of ants comes because they're communicating chemically, the location. We didn't have any chemical communication among our robots. We tried, we failed, to have any kind of chemical communication among our robots. So we just used waypoints. We just used abstract notions of where something was in space and the robots would communicate it. If they found something in a good place the other robots could come there.
It turned out this was a horrible failure. And the reason was that when you communicate an abstract location, if you're lost, then you just mislead all of the other robots, they go to the wrong spot. So it really drove home the importance of embodied communication, which both T cells and ants have, they actually lay a chemical trail. They can't lay the trail where they are not. It is, by definition in the place where they are. They might make a mistake in deciding this is a good place, but they at least are putting the signal in the right place, and anyone who's following their gradient will get to that same place. So it drove us to really look at, what are the mechanisms that are sort of robust to the error that individual cells or ants are going to make in their search process?
Michael: I guess my last question for you would be, how has this research into search changed the way that you search? [Laughs.] How has it changed the way that you handle whether you design for error robustness in your own life?
Melanie: So, I’ll turn the question around. I study search processes because I'm very, very bad at finding things. I can't ever find my keys. I can't find the piece of paper where I wrote down the important thing. Yeah, finding things is not a skill I have. So I was hoping to design systems that could do that for me. Yeah, the main thing is I'll do a little advertisement for Tile, the little tiles that you can attach to your keys, because now I have a sound, it's almost like a pheromone that tells me where my important things are. And I'm now completely reliant on that. And I've given up on sort of the usual random search process that I would use to find my keys.
Michael: So you're becoming more of a liquid brain?
Melanie: Yeah. [Laughs.] Maybe in many bad ways I'm becoming more of a liquid brain. I've also outsourced a lot of my brain to various computer systems. Yeah, that's just one example.
Michael: Or the elephant with a huge backpack, and you're vastly less productive now?
Melanie: Absolutely, yeah. I don't have to carry it around with me. It's out in the cloud. So that's a help.
Michael: Awesome, Melanie, it's been a pleasure talking with you.
Melanie: Thank you so much! I really enjoyed it.