PredPreyGrass
What Is PredPreyGrass?
The Predator-Prey-Grass project studies multi-agent behavior in a simple, closed grid world. It only partially applies to modern humans, but it can shed light on ancestral human behavior: for most of Homo sapiens' history — the hunter-gatherer period, until roughly 10,000 years ago — humans could be predator or prey depending on context, and grass stands in for the limited resources available further down the food chain.
The environment is trained with multi-agent reinforcement learning using Proximal Policy Optimization (PPO). Predators (red) and Prey (blue) both expend energy moving around the grid and replenish it by eating: Prey eat Grass (green), and Predators eat Prey that land on the same cell. In the base configuration, agents get all their energy this way — in real ecosystems, ecological efficiency between trophic levels is closer to 10%, not 100%, but the simplification keeps the environment tractable.
Predators die of starvation when their energy reaches zero; Prey die of starvation or of being eaten. Both reproduce asexually once their energy crosses a threshold, and offspring spawn near their parent. Grass regrows at the same spot after being eaten — it is treated as part of the environment rather than as a learning agent. Predator and Prey are trained independently, each learning movement strategies from partial observations of the grid. In the base configuration, agents are rewarded only for reproducing — a deliberately sparse signal that tests whether the ecosystem can sustain itself on survival and reproduction alone, without any additional reward shaping.
Despite that sparse reward structure — or arguably because of it — training produces concrete, recognizable behavior: predators hunting prey, multiple predators collaborating (and competing) to raise their odds of a catch, predators hovering near grass patches to ambush prey that come to graze, and prey learning to find grass while evading predators. At the population level, trained policies keep the ecosystem running far longer than a random policy does, and population counts settle into a recognizable Lotka-Volterra predator-prey cycle:
That covers the mechanics. For a closer look at the energy economics behind them — why passive decay, not movement or reproduction, is what actually forces the population to compete for a scarce resource — see Energy and Entropy in PredPreyGrass. For why training agents to behave this way is a genuinely hard multi-agent learning problem, not just a bigger single-agent one, see Challenges of Multi-Agent Reinforcement Learning in PredPreyGrass. For how PredPreyGrass relates to other MARL, agent-based ecology, and artificial life research, see Related Research. The rest of this page builds on the mechanics to ask a different question: what crosses the generational boundary between parent and offspring, and what that implies about Darwinian, Baldwinian, and Lamarckian dynamics.
A Concrete Eco-Evolutionary Analogue
PredPreyGrass gives a concrete simulation analogue for the general distinction between Darwinian selection, Baldwinian interaction, and artificial Lamarckian inheritance.
The general distinction is still the same: what matters is what crosses the generational boundary.
In predpreygrass.eco_evolutionary, the active heritable trait is a speed genome. Offspring inherit the parent's speed genome with bounded mutation. The learned PPO policy weights are not part of the inherited genome in the base experiment.
In predpreygrass.eco_evolutionary_cadence, speed controls movement frequency rather than movement distance. The policy can observe both the agent's speed and whether movement is available on the current step. This makes the Baldwinian mechanism especially explicit: inherited speed changes the body-environment interface, learning can condition behavior on that inherited trait, and ecological success determines which genomes reproduce.
A heritable speed genome is morphological evolution — the body changes, not the strategy. The learned policy itself is still shared across the whole species in these experiments. For the precise distinction between that and genuine behavioral (policy) evolution, and why it matters for how to read the trial results below, see What Counts as Evolution in PredPreyGrass?
Interpretation
| Mechanism | What is learned during life | What is inherited | PredPreyGrass interpretation |
|---|---|---|---|
| Baldwinian | Behavior or policy improves through experience. | Parameters that support learning, not the acquired behavior itself. | Best fit for the base eco-evolutionary speed-genome experiments. |
| Lamarckian | Behavior, policy weights, or acquired parameter state changes through experience. | The acquired state itself is copied into descendants or replacement populations. | Closest fit for variants with policy cloning, continued weights, or PBT-style copying. |
| Darwinian selection without learning | No within-lifetime policy learning is required. | Heritable traits vary and are filtered by reproductive success. | Baseline when speed affects movement and fitness, but policy behavior is blind to the genome. |