Wikipedia cites altruism as “an evolutionary enigma”, because under current paradigms it is “evolutionary unstable”.
It means that when an altruistic individual appears in a group for the first time (by genetic mutation), its has a lower probability to pass its genes to the next generation, so altruism should always disappear shortly after appearing: altruism may benefit other members of the group but is detrimental for the altruistic individual itself. Even if altruism genes do spread in the whole group, if a single member evolves a selfishness gene, it will be advantaged by cheating on the other members and the gene for selfishness should take over the whole group.
Diverse models have been built to explain how altruism can have spread through a population, without disappearing from the start or from competition with selfishness. All are evolutionary unstable, so the puzzle is not solved.
Here is my model, and I do believe that it is evolutionary stable. Hopefully I will have time to code a simulation.
Hypothesis I: Vindictive behaviour is a precondition to the formation of societies.
Hypothesis II: A necessary condition for the apparition and continuation of altruistic behaviour is vindictive behaviour.
Hypothesis III: The individual cost of altruistic behaviour must always be balanced by the cost of retribution in case of non-altruistic behaviour.
These are three strong hypotheses… Let me explain what I mean by giving an example: food sharing in wolves. How could this real life behaviour have appeared?
Say you’re a lonely carnivore, ancestor of today’s wolves, but not living in groups. You hunt a prey and start eating, but then some creature comes and steals your food from you. Clearly, if your descendants evolve some genes that make them attack people who try or does steal their food, they will be better off than their naive conspecifics. It is even possible that the same genes that make you attack preys also make you attack other people, or other people’s preys… Maybe are you even one of the thieves that to steal other wolves’ preys in the first place? There is not much difference between a sick rabbit and a freshly killed rabbit, or between your dead rabbit and their dead rabbit… It is difficult to sort out the order in which these related behaviours (hunting, stealing, defending one’s food) appeared, and it is plausible that they all appeared conjointly.
Now say that for some reason, you find yourself stuck with several other pre-wolves on a small area. Maybe the population had a sudden increase in density. Maybe you’re all following the same herbivore migration. Anyway, now several of you have to eat their own prey at relatively short distance from each other. You’re not yet a society, but you do live together (think about today’s bears, who usually live alone or with their cubs but form big groups when it’s salmon season).
The first thing to happen might be that cubs stay closer to their mother, even as young adults, simply because there is not much space. Obviously mums share food with their cubs, but they also protect their cubs when they are eating. If, simply because they live close to each other, this behaviour persists once they are adult, the family will have an obvious evolutionary advantage by protecting each other’s food. They might even team up to steal solitary wolves’ food, or hunting bigger preys. On the other hand, those who don’t even bother to protect their own food don’t stand a chance in this new setting.
At this point, what prevents one member of the pack from cheating? You could eat more than your share, and stay away from battles to avoid danger. That would confer you a big advantage. This is what makes theories of altruism evolutionary unstable. Altruism should not be able to survive cheaters.
… Except if there is retribution. If you tend to take the biggest part of the prey and go away to eat it in peace, it might trigger the thief detector of your colleagues and they will attack you. If you don’t take part to the hunt, you may be considered as an outsider and attacked when you try to eat with the others. The apparition of such vindictive behaviour may not require much genetic change, but it has obvious advantages: it protects the group from cheaters, and it also represents a disadvantage for the cheater, who can be harmed, killed, or just starved as a result of its behaviour. In this group, cheating is the evolutionary unstable behaviour, while cooperation is stable.
But what about altruism? Imagine that instead of hunting all together, some wolves go hunting and then share with the whole pack (maybe because some members have to stay home to protect the cubs). In that case, they must obviously share with those who didn’t go hunting. Maintaining cheaters at bay means insuring that you don’t end up hunting alone while a whole group of lazy adult wolves wait for you to bring food, an easy way to game the system. Being vindictive or resentful is a defence mechanism that should bring the group to punish free riders before reaching that extreme situation.
Meanwhile, altruism should be partly motivated by the fear of social retribution, which is learned, and partly by genetic predispositions. I say that altruism should be learned, because cheating remains beneficial for a given individual, provided that the cheating is not big enough to be caught and punished and behaviours that are beneficial have not reason to disappear from the gene pool; but the punishment threshold depends on the current food resources and the character of other group members so it cannot be genetically encoded. Same goes for vindictive behaviour, which should be proportional to the offence to make evolutionary sense.
A consequence of this theory is that genes for the fear of social retribution should also be evolved, since it prevents the individual to get into too much trouble. At the same time, a race between better cheaters that don’t get caught and those who catch and punish them could also appear. Good cheaters will pass more genes on (and possibly also their tricks as knowledge), but they might also be better at catching members who use the same tricks as them, maintaining balance.
It is possible to game the system by not exhibiting vindictive behaviour. It is costly to monitor and punish cheaters, so you can try to count on others to do it for you and save your energy for more important things. Except of course if this kind of slacking is also punished (just think about all the people who get angry both at what they see as immoral behaviour and at those who refuse to be indignant at such behaviour). Who would have believed it! Vigilantism, self-righteousness, jealousy and charity, sharing, benevolence, all linked together… (I do not endorse vindictive behaviour, by the way.)
This walkthrough can, I think, be applied to most altruistic behaviours. Some howling monkeys give alarm calls when a predator approach the group, which make them more likely to be spotted and killed by said predator. This is a behaviour that is clearly very costly in terms of survival chances. The group can only resist to cheaters if there is a form of punishment that is even more costly (I don’t know if cheaters are punished in these groups of monkeys, but I expect so). The loss caused by altruistic behaviour must always be lower than the cost of retribution to maintain evolutionary stability.
Once it has appeared and found stability, altruistic behaviour can be enforced by other means than retribution, for example by ensuring that the individuals that have the possibility to cheat do not reproduce (like in social bees or mole rats). After all, it is also costly to the group to monitor and punish cheaters…