There are two types of ideas which appear very similar yet behave very differently and the ability to distingush can be very useful. Legitimate ideas are not neccesarily ideas that are right but their distinguishing factor is that they have an adequate response to all developed criticism. That means one of the hallmark properties of a legitimate idea is that if you’ve heard a criticism against it, that criticism is probably flawed. What’s more, the mere attempt at trying to demolish it looks foolish and woefully ignorant. If a legitimate idea is demolished, it’ll be demolished from the top, by the people who are most intimately familiar with it. Evolution, I believe is a legitimate idea. All attempts to debunk evolution merely reveal the debunker’s lack of understanding of evolution.

Cargo cult ideas are those which have all the outward trappings of a legitimate idea but without the social process that causes those trappings. It survives not because it can survive criticism, but because it carefully prevents legitimate criticism from affecting it. Cargo cult ideas can survive for a surprisingly long time despite the presence of arguments against it because their very survival involves aping legitimate ideas so closely that it can use the same refrain: “if you’ve heard a criticism against it, that criticism is probably flawed”. Cargo cult ideas are undermined from the bottom up rather that from the top down as it involves a loss of faith in the system.

The structure of argument is so different between legitimate and cargo cult ideas that it becomes impossible to argue unless both people are on the same side. You can see this tension play out with the atheist vs Christian arguments where atheists take the side that Religion is a cargo cult idea masquarading as a legitimate one. Christians on the other hand feel safe dismissing atheist arguments because well, even though they personally don’t know the rebuttal to the argument, they’re sure some learned Christian scholar surely does. Why, this athiest simply doesn’t understand the full subtlety and intricacy of the Christian position and all they are doing is revealing their ignorance of it.

How do you distinguish between the two? There are some useful zero knowledge proofs but they’re tricky because ultimately, the goal of a cargo cult idea is to become indistinguishable. I think in order to do so, you need to learn enough about a subject to reach the cliff. The cliff is the point where a simple question, well stated will be rebuffed rather than answered. No good answer will be forthcoming. The problem with this is twofold: It’s impossible to assert that no answer can be found unless you’ve read all of the literature on the problem and it’s impossible to ever assert the cliff does not exist unless you reach the end of the field. Neither of these are practical goals so only probabilistic measures are possible.