I’m in love with statistics. Knowing statistics has changed how I view the world and it’s often hard for me to convey this to people because statistics has been tragically misrepresented to the public. Most people think that statistics is a subset of math but I believe that, fundamentally, statistics is a philosophy class and I wager that if it were sold as that, it would be much more popular.
At it’s core, statistics is an epistimology (the philosophy of knowledge) that happens to use math as it’s language. It’s about probing the nature of certainty and doubt, understanding the power of knowing and the limits of knowledge.
Let me give a simple example: Your friend has a coin which is either fairly weighted or weighted to land Heads 80% of the time. You observe a series of coin tosses and it comes down HTHHHTHTHHTTTHTHTHHHH. What does this tell you about either hypothesis? What does this new knowledge now allow you to infer about the nature of the world around you? Notice that certainty is impossible, no matter what sequence of coin tosses you observe, it’s possible for it to be generated by either hypothesis. The knowledge you are gaining is inherently probabilistic, inherently statistical.
How does each additional coin toss influence your beliefs? How many coin tosses are required for you to have any useful knowledge? If you have less than that number, what is the nature of your belief? All of these questions are deeply philosophical but they cannot be answered without an analytical toolkit.
Understanding statistics rewired my brain, made my see everything in the world around me in a different light. It was a mental augmentation that made me a quantum leap smarter. But I’m not going to lie, statistics also kicked my ass. I rarely struggle to master anything but the first statistics class I took, I got a 68/100 and came out of it unimpressed. I came into statistics like I did any other math class and I focused on learning statistics as a skill to be mastered. And the work was challenging enough that I never thought to look for the bigger picture, to look for a mental framework to fit it all under. As a result, I could grind out the calculations and know what the result was but the understanding was not there. It wasn’t until I took statistics again in Graduate School and had some background in what I was learning that I started to see the underlying roots of statistics.
I think the way statistics is taught now has had a profoundly detrimental impact on how it has been applied. There are some that argue that the recent financial crisis is fundamentally rooted in financial quants who were only interested in applying statistical tools without being fully aware of the nature of what they were doing.
If statistics had been described to me as a philosophy class, I would have come in much more aware of the conceptual side of it rather than merely focusing on the tools and techniques. I would have understood it as a way of thinking. The problem is, you can’t at the same time divorce statistics from the math. Without the mathematical rigor, statistics is an empty husk. Philosophy majors take philosophy precisely to get away from math and Engineering/Science majors took their subjects to get away from the wishy washy abstract thinking of philosophy. It’s hard to find people who have an affinity to both and when you only have one semester to get through as much materiel as possible, covering the philosophical side is going to severely limit how deeply you can dive into the material.
Still, after speaking to a friend who revealed to me her choice of major hinged solely on not having to take a statistics course, I wonder if things had been different if she knew it was all about philosophy?