Create a Hypothesis Like a Data Scientist Pro
Long ago, there was a beautiful Hypothesis. It could predict everything, including the weather, Stock market, and the outcome of World Wars. But, no one believed it. Why? Because the Hypothesis was invalid.
Hypothesis is very cruel in this world. They are put to the test time and again. They are judged. Many Hypotheses will tell you they feel used. They curse their Data scientists parents for forcing them through such rigorous tests. The ones who pass these rigorous tests are called ‘The Validated’.
You will need to test the Hypothesis if you want to become a Data Scientist. You can use your own and those of others to create an amazing Mathematical Model.
What is Hypothesis Testing?
A Hypothesis is a presumption. People who eat junk food and Earth are round are examples of hypotheses. Hypothesis testing is simply the process of designing a Hypothesis test, carrying it out and determining if the Hypothesis is correct.
While the definition may seem trivial, the impacts are important. Take a look at the way the Atomic Theory was discovered. First, they believed that Atom was indivisible. JJ Thompson, who discovered Electron and gave us the Plum Pudding Model, shattered this Hypothesis. Rutherford, then Bohr, gave us the Classical Atomic Model in which Electrons revolve in Peace about a Nucleus. The Quantum Model was finally created by Einstein. Different Hypotheses were eventually proposed over time, and then proved to be correct. Later, they were replaced by incorrect ones.
This is how any field develops. New Theories are created and tested, while the existing ones are rejected. Hypothesis testing and Hypothesis construction are more than a mathematical concept. It is the realization and application of analytical thinking, which is why it is so important for Data Scientists.
Let’s now learn how to form a Hypothesis. It all starts with asking a question that the Hypothesis will answer. For example, how many people would buy soap if it smelled like butterscotch or what an Atom looks like. Next, you collect the information relevant to the question. The information would then be analyzed by a scientist to answer the initial question. This is his Hypothesis.
Let’s look at the Atomic Theory to better understand this. It was the idea of an indivisible particle that scientists and Greeks believed before 1897. This can be called their Null Hypothesis. A Null Hypothesis, unless proven incorrect, is the one that is assumed to be true. Because no one but God was able at that time to split the Atom (and he didn’t tell anyone), the Hypothesis that the Atom was indivisible was presumed to be true.
J.J. Thomson discovered and created the Electron in 1897. He had successfully split the Atom and proved it by his experiment, disproving the Null Hypothesis. J.J. Thompson accepted another Hypothesis. Atom is divisible and looks like Pie. Electrons are trapped inside. This sweet Hypothesis was rejected by J.J. Thompson.
This explains the concept of an Alternate Hypothesis. If your experiment rejects your Null Hypothesis the Alternate Hypothesis will be accepted as true.
Mathematicians and scientists love to write things differently, so the Null Hypothesis is commonly written as H0 and the Alternate Hypothesis as H1.
When forming the Null Hypothesis, be careful. You could prove English is the only language spoken in Texas by conducting a survey. It could have a devastating impact if the promotion boards for Chocolate Brownie are only in English. People from cultures where English is not spoken or chocolate is not eaten might be unable to understand the message.