Probabilistic Decision Making
This is the first post in a series of posts in which we will be going over using probability and statistics to better influence decision making. In this post, we will go over the benefits of using probability to rationalize decisions.
Probabilistic Decision Making in its Simplest Form
Given a bag where we have 6 apples and 4 orange with replacement (placing fruit back in the bag), we can say that the probability of an apple is 60% and the probability of an orange is 40%. So, with just this information, if we were to create some machine learning classifier, the optimal decision would be to always output apple. Why you might ask? Since the probability of an apple is always greater than the probability of an orange, we need to assume that this will hold in the long run. Is there a way to do better? In the long run, we would expect to have an error of 40% which is not that great for such a simple problem. Let’s see if adding more information helps our case.
Expanding on Probabilistic Decision Making using Bayes Theorem
The image above is Bayes Theorem. Just a quick aside, the left hand of the equation should be read as “the probability of A given B.” So what can we do with this equation? Say you have a period where you can feel some characteristics about the fruit that you currently have in your hand. The person who is facilitating this game explained that apples are smooth 80% of the time while oranges are smooth 50% of the time. How can we use this information? Let’s say we grab a fruit and it’s smooth. We want to compare the probability of apple given smooth with the probability of orange given smooth.
Since the probability of apple given smooth is greater than the probability of orange given smooth, we will select apple if the object is smooth. Now, what if the fruit we are touching is not smooth?
Since the probability of orange given not smooth is greater than the probability of apple given not smooth, the better output would be orange.
So, Where Do We Go From Here?
This is just the start of probabilistic decision making, even with this simple model, we are still yet to discuss evaluating how well this model performs (calculating the error of the model). If you all would like me to go over Bayes theorem and conditional probability, let me know! These posts are intended to build a community around making machine learning/statistical learning more understandable to non-computer scientists. There is also an abundance of other problems including deciding which features are relevant, estimating densities, and supervised and unsupervised learning. My goal is to teach these concepts in the simplest way possible while only taking up 5 minutes of your day, and I hope you stick around for the journey. If anything doesn’t make sense, leave a comment! I hope that all of you, including myself, will learn and grow from these blogs.
