This paper considers two important aspects of
the probability theory: The maximum entropy method and the Generalized Linear
Model. The probability theory and statistics methodology provide scientists
with a wide range of methods for solving empirical problems, starting from the
simple linear regression model and up to more advanced methods such as Generalized
linear model. Modeling becomes very common tool for economics analysis.
However, such tremendous variety of the methods and models brings even more
questions. Which method should be applied? Which model will represent the
empirical problem the most precisely? The answer on these questions highly
depends on the amount of empirical information that is available and on the
natural constraints of that data. The
main problem with the real-world data and the theoretical modeling is that the
first one highly depends on the uncertainty that should be modeled in the right
way. But how can a researcher model something that is uncertain, obscure
itself? The answer is simultaneously very simple and very complex. Scientist
can represent uncertainty by applying the appropriate distribution. To do so,
we must choose for each parameter a prior distribution as well as a
likelihood function. How such distribution should be chosen? Exploring entropy helps to get an answer.
principle of maximum entropy applies the measure of uncertainty to
the problem of choosing among probability distributions. Entropy provides a way to use as much as all
assumptions known for the researcher about constraints on the outcome variable
for solving the problem. These assumptions
are used to choose the likelihood function that is the most conservative
distribution and the most compatible with the known constraints.
important concept is the Generalized linear model (GLM). This is a good example
of more complex methodology. The choice of the researcher highly depends on the
natural restrictions and if during his work the researcher meet additional
restrictions on the outcome, that is if the outcome variable is restricted to
be discrete or bounded, then the scientist will refer to the GLM method.
Generalized linear model is a model that uses an appropriate link function,
that is determined separately from the distribution. So, for GLM we need to
define both: the likelihood and the link function. GLMs in practice can use any
likelihood function. This returns back to the questions, which likelihood
function should be chosen. In this context, the Principle of maximum entropy
could be a useful instrument while working with the Generalized Linear models.
consists of Introduction, three main sections, conclusion and references. First
section is dedicated to the concept of maximum entropy and its application.
Second chapter discusses different distributions and how we can apply the
method of maximum entropy to choose among them. The third chapter includes
theoretical methodology about the GLMs and how to choose the appropriate
distributions and link functions for them.