This paper considers two important aspects of

the probability theory: The maximum entropy method and the Generalized Linear

Model. The probability theory and statistics methodology provide scientists

with a wide range of methods for solving empirical problems, starting from the

simple linear regression model and up to more advanced methods such as Generalized

linear model. Modeling becomes very common tool for economics analysis.

However, such tremendous variety of the methods and models brings even more

questions. Which method should be applied? Which model will represent the

empirical problem the most precisely? The answer on these questions highly

depends on the amount of empirical information that is available and on the

natural constraints of that data. The

main problem with the real-world data and the theoretical modeling is that the

first one highly depends on the uncertainty that should be modeled in the right

way. But how can a researcher model something that is uncertain, obscure

itself? The answer is simultaneously very simple and very complex. Scientist

can represent uncertainty by applying the appropriate distribution. To do so,

we must choose for each parameter a prior distribution as well as a

likelihood function. How such distribution should be chosen? Exploring entropy helps to get an answer.

The

principle of maximum entropy applies the measure of uncertainty to

the problem of choosing among probability distributions. Entropy provides a way to use as much as all

assumptions known for the researcher about constraints on the outcome variable

for solving the problem. These assumptions

are used to choose the likelihood function that is the most conservative

distribution and the most compatible with the known constraints.

The second

important concept is the Generalized linear model (GLM). This is a good example

of more complex methodology. The choice of the researcher highly depends on the

natural restrictions and if during his work the researcher meet additional

restrictions on the outcome, that is if the outcome variable is restricted to

be discrete or bounded, then the scientist will refer to the GLM method.

Generalized linear model is a model that uses an appropriate link function,

that is determined separately from the distribution. So, for GLM we need to

define both: the likelihood and the link function. GLMs in practice can use any

likelihood function. This returns back to the questions, which likelihood

function should be chosen. In this context, the Principle of maximum entropy

could be a useful instrument while working with the Generalized Linear models.

This paper

consists of Introduction, three main sections, conclusion and references. First

section is dedicated to the concept of maximum entropy and its application.

Second chapter discusses different distributions and how we can apply the

method of maximum entropy to choose among them. The third chapter includes

theoretical methodology about the GLMs and how to choose the appropriate

distributions and link functions for them.