Normally (statistics pun!) for the sampling distribution to be approximately normal we use at least n > 30. But there is a whole sub-field of statistics on sample size determinations: http://en.wikipedia.org/wiki/Sample_size_determination
Your first hint is that the random variable doesn’t take on arbitrarily large values. There are also skewness/kurtosis tests you can perform to see which distribution the sample likely came from. If it came from, for example, a Cauchy distribution (which has an infinite mean), then life becomes harder, but there are still ways to estimate the parameters of the distribution (for example, using maximum likelihood estimators).
From the sampling technique.
Irrelevant. All remaining variables are lumped into an error term. If those variables have significant effects, the variance of the estimators will be extremely large (to the point where they’re not very informative). And often you’re only trying to find the marginal effect of one of the regressors, not produce a fitted value. In that case you only need to account for the variables correlated with the regressor.
Well some variants have stronger/weaker assumptions. Which one you use depends on how well the assumptions hold.
Again, through your sampling technique.
May I suggest that if you’re interested in understanding statistics, read a statistics textbook? Introductory Econometrics: A Modern Approach by Jeffrey M. Wooldridge is very good.
MArginal Interest should take the time and read The Black Swan by Taleb.
If econometrics is so great at telling us things about the real world then I would like you to explain how it overcomes the following. All the talk in this thread has been that econometrics wowrks to tell us information about the relationship between X and Y. Well, what about the other possible variables A,B,C,D,E,F,G,H,I,J,K…? Which variable is ever constant in economics?
Taleb’s main point was that historical data will underestimate variance when the distribution is fat tailed. This can be a problem, but that doesn’t mean that it makes econometrics useless.
As for omitted variables, there are a number of ways to deal with them, although that may not be necessary. They only bias results in specific circumstances. David Hendry, for example, has written about methodical approaches to modelling that attempt to remove such bias by starting with a General Unrestricted Model and slowly removing insignificant variables through misspecification testing.