미시계량을 위한 스터디 프로젝트 (1) 스터디



사실 대학생(학부생)인 내가 스스로 공부를 한다는 것은 쉬운일이 아니다. 학기중에는 중간 기말고사와 수업에 치이고, 수학적인 백그라운드도 부족해서 논문에 나온 Proof조차 읽기 힘든 게 현실이지만, 그럼에도 해보면 좋겠다고 느껴서 주제를 잡고 학교 공부랑은 좀 동떨어지지만 깊이 공부를 하려고 한다. 관심분야는 미시계량, 특히 소비자 선택과 관련한 계량경제학적 모델링이다. 아마 통계학에서는 Statistical Decision/Choice model 등등으로 부르는 것 같다. 일단 logit에 대한 이해에서 부터 시작하고자 한다.

What is logit model : better than LPM, Prob \( \in\) [0,1] should hold.
$$odds_{i} = \frac{\pi_{i}}{1-\pi_{i}} $$ may be ratio of favorable to unfavorable cases
$$ \eta_{i} = logit(\pi_{i})=log \frac{\pi}{1-\pi} $$: maps from (0,1) to \((-\inf,+\inf) \), (note: \(\frac{1}{2}\) to zero)
$$ \pi_{i} = logit^{-1}(\eta_{i}) = \frac{e^{\eta_{i}}}{1+e^{\eta_{i}}} $$ $$ Y_{i} \sim B(n_{i},\pi_{i}), \; n_{i}=1 \\ $$ $$\bigstar \; logit(\pi_{i}) = X_{i}' \beta $$ $$ \pi_{i} = \frac{exp(X_{i}'\beta)}{1+exp(X_{i}'\beta)} \\ \frac{d\pi_{i}}{dx_{ij}}= \beta_{j}\pi_{i}(1-\pi_{i})$$

Estimation : MLE
$$log L(\beta) = \sum {y_{i} log \pi_{i} + (\eta_{i}-y_{i})log(1-\pi_{i})}$$
** Fisher scoring method **
  Score function related to sensitivity of \( f(y| \theta )\). \( f(y| \theta )\) is i.i.d. and twice differentiable
$$ u(\theta) = \frac{\partial}{\partial \theta} log f(y| \theta) = \frac{1}{f(y| \theta )}\frac{\partial}{\partial \theta}f(y| \theta ) $$
Recall Taylor expansion of the score function, \( V(\theta) \) about \( \theta_{0} \)
$$ V(\theta) \simeq V(\theta_{0}) - \mathcal{J}(\theta_{0})(\theta - \theta_{0}) \\ \mathcal{J}(\theta_{0}) = -\sum_{i=1}^{n} \triangledown^{2}|_{\theta=\theta_{0}} log f(Y_{i}|\theta)$$
\(\mathcal{J}(\theta_{0})\) is observed info at \(\theta_{0}\). Set \( \theta = \theta^{*}, V(\theta^{*})=0 \)
$$\theta^{*} \approx \theta_{0} + \mathcal{J}^{-1}(\theta_{0})V(\theta_{0})\\ \theta_{m+1}=\theta_{m} + \mathcal{J}^{-1}(\theta_{m})V(\theta_{m})$$ under certain regularity condition.
Replace \(\mathcal{J}(\theta_{0})\) to Fisher information \( \mathcal{I}(\theta_{0})\)
Why is it called information? : It would be easier to get true \( \theta \) with large likelihood function's variance...
$$E[\frac{\partial}{\partial \theta} log f(X|\theta)|\theta] = \int \frac{\frac{\partial}{\partial \theta}f(x|\theta)}{f(x|\theta)}\cdot f(x|\theta)dx =\frac{\partial}{\partial \theta}\int f(x|\theta)dx = \frac{\partial}{\partial \theta}1 = 0$$ $$\mathcal{I}(\theta) = E[(\frac{\partial}{\partial \theta}log f(x|\theta))^{2}|\theta] = \int (\frac{\partial}{\partial \theta}log f(x|\theta))^{2} f(x|\theta) dx$$
\( claim) \;\; \mathcal{I}(\theta) = -E[\frac{\partial^{2}}{\partial\theta^{2}}log f(x|\theta) | \theta] \) $$ \because \frac{\partial^{2}}{\partial\theta^{2}}log f(x|\theta) = \frac{\frac{\partial^{2}}{\partial\theta^{2}} f(x|\theta)}{f(x|\theta)}-(\frac{\frac{\partial}{\partial \theta} f(x|\theta)}{f(x|\theta)})^{2} = \frac{\frac{\partial^{2}}{\partial\theta^{2}} f(x|\theta)}{f(x|\theta)} - (\frac{\partial}{\partial \theta}log f(x|\theta) )^{2}$$ $$ and \;\; E\begin{bmatrix}\frac{\frac{\partial^{2}}{\partial\theta^{2}} f(x|\theta)}{f(x|\theta)}|\theta\end{bmatrix} = \frac{\partial^{2}}{\partial\theta^{2}}\int f(x|\theta)dx = 0 \square $$
Therefore, Fisher info can be seen as the curvature of the "support curve" of the log-likelihood function!.
The Fisher scoring method is equivalent to IRLS : iteratively reweightd least square.
note) We usually can not obtain exact solution by \( \triangledown_{\beta}l(\beta|y)=0 \), because of transcendental equation -> no closed form sol!

덧글

댓글 입력 영역


MathJax