?

Log in

No account? Create an account
meh: http://mathb.in/42756 $$f = \sum_t \left|Wx^t -… - Notes from a Medium-Sized Island [entries|archive|friends|userinfo]
Jason

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

[Sep. 12th, 2015|05:29 pm]
Jason
[Tags|]

meh: http://mathb.in/42756

$$f = \sum_t \left|Wx^t - x^{t+1}\right|^2$$
$$= \sum_t \left|\left(\lambda k.\sum_i W_{ki}x^t_i\right) - x^{t+1}\right|^2$$
$$= \sum_t \sum_k \left(\left(\sum_i W_{ki}x^t_i\right) - x^{t+1}_k\right)^2$$

$$0={\partial \over \partial W_{KI}}f =
\sum_t 2\left(\left(\sum_i W_{Ki}x^t_i\right) - x^{t+1}_K\right)x^t_I$$

$$ \sum_t \left(\sum_i W_{Ki}x^t_i\right)x^t_I = \sum_t x^{t+1}_K x^t_I$$

$$ \sum_i W_{Ki}\sum_t x^t_i x^t_I = \sum_t x^{t+1}_K x^t_I$$

$$ W_{KI}\sum_t x^t_I x^t_I = \sum_t x^{t+1}_K x^t_I$$

$$ W_{KI} = \sum_t x^{t+1}_K x^t_I \big/ \sum_t x^t_I x^t_I$$

---

$$f = \sum_t \left|(s^tW + (1-s^t)V)x^t - x^{t+1}\right|^2$$
$$= \sum_t \left|\left(\lambda k.\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}\right|^2$$
$$= \sum_t \sum_k \left(\left(\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}_k\right)^2$$

$$0={\partial \over \partial W_{KI}}f$$

$$=\sum_t 2\left(\left(\sum_i (s^tW_{Ki} + (1-s^t)V_{Ki})x^t_i\right) - x^{t+1}_K\right) s^t x^t_I$$

$$ \sum_t \left(\sum_i (s^tW_{Ki} + (1-s^t)V_{Ki})x^t_i\right)s^tx^t_I = \sum_t x^{t+1}_K s^tx^t_I$$

$$ \sum_t \left(\sum_i (s^tW_{Ki}s^tx^t_Ix^t_i + (1-s^t)s^tx^t_IV_{Ki}x^t_i)\right) = \sum_t x^{t+1}_K s^tx^t_I$$


$$ W_{KI}\sum_t s^ts^tx^t_Ix^t_I + V_{KI} \sum_t(1-s^t)s^tx^t_Ix^t_I = \sum_t x^{t+1}_K s^tx^t_I$$

---

$$f = \sum_t \left|(s^tW + (1-s^t)V)x^t - x^{t+1}\right|^2$$
$$= \sum_t \left|\left(\lambda k.\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}\right|^2$$
$$= \sum_t \sum_k \left(\left(\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}_k\right)^2$$

$$0={\partial \over \partial V_{KI}}f$$

$$=\sum_t 2\left(\left(\sum_i (s^tW_{Ki} + (1-s^t)V_{Ki})x^t_i\right) - x^{t+1}_K\right) (1-s^t) x^t_I$$

$$ \sum_t \left(\sum_i (s^tW_{Ki} + (1-s^t)V_{Ki})x^t_i\right)(1-s^t) x^t_I = \sum_t x^{t+1}_K (1-s^t) x^t_I$$

$$ \sum_t \left(\sum_i (s^tW_{Ki}(1-s^t) x^t_Ix^t_i + (1-s^t)(1-s^t) x^t_IV_{Ki}x^t_i)\right) = \sum_t x^{t+1}_K (1-s^t) x^t_I$$


$$ W_{KI}\sum_t s^ts^tx^t_Ix^t_I + V_{KI} \sum_t(1-s^t)s^tx^t_Ix^t_I = \sum_t s^tx^t_I x^{t+1}_K$$

$$ W_{KI}\sum_t s^t(1-s^t)x^t_Ix^t_I + V_{KI} \sum_t(1-s^t)(1-s^t)x^t_Ix^t_I = \sum_t (1-s^t) x^t_I x^{t+1}_K$$

---
$$f = \sum_t \left|(s^tW + (1-s^t)V)x^t - x^{t+1}\right|^2$$
$$= \sum_t \left|\left(\lambda k.\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}\right|^2$$
$$= \sum_t \sum_k \left(\left(\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}_k\right)^2$$

$$0={\partial \over \partial s^T}f$$

$$= \sum_k 2\left(\left(\sum_i (s^TW_{ki} + (1-s^T)V_{ki})x^T_i\right) - x^{T+1}_k\right) \left(\sum_i (W_{ki} - V_{ki}) x^T_i\right)$$
Let $z_k = W_{kx^T}-V_{kx^T}$

$$= \sum_k \left(\sum_i (s^TW_{ki} + (1-s^T)V_{ki})x^T_i\right) z_k = \sum_k x^{T+1}_k z_k$$

$$= \sum_k \sum_i (s^T (W_{ki}x^T_i z_k - V_{ki}x^T_i z_k) + V_{ki}x^T_i z_k) = \sum_k x^{T+1}_k z_k$$

$$= \sum_k \sum_i (s^T (W_{ki}x^T_i z_k - V_{ki}x^T_i z_k) ) = \sum_k x^{T+1}_k z_k - \sum_i V_{ki}x^T_i z_k$$

$$s^T = {\sum_k z_k (x^{T+1}_k - \sum_i V_{ki}x^T_i ) \over \sum_k z_k\sum_i W_{ki}x^T_i - V_{ki}x^T_i }$$



$$s^T = {\sum_k z_k (x_k^{T+1} - V_{kx^T} )
\over \sum_k z_k z_k }$$
LinkReply