meh: http://mathb.in/42756

$$f = \sum_t \left|Wx^t - x^{t+1}\right|^2$$
$$= \sum_t \left|\left(\lambda k.\sum_i W_{ki}x^t_i\right) - x^{t+1}\right|^2$$
$$= \sum_t \sum_k \left(\left(\sum_i W_{ki}x^t_i\right) - x^{t+1}_k\right)^2$$

$$0={\partial \over \partial W_{KI}}f = \sum_t 2\left(\left(\sum_i W_{Ki}x^t_i\right) - x^{t+1}_K\right)x^t_I$$

$$\sum_t \left(\sum_i W_{Ki}x^t_i\right)x^t_I = \sum_t x^{t+1}_K x^t_I$$

$$\sum_i W_{Ki}\sum_t x^t_i x^t_I = \sum_t x^{t+1}_K x^t_I$$

$$W_{KI}\sum_t x^t_I x^t_I = \sum_t x^{t+1}_K x^t_I$$

$$W_{KI} = \sum_t x^{t+1}_K x^t_I \big/ \sum_t x^t_I x^t_I$$

---

$$f = \sum_t \left|(s^tW + (1-s^t)V)x^t - x^{t+1}\right|^2$$
$$= \sum_t \left|\left(\lambda k.\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}\right|^2$$
$$= \sum_t \sum_k \left(\left(\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}_k\right)^2$$

$$0={\partial \over \partial W_{KI}}f$$

$$=\sum_t 2\left(\left(\sum_i (s^tW_{Ki} + (1-s^t)V_{Ki})x^t_i\right) - x^{t+1}_K\right) s^t x^t_I$$

$$\sum_t \left(\sum_i (s^tW_{Ki} + (1-s^t)V_{Ki})x^t_i\right)s^tx^t_I = \sum_t x^{t+1}_K s^tx^t_I$$

$$\sum_t \left(\sum_i (s^tW_{Ki}s^tx^t_Ix^t_i + (1-s^t)s^tx^t_IV_{Ki}x^t_i)\right) = \sum_t x^{t+1}_K s^tx^t_I$$

$$W_{KI}\sum_t s^ts^tx^t_Ix^t_I + V_{KI} \sum_t(1-s^t)s^tx^t_Ix^t_I = \sum_t x^{t+1}_K s^tx^t_I$$

---

$$f = \sum_t \left|(s^tW + (1-s^t)V)x^t - x^{t+1}\right|^2$$
$$= \sum_t \left|\left(\lambda k.\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}\right|^2$$
$$= \sum_t \sum_k \left(\left(\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}_k\right)^2$$

$$0={\partial \over \partial V_{KI}}f$$

$$=\sum_t 2\left(\left(\sum_i (s^tW_{Ki} + (1-s^t)V_{Ki})x^t_i\right) - x^{t+1}_K\right) (1-s^t) x^t_I$$

$$\sum_t \left(\sum_i (s^tW_{Ki} + (1-s^t)V_{Ki})x^t_i\right)(1-s^t) x^t_I = \sum_t x^{t+1}_K (1-s^t) x^t_I$$

$$\sum_t \left(\sum_i (s^tW_{Ki}(1-s^t) x^t_Ix^t_i + (1-s^t)(1-s^t) x^t_IV_{Ki}x^t_i)\right) = \sum_t x^{t+1}_K (1-s^t) x^t_I$$

$$W_{KI}\sum_t s^ts^tx^t_Ix^t_I + V_{KI} \sum_t(1-s^t)s^tx^t_Ix^t_I = \sum_t s^tx^t_I x^{t+1}_K$$

$$W_{KI}\sum_t s^t(1-s^t)x^t_Ix^t_I + V_{KI} \sum_t(1-s^t)(1-s^t)x^t_Ix^t_I = \sum_t (1-s^t) x^t_I x^{t+1}_K$$

---
$$f = \sum_t \left|(s^tW + (1-s^t)V)x^t - x^{t+1}\right|^2$$
$$= \sum_t \left|\left(\lambda k.\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}\right|^2$$
$$= \sum_t \sum_k \left(\left(\sum_i (s^tW_{ki} + (1-s^t)V_{ki})x^t_i\right) - x^{t+1}_k\right)^2$$

$$0={\partial \over \partial s^T}f$$

$$= \sum_k 2\left(\left(\sum_i (s^TW_{ki} + (1-s^T)V_{ki})x^T_i\right) - x^{T+1}_k\right) \left(\sum_i (W_{ki} - V_{ki}) x^T_i\right)$$
Let $z_k = W_{kx^T}-V_{kx^T}$

$$= \sum_k \left(\sum_i (s^TW_{ki} + (1-s^T)V_{ki})x^T_i\right) z_k = \sum_k x^{T+1}_k z_k$$

$$= \sum_k \sum_i (s^T (W_{ki}x^T_i z_k - V_{ki}x^T_i z_k) + V_{ki}x^T_i z_k) = \sum_k x^{T+1}_k z_k$$

$$= \sum_k \sum_i (s^T (W_{ki}x^T_i z_k - V_{ki}x^T_i z_k) ) = \sum_k x^{T+1}_k z_k - \sum_i V_{ki}x^T_i z_k$$

$$s^T = {\sum_k z_k (x^{T+1}_k - \sum_i V_{ki}x^T_i ) \over \sum_k z_k\sum_i W_{ki}x^T_i - V_{ki}x^T_i }$$

$$s^T = {\sum_k z_k (x_k^{T+1} - V_{kx^T} ) \over \sum_k z_k z_k }$$
Tags:
• #### (no subject)

A paper on describing circuits in an agda DSL: http://www.staff.science.uu.nl/~swier004/publications/2015-types-draft.pdf

• #### (no subject)

Going more carefully now through this little tutorial on fpga programming with the iCEstick. It's in spanish, which makes it slightly more…

• #### (no subject)

Some further progress cleaning up the https://xkcd.com/1360/ -esque augean stables that is my hard drive. Tomato chicken I made a couple days ago…

• Post a new comment

#### Error

Anonymous comments are disabled in this journal

default userpic