Log in

No account? Create an account
Here's a cute little decision-theoretic puzzle that I fished out of a… - Notes from a Medium-Sized Island [entries|archive|friends|userinfo]

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

[Apr. 19th, 2010|12:03 pm]
[Tags|, ]

Here's a cute little decision-theoretic puzzle that I fished out of a book:

Suppose you have a starting blob of money, say X dollars. Each day, you choose how to divide your money between consumption and investment. If you consume Y dollars, then you get ln(Y) utility out of that act of consumption. If you invest Z dollars, (really, Z = X - Y; any money you don't spend automatically gets "invested") then you get back R*Z dollars on the next day, where R is a number given to you by the universe, indicating the return you can manage to get on investments. Finally, we make the assumption that you're impatient: the actual value of you today of a util earned far in the future, say on day T, is really only D^T, where D is some "discounting factor", a number less than (but perhaps quite close to) one.

Given that, what strategy optimizes the sum of the present values of all future rewards?

So the book said it was to always consume (1-D)X and to invest DX, which surprised me, because it was totally independent of R!

Suppose the optimal strategy is at least of the above form; always to consume some fixed fraction k of my current money. The answer is ostensibly that k = 1 - D.

Let's abstract away from my logarithmic utility function for a moment and say that spending Y dollars gives me f(Y) utils. Then my total utility function is

f(kX) + D f(kXR(1-k)) + D^2 f(kX(R(1-k))^2) + ... + D^n f(kX(R(1-k))^n)

because immediately I consume kX, but I keep (1-k)X around, which grows to XR(1-k), and then I consume k of that, giving me f(kXR(1-k)) utility, but that all happens tomorrow, so it only counts as D f(kXR(1-k)) to me today. Then on the next day, the XR(1-k)^2 that I still invested becomes XR^2(1-k)^2, and I consume k of it, giving me f(kX(R(1-k))^2) utils, but that's two days into the future, so it really only counts as D^2 f(kX(R(1-k))^2). And so on.

To optimize, I take the k-derivative and set it to zero. Let me just rewrite the above compactly as
sum_n D^n f(kX(R(1-k))^n)
and then what I want to solve for is
0 = sum_n D^n R^n X f'(kX(R(1-k))^n) ((1-k)^n - nk(1-k)^(n-1))
Now if f is ln, then the derivative is just the reciprocal, and things cancel out a lot:
0 = sum_n D^n (1 - k - nk) / (k(1-k))
and we can multiply both sides by k(1-k) to get
0 = sum_n D^n (1 - k - nk)
Now I'm going to use the fact that sum_n D_n = 1/(1-D) and sum_n n D_n = D/(1-D)^2 to get
0 = (1 - k)/(1 - D) - kD/(1-D)^2
0 = (1 - k)(1-D) - kD
and so
k = 1 - D

There you have it! I guess both the logarithmic utility and the exponential discounting of utility over time are pretty sketchy assumptions, but it's interesting it works out this way. It's still completely weird to me that it doesn't depend on R.

[User Picture]From: easwaran
2010-04-19 06:11 pm (UTC)
I was worried that it would end up being trivial - if DR<1 then the optimal strategy would be to consume it all at once, while if DR>1, then there is no optimal strategy but smaller positive values of k give better outcomes. I wonder if the fact that the utility function is logarithmic is relevant here.
(Reply) (Thread)
[User Picture]From: easwaran
2010-04-19 06:13 pm (UTC)
Actually, if we take the log of everything, then we get a linear utility function, and we see that discount is just subtraction and the interest is constant addition. I still don't see why it ends up actually depending on D but not R.
(Reply) (Thread)
[User Picture]From: jcreed
2010-04-19 06:15 pm (UTC)
I thought that at first too, but the discounting factor D itself still hides outside the log.
(Reply) (Parent) (Thread)
[User Picture]From: physics_dude
2010-04-20 06:46 am (UTC)
Hm, I think this problem was deliberately designed to give a simple unintuitive result, since the logarithmic utility is exactly what you need to make the R, X, and k (here k is your strategy vector, you don't need to assume a constant strategy) terms of the total utility function completely independent. So R and X just drop out and you have only to find an optimal k given D. Then, the fact that the optimum investment strategy doesn't change based on X tells you you should do the same thing every day, and the rest is as you have shown.
(Reply) (Thread)
[User Picture]From: techstep
2010-04-21 03:42 pm (UTC)
As far as the assumptions go, the ones you presented are fairly standard in economic models. And really, it looks like the basic infinite-horizon problem one sees (a generalization of the standard intertemporal consumption model).

Exponential discounting is typical -- the question is usually to what extent people discount the future. Though researchers, especially behavior economists, are questioning the constant discounting rate assumption, and are beginning to prefer hyperbolic discounting models.

(One of the things that you lose with hyperbolic discounting, by the way, is the idea of "time consistency" -- in this case, your consumption/savings decision you planned for a future time t is not necessarily the same one you would prefer (and thus carry out) once you hit t. There's been a lot of work done on this, especially with regards to policy rules.)

Log utility functions are also a standard assumption. They're fairly nice because the derivatives are simple, it's a special case of the Cobb-Douglas function (a standard functional form in econ for utility and production), and they're a good and plausible-ish example of a function coming from risk-averse preferences.

This is a really basic model: no borrowing, a single asset, no taxes, no inflation, no exogenous shocks, a constant discount rate, a single agent with log utility, and so forth. Consumption smoothing would dictate that the best strategy would be to consume a fixed amount in each period, saving the rest. It is a little odd, though, that the interest rate wouldn't matter -- naively, I would expect that by consumption smoothing, I would consume more today by a factor of the interest rate, so that I could consume the same amount tomorrow.

(I need to think about this some more -- this is the kind of stuff, actually, that they don't teach in PhD programs (at least at Northwestern's).)
(Reply) (Thread)