Top.Mail.Ru
? ?
Notes from a Medium-Sized Island [entries|archive|friends|userinfo]
Jason

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

[Sep. 3rd, 2013|03:30 pm]
Jason
[Tags|, ]

A thing somebody should do if they are already super familiar with OpenCV or something like that:

Take a movie, capture all faces in all frames. Let a "faceframe" be a frame of the movie scaled and rotated and translated and restricted to a rectangle so that the eyes are in some canonical position and the rectangle is roughly the size of the whole face.

Run a PCA on all faceframes. This gives an axis of maximal variation. Sort all faceframes by their value projected onto this axis. Display that as an animation.

I wonder if you'd actually end up with a very good clustering that tends to put the same person's face (in different orientations) in a contiguous section of the resulting animation. Still would be neat to watch. Maybe you'd want to do a highpass on each frame before the PCA to make sure you don't end up with just a ramp from "darkest ambient lighting" to "lightest ambient lighting".
LinkReply

Comments:
From: queen_elvis
2013-09-03 11:50 pm (UTC)
From what little I know, I think the artificial intelligence community actually has the technology to do this. The result would probably be creepy, though.
(Reply) (Thread)
[User Picture]From: jcreed
2013-09-03 11:51 pm (UTC)
Yeah, OpenCV is one such library that has a passable face-detection algorithm. I am just too lazy to figure out how to do all the necessary plumbing.
(Reply) (Parent) (Thread)
[User Picture]From: cdtwigg
2013-09-07 04:33 pm (UTC)
The "ambient lighting" would at most be a single dimension (an eigenvector of all 1s) which you could remove by normalizing all the images to the same absolute brightness. However, there's a lot more variation in a single face under different lighting conditions and different worldspace orientations than there is between _different_ faces in the same lighting and orientation. Lighting in particular is tricky because shadows and lighting direction makes a huge difference that your brain is automatically factoring out. One attempt to solve this uses multilinear models, though I'm not sure how much I trust their results: http://www.cs.ucla.edu/~dt/papers/eccv02/eccv02.pdf

"Eigenfaces" where the faces have been captured in a canonical orientation and lighting has a pretty long and storied history: http://en.wikipedia.org/wiki/Eigenface
(Reply) (Thread)
[User Picture]From: jcreed
2013-09-07 04:45 pm (UTC)
Oh derp I had totally heard of eigenfaces. Right of course.

Still I'd kind of like to see what it looked like animating it from one end of the principal axis of variation to the other ~ maybe it would look neat seeing a bunch of faces in similar orientation and lighting to each other, flickering by changing whose face it was.

Edited at 2013-09-07 04:45 pm (UTC)
(Reply) (Parent) (Thread)