We've been looking for an algorithm capable of detecting human activity from UAV (Predator) video.  Once again, thanks to an introduction by Dennis Moellman, we talked at length with Ram Nevatia from the University of Southern California.  Ram has done a lot of work in this area and their site has some great links to the work they've done.