A mashup of factors could account for the decline in ratings for Jay Leno and David Letterman. In THE NEW YORK TIMES, Bill Carter reports that Leno lost 4% of viewers from last year and Letterman 15%. Meanwhile "Nightline" is up 7%.
One major reason for the fading popularity of the late-night comics is that the genre is getting very long in the tooth. Jack Paar started it. Johnny Carson made it an art. Leno and Letterman added some fresh features. But, in entertainment, a mature show is in a dangerous place. Often it can't be revived, at least not in a sustainable way. The hit "Desperate Housewives" tried gimmicks. Those just wore us out. One of the few shows to be able to have enduring comebacks has been "Nightline." Sure, it had its brushes with stale but could climb out of that.
Another factor is the paradigm shift taking place in America the optimist. Given the three downturns since the beginning of the 21st century, we are becoming darker aka European in our world view. That predisposes us to the serious content of "Nightline."
The third reason is tied to success. Successful entities have a hard time taking risks. They tend to keep trying harder at making the old formulas work. When they finally realize that a turnaround is needed, it's usually too late. Too many viewers have gone elsewhere and established new TV preferences.