Microsoft's original spin on Silverlight was that it "provides enhanced Web audio and video streaming and playback." That's still front-and-center with Silverlight 4.0, including delivery of DRM-protected content in dedicated devices. So Microsoft is certainly realistic about what people really do with Silverlight -- most of the time it's just being used as a video player. Video playback is a common use for Flash as well, but at least Flash has a decent-sized installed base of non-video-player applications.
<video> tag, even the support-role of video player is threatened.
Ultimately, it seems like Silverlight is destined to become a niche product. Microsoft plans to use it for Windows Phone 7 development, so it's not going to disappear. Yet it's very unlikely that Silverlight will (or should) be used for developing Internet web sites.