A form of changelog for a site. It may also contain the full text of each page/post/article that appears in it.
What you do with a NewsFeed is subscribe to it using an application called an aggregator. The aggregator will poll your subscribed feeds periodically and let you know where new stuff was posted. (If the full text is provided, you can read it right in the aggregator.) This way, that you can stay up to date with a much larger array of information sources than you'd have the energy or time to poll manually.
Aggregators can take a large variety of forms:
Straw (apt-get install straw),
http://www.freshnews.org/,
jid:wlug@rss.rifetech.com Jabber bot which will give you the headlines from the WlugWiki every half hour,
feed
feed
ScaredBitless, a site that offers a BitTorrent trackers
feed
PlanetGNOME, an site aggregated from GNOME people WebLogs
feed