Microsoft is a perfect example of this dichotomy. Each new version of a product they release includes tonnes of new features which will be of no use to 80% of users whilst at the same time making the product more difficult and obfuscated to use. Just look at all the toolbar and menu items available in Word for a perfect example.
Whilst radical innovation may include completely new ideas and concepts there is a clear distinction between this and simply adding new features for the sake of maintaining a false sense of progress and momentum. Apple understands this and hence produces products which are orders of magnitude easier to use than anyone else. Everything has clearly been well thought through and is joined up ~ using an Apple product is a pleasure rather than a chore.
At its core innovation is not about adding new features (although may sometimes include this); it’s about creation and improvement through research and experimentation to provide valuable user experiences. Concentrate on the critical core 20% of your service or application and make improvements there before you start bolting on new things (if ever). Careful thinking is always required to exploit new ideas and whilst this is an individual process many people simply don’t seem to get it.
In summary: change is good, change for change’s sake is not. Innovate, don’t obfuscate!