This does have some disadvantages, of course. I have experienced problems with 2.1.x qrunner running away and using all the CPU on occasions (a problem with temporary failures for local delivery being handled by queuing the message for immediate re-delivery). I found that the particular problem I ran into had been fixed on the HEAD branch though, which I am now running on my mail server. It should handle the higher loads though.
If you want to remove some of the intermediate steps you can use my to get mailman to call spamassassin directly. This means that list traffic will only go through MTA, Mailman and Spamassassin. It will also allow mailman to do some moderation decisions based on message scores (ie. discard messages above a particular score without moderation).