I've been answering questions on this site for a while now and in the vast majority of cases, my answers have been to questions on the normal queue. I've put answers to questions that have then been put on bounty, and I've noticed that there are a few cases where some 'interesting' interpretations of the question appear.
That said, recently the extinction equilibrium question got a bounty on it. I'd not provided an answer to it previously, but when I noticed it on bounty I have to admit I thought about it just a little harder and came up with an approach that I thought might work. It's currently the highest ranked answer on the question and for what it's worth, I also got a bounty out of it.
So; I'm not saying I'm the best answerer on this site; far from it in fact. That said, I do consider myself a 'senior answerer' these days and a provider of (at least) good quality answers.
Therefore, I'm of the opinion that the bounty works both ways; it attracts attention to be sure, but not all of that is bad attention. It can attract good attention as well, but you as someone who posts a bounty, you're obligated to sort out which is which.
It's already been stated that bounties are more than any other factor a marketing device and I concur with that assessment. The trick is to only offer a bounty if the answer you select:
1) Is the right answer
2) Is of good quality
I'm of the view that a poor answer to a question doesn't deserve a bounty 'just because it's the best of a bad bunch'. That said, that is a decision that the poster of the bounty has to decide for his or herself. Yes, I've won a few bounties in my time, but I'd like to think I won them on merit, rather than a lack of options.
The point of all this background is that I think the answer to this is quite simple; if you post a bounty on a question to draw attention to it, you're not obligated to give that bounty to a poor answer just because it's the best of the poor answers. Doing so only drives the behaviour described in this question and ultimately bounties that are truly earned will lead to more of the 'right' attention for future bounties.
Never having posted a bounty I can't comment on what bounty posters think, but I can say after tracking a number of bounty questions, the added attention does not skew the overall quality of answers provided either to higher or lower quality according to my own sample set.