I just remove the muffler and reinstall the hardware so I can retain the factory look.

Pretty sure that was supposed to be done to ALL of them. Every one of them I've ever needed to take off a drum is like this. You can usually force them off but cleaning up the thread with a thread die is the far better way to do it.

Easy fix. Take the whole thing apart. Take the arm(connected to the muffler) to Lowes and pick up some rubber O-rings. They should be small enough to fit snug around the rivet, but big enough that you can actually get them to slide down the arm. It will be a tight fit on the arm. Once you have them in place, they will never rattle again. I have done this to all my drums that have them with 100% success.

This is great..

I found that both my mufflers & sometimes (grommets) rattle..

Thanks for the help..