Well everyone has an opinion so why not put mine in the mix.
Yes get rid of the stock header the cat thing is where the two become one. This will allow the bike to breath more efficiently. However once this happens you need to compensate with a richer fuel to air mixture. This is where the tuners and or the dyno questions come into play. For a stock bike, heads cams so on, I would just get a tuner or I believe Harley has the download stage one like for the 96 engines. If memory serves correctly the bike came stock at 14.7 to 1 and the download shoots for 14.1 to 1. With the tuners you can do a little better. Most tuner guys running a dyno will shot for 13.7 to 1 with it being leaner at your cursing speeds to help with mileage. I don’t feel that the cost of a computer access chip, Harley Super Tuner for example, and the cost of the dyno is worth the added performance although it is my opinion. You might gain a few more horse by doing that however the cost total will be around $1100 or better depending on how good your dyno guy is. The stage one download is around $150. I guess it depends on how picky you are and what you expect out of the bike. I gained about 5 horses by doing the dyno however I can’t compare to the download as I never did it then had it tested. This was also with a 96. I have a 2007 FXSTSSE and a 2009 FLTRSE3. I’m very happy with just the new header and the dyno on the Springer, the Road Glide I had a major modification done and it turns 125HP at 126FT/LB. I like the Crusher headers and pipes from Kuryakin. You can get the quite baffles which I did as the stock ones set off car alarms in town.
So now you have a pile more to think about good luck and may your road be ever winding.

 

If you are just wanting some better sound all you need is some slip on mufflers of your choice. Its not necessary to change anything else.

 

First off congrats on the new ride, I am sure you will enjoy it no matter what avenue you choose, and I know you will get many opinions on this matter, I will share my experience with you to further add to your confusion. I just purchased a 2013 Ultra CVO. I know from experience that the CAT creates lots of heat and also restricts the ability for the engine to breath. To validate my resoning I had the bike put on the dyno completely stock, pipes and mufflers. 85 HP, 100 TQ. I purchased a V&H xpipe and left the stock SE slip ons on the bike, I purchased a SE tuner and back on the dyno, 96 HP, 108 TQ. The bike sounds like a Harley, not crazy but has that nice sound to it where anyone knows it is a Harley. I have put 2300 miles since then and it now sounds better as the pipes are burning in. I think the factory tune is mellow and yes the pipe alone probably did not account for all the gain but I beleive it accounted for the majority of it. I know lots of my fellow riders are sold on true duals but unless you are racing your bike you benifit more on the low end grunt from a two into one configuration more as you need high RPMs to take advantage of true duals, the xpipe gives you the performance you want, no cat and you keep your stock mufflers which will run you 4 to 8 hundred depending on the type you get. I hope this helps you in your decision process. Ride safe and enjoy.