I was a naysayer for a long time but after having the adjustable HD backrest on my other bike as soon as I got my next bike it was one of the first mods I did. I can move it back far enough so it's not there and I adjust it while riding so it does not feel like I'm locked into one position. About the only complaint I have is between the Tour-Pac and the backrest it makes getting on and off the bike harder, I'm 5' 10" I guess taller people have less of a problem.
You said you don't have back problems. LUCKY YOU!!! Keep it that way and get a backrest. On barhops with the OL I don't use it but any kind of road trip at all and it's a must have.