I weigh 140 lbs. The bike weighs close to 600 (windshield, saddlebags, sissy bar, luggage rack), and I personally am in 5th at 50 if on a level road at a steady speed. If I want to pass, I'll drop down into 4th to do it. If I simply want to increase speed I'll just simply and slowly roll on the throttle. If the road has a gentle grade, I might be in 4th uphill and 5th downhill at 50 mph. I like being in the highest gear where the engine doesn't lug while slowly rolling on the throttle. Whatever speed that is for ya, go for it! But no, it won't hurt anything to be running 4th gear cruising at 50-55. Fuel economy is really the only thing.
Teek