Yes, you need to remove the boards/pegs from the frame so the arm and the pegs go with the respective rest. I swapped my boards with another HDForum member last year, worked out great!
Well, I'm currently not looking to swap them, yet. But, I'm 5'9" and when I ride with the boards on, I got tired of them hitting my calfs in stoplight traffic.Wife rarely rides with me because we two young kids, so I took them off. It's only one allen bolt that holds the whole board/bracket assembly on the bike, so I was wondering about the FLHX pegs for 90% of the time. When she does decideto go with me on long rides, it should be an easy swap out.
I am riding with a buddy tomorrow that rides an FLHX. I'm gonna check outthe mounting bracket on them.