Swords played on the Holy Cross rugby team for four years. While it is not a varsity team, they competed against some of the top rugby teams in the East. The team had a winning record each of his four years. He led the team to a win over Army, a perennial rugby power in 1982. He continued to play the sport after college and was considered to be the greatest player in U.S. rugby history during his playing days. Swords retired from international rugby competition with 36 caps (a cap being awarded to players on the national rugby team for competition against a national team from another country). He retired as the most capped player in the U.S. rugby history, captained the U.S. World Cup team in 1991, was named to several All World teams, and played on the U.S. National Team from 1985 to 1994.