Born in Tulsa, Oklahoma, but raised in Los Angeles, Swartz was a left-handed relief pitcher who appeared in five games for the St. Louis Browns in 1947. While in high school, he led his team to a city championship, posting a record of 7-0. The son of a minor league catcher, he was courted by a number of big league teams, but he signed with the Browns after the team agreed that he could make his major league debut in his first professional season. The team honored its commitment, and so, on July 12, 1947, at the age of 18, Swartz pitched a scoreless ninth inning against the New York Yankees. His final four appearances that season were far rockier, and he was sent back to the minor leagues, never again to resurface at the major league level. In his only big league at-bat, however, he singled off Vic Raschi.