Jim Busby was born in Kenedy, Texas on January 8, 1927. He earned a football and track scholarship to attend Texas Christian University in Fort Worth, in the fall of 1944. Freshmen were eligible for varsity sports during World War II, but there were no student deferments from military service. The draft chipped away at the team throughout the season, but the undermanned team won the Southwestern Conference championship and a trip to the Cotton Bowl.

The army claimed Jim soon after football season ended. Stationed at Camp Hood, Texas, he played football and was named most valuable player on the baseball team, suffering the first of several injuries to his right shoulder during his military service.

When he returned to TCU after his discharge in late 1946, the head coach recruited him for the baseball team. Apparently because of his bad shoulder, he played first base in 1947, and was among the conferences leading hitters. When he could find time, he competed for the track team in the 100-yard dash and broad jump, but he played no more football. After he led the conference with a .509 batting average in 1948, Jim signed with the Chicago White Sox. 

The White Sox took a look at him during spring training in 1950. When the season opened he hit .208 with no extra-base hits, and was sent to Triple A Sacramento at the end of May.

Playing in the Pacific Coast League against many former big leaguers, he broke out with a .310 BA. He did impress one opposing manager, Paul Richards of Seattle, who was later hired to manage the White Sox in 1951. Taking over a sixth-place team, Richards handed the center-field job to the speedy Busby. Richards built a roster that emphasized pitching, defense, and speed. Jim put his stamp on the team in the home opener at Comiskey Park and it was the birth of the "Go-Go Sox".

Winning with a staff of no-name pitchers, Richards credited his up-the-middle defense of Jim, shortstop Chico Carrasquel, and second baseman Nellie Fox, all three who made the All-Star team. Jim led the AL in batting during the early weeks, but slumped along with his team in the second half. 

But the White Sox also needed a power bat, and when the 1952 season was less than three weeks old, Jim was traded to Washington. He hit only .244 for the Senators, but he led AL center fielders in putouts. The next two seasons, 1953 and 1954, were the apex of his career, batting .312 and .298. Then he seemingly forgot how to hit and for the rest of his career, he batted .239.  A series of injuries finally caught up with him because he played all-out, all the time. Washington later traded him back to the White Sox for three players in May 1955.

He hit .243 for the Sox as they finished third and he was sent on to Cleveland. Jim played his usual superlative center field, but never got close to hitting .300. His .235 batting average in 1956, was his lowest to that point.

In June 1957, he was traded to Baltimore. Shortly after he joined the club, he took off on a 12-game hitting streak, but ended the season batting just .250 and was his last year as a regular.

In 1958 he was traded to the Boston Red Sox, where he batted .225 and after one game in 1960, was released and went back to the Orioles. At the end of 1961 he was released again and signed with the expansion Houston Colt 45s, as a player-coach.

After ending his playing career in 1962, Busby stayed with Houston as third-base coach. 

Jim Busby died on July 8, 1996, at age 69, in Augusta, Georgia.