When only four years old, Jim Woods became the mascot for the Triple-A baseball Kansas City Blues; and when only eight, the team's batboy and reader of scores on local radio. He attended the University of Missouri for a year before taking a job at KGLO in Mason City, Iowa. In 1939, he replaced Ronald Reagan as the Iowa Hawkeyes football announcer.

He joined the Navy in 1942, where he spent four years as a Chief Petty Officer on the Navy War Bond circuit, working with stars such as Farley Granger, Dennis Day and Victor Mature. After the war he joined WTAD radio in Quincy, Illinois, where he spent two years before moving to Atlanta as an announcer for the Triple-A Atlanta Crackers, replacing Ernie Harwell.

In 1953, he was hired to call New York Yankees games alongside Mel Allen and Red Barber. He was fired after the 1956 season when sponsor Ballantine Beer wanted to make room for former Yankee shortstop Phil Rizzuto. Yankee general manager George Weiss was opposed to this, and told Woods apologetically it was the only time he had had to fire someone for no reason at all.  Woods picked up his nickname of "Possum" while with the Yankees.

In 1957, Woods called both New York Giants games with Russ Hodges and the NBC Game of the Week with Lindsey Nelson and Leo Durocher. The Giants moved to San Francisco after the season without him; Giants owner Horace Stoneham wanted someone who knew the Bay Area to work alongside Hodges.  So in 1958, Woods moved to Pittsburgh as Bob Prince's sidekick on Pirates games, where he achieved his greatest fame.  KDKA was notorious for low pay, unwilling to give him a raise, and he eventually accepted a better offer in St. Louis, where Harry Caray had been abruptly fired after the 1969 season. Jack Buck moved into Caray's No. 1 slot, and Woods took over Buck's spot as No. 2 announcer. He always preferred the subsidiary role because he did not like to do the public relations work that came with being the primary announcer.  Woods and Buck did not get along, however, and Woods left the Cardinals after the 1971 season.

Woods then moved on to the Oakland Athletics, where he called games for the eventual world champions in 1972 and 1973; but A's owner Charlie Finley let him go after only two years because he did not root hard enough for the A's behind the mike.

A few months later, he was hired by the Boston Red Sox as Ned Martin's assistant. On the same day Woods signed his contract with the Red Sox, Finley had second thoughts and asked him to come back to Oakland, but Woods turned him down.  Woods and Martin worked together from 1974 through 1978. By his own admission, Woods toned down his style considerably from his days in Pittsburgh, believing that Red Sox fans preferred their announcers to be more restrained. Despite their popularity, both were fired by the team's flagship radio station, WITS, in the aftermath of Boston's playoff defeat at the hands of Bucky Dent (see below) and the Yankees in 1978 for not cooperating fully with the team's radio sponsors. Martin remained in Boston, moving over to the TV side of Bosox broadcasts. Although he left Martin, Boston and the Red Sox behind, Woods also switched to television. His final play-by-play announcing stint was for The USA Thursday Game of the Week.