As all have said it WAS very popular.Hear me out on this. Take for example American football, a sport that's popular in America. Every year pretty much every team that makes the playoffs has at least a fair amount of national interest, and the heavy favorites get a lot of interest. Then there are a few teams like the Cowboys that get national interest no matter how bad they are. And every team that doesn't make the playoffs still has a decent amount of interest from their hometown region.
In boxing, at any given time there usually weren't that many boxers that got a lot of interest. It was usually a few heavyweights, a few middle weights, and then an endless list of names no one cared about except for die hard boxing fans. Those few heavyweights and middle weights were popular and a lot of people watched their fights. But with so few boxers that people really cared about at any given time, would you say boxing was never, or usually not popular?
Back in the radios days outside of maybe baseball no sport was more popular.
It wasn't just the names that were famous either. Evreyone knew everyone and knew their stats and could cite their statistics etc