I really like the thinking behind the big 3G network test compiled by our old pal Mark Sullivan over at PC World, but I just wish Mark and the team had gone further and asked more questions about how robust the big providers’ 3G networks really are.
The big question that goes unanswered by this and other tests is asking how well these networks are built to handle increasing numbers of users. Will network crunches like the now-famous SXSW debacle become more commonplace as more users turn on their iPhones? Unfortunately, the PC World review seems a little too cautious and perhaps advertiser-sensitive, leaving a feeling of incompleteness after wading through all six pages and the assorted charts.
You can see some nibbling at the edges — in good objective journalistic fashion Sullivan shows that having a lot of bars on your phone means pretty much nothing when it comes to determining actual potential service speed. Though Sullivan and his team found that all the providers pretty much hit their advertised promises of speeds and reliability, he also notes that the terms of service are worded so loosely that the bar isn’t set too high:
Do wireless providers deliver the connection speeds they promise for their 3G networks? In our tests, on average, they did. However, the services promise speeds within a wide range–if they provide a low end to the range at all–due to the wide variability of network performance from day to day and from neighborhood to neighborhood. So in practical terms, these ranges don’t represent much of a commitment to consumers.
On the minus side, it doesn’t appear that Sullivan and his team were able to get any real input from the providers — there is a huge self-serving quote from an AT&T rep that essentially tries to argue that AT&T tests its network itself and everything is just fine, thanks for asking. (And you wonder why the tech press doesn’t trust Ma Bell!) What we would like to see, of course, is more thorough questioning, along the lines of whether or not AT&T and Verizon actually have adequate spectrum to really launch the so-called 4G services they are planning. Maybe in the next study!