Basically, Google takes GPS information transmitted from other phones that do have GPS and corresponds that with nearby cell towers. When your phone connects to a specific cell tower, Google uses this GPS info and relates it to your device. This means it won't be nearly as accurate as GPS. Google is touting accuracy in the 500 to 5,000 meter range (that's up to 3 miles).
The denser the concentration of cell towers, the more accurate user results will be. This means it should work fairly well in cities, but not so well out in more rural areas.
We had a chance to demo this new beta yesterday and it works pretty well. Using a BlackBerry Curve indoors, we were able to pinpoint our location to within two blocks in midtown Manhattan. You simply open up the menu in the Maps application, go to "My Location," and it uses nearby cell tower information to approximate your location. It worked very fast. It generated a locator dot on the satellite image of New York City almost immediately. Sure, GPS would have been more accurate, but two blocks off isn't a big deal.
You can watch a video of how it works here:
Or go here and click on the Mobility Channel.