Re: Automated testing is vastly improved
@Charlie, one of the differences SOASTA points to is that they are using simulated "real" traffic while Keynote uses synthetic traffic. I've looked at Keynote over the years, and at various times the folks that are now SOASTA and the lab I ran were running the largest Mercury Interactive tests ever conducted.
It's easy to sail off into the weeds on the discussion, and there are purposes for which each kind of traffic excels, but in general I prefer "real" traffic if you're doing anything above layer 3 in the ISO stack. Synthetic traffic ("bit blasting") is fine if you're just checking on wire speed or basic switching, but if you're looking at real application delivery, you should be using some sort of real transaction traffic.
And now you know how I spent a lot of my time on the phone with SOASTA...