Things we learned while running a small beta
engineering, process by sho
Photo by Justin Marty via Creative Commons
Both Mike and I have run beta programs before, but never for a two person effort. In the past, my typical engineering team has been around ten software engineers and ten QA engineers for about 15 months, but I’ve been responsible for teams with as many as 25 software engineers + 25 QA engineers for 2+ years, and as small as 2 engineers + 1 QA.
For Note*spark, we had a team of two engineers + 0 dedicated QA engineers, which is similar to the smallest sized team I’ve ever worked with. But instead of taking a year to develop, we sprinted the whole way, working long hours for about two months to finish the server, web and iPhone pieces.
Once we were done with feature development, we almost decided not to do a beta at all, on the theory that two months of engineering was small enough for us for us to QA ourselves. And besides, we had written a bunch of unit tests, so there probably weren’t that many bugs left, right? Boy, were we wrong! By running a “real” beta, we found a ton of bugs that we never would have found ourselves.
Along the way, we made some decisions about how we thought we should run a small beta. We also learned some things specific to running betas for the iPhone. None of this is rocket science, but I thought I’d post what we learned in case it’s helpful to anyone.
