OC CIO Minutes April 14, 2011

Southern California/Orange County CIO Breakfast Round Table
April 14, 2011 meeting

Present: Joe Desuta, Ashwin Rangan, Jim Sutter, Jeff Hecht, Subbu Murthy, Hicham Semaan, Jennifer Curlee, Dave Phillips

The following is a list of future topics and speakers – note the change in June and July:

5/12/11 Enterprise social networking David Mann
6/9/11 Cloud computing update Jeff Hecht
7/14/11 IT as a service Jennifer Curlee
8/11/11 i-pad and mobility Ashwin Rangan

Topic: The value of QA

Joe Desuta used the Wikipedia definition of QA – the systematic monitoring and evaluation of an activity to improve the probability of attaining a quality product or process, but it does not guarantee the outcome. QA as an IT function is not limited to software - any testing falls under QA. It is hard to quantify the ROI. It is relatively easy to find bugs in the features, but how does one put a value to that? How do you QA performance and/or data errors? Testing takes time and tends to delay the release but it is important when problems with the new release or product affect clients (or worse in a regulated industry), not just internal staff. The value of QA varies significantly, depending on the size and type of company, as does the ratio of tester to developer. Should the test team be part of the development team? Who is responsible for data integrity? The approach to QA is changing, from the traditional waterfall method (which makes a lot of sense when going to a new release of a package), to a more agile method where QA occurs a lot earlier and is part of the development team, sometimes part of the requirements team, where you can define for testability. Testing for security is becoming part of the requirements. The trend in QA is to use script free approaches, and to use testing tools in the cloud, tools which are more supporting of the Agile approach. Joe included the magic quadrant for integrated software quality suites.

We asked those present to share with us their experiences with QA.

Hicham wondered about the performance testing issues of QA. QA adds value by improving the product by finding errors before it is deployed, but is this enough?

Jennifer is a fan of QA but feels that it could add more value if it focused also on improving documentation and training.

Jeff is also a fan of QA but has in the past tended to include that effort as part of the development team.

Ashwin is now part of a heavily regulated industry, where testing is mandatory, with a track and trace requirement. About 10% of the 7,000 employees are in QA. In IT he has a 6-person QA team doing code testing, and an independent IT team involved way ahead in the protocol definition effort.

Jim said that he might be a bit out-of-date on this issue. He was on the board of WaveMaker, which was sold to VMWare and which saw a huge improvement going Agile. The winery tends to not have a separate IT QA group, as quality is not as big an issue as performance. He noted that in the auto industry, defects per 1,000 is used as a measure to quantify quality.

Subbu noted that one could be surprised by defects in things that you would assume were error free. He mentioned that social security numbers are not unique!

Thank you, Joe, for the interesting introduction and the handout. Joe's slides are at: http://www.slideshare.net/occio .