Introduction

Kitware Inc. is a company that uses an open source development model to produce high quality leading edge software in the fields of computer vision, medical imaging, visualization, 3D data publishing, and software development. They are probably best known in the graphics programming world for the Visualization Toolkit (VTK), and in the programming world more generally for their build tool, CMake.

We were very pleased to speak with Bill Hoffman, Kitware's Vice
President and CTO, who gave us some insight into why his company uses
Squish and the benefits they get from doing so. Bill has a lot of
experience to draw on—he's been doing software testing for almost
twenty years.

Kitware is based in the U.S., and has around seventy employees. The
company develops and maintains open source collaboration platforms,
including the development of software for sale to other businesses, web
applications, and software used in-house. Kitware uses in-house
testing tools written in Tcl and in Python, and for their commercial
customer projects they have started to use Squish, with two
Qt 4-based projects using Squish so far. (Kitware cannot reveal any
more details due to customer confidentiality requirements.)

Kitware use Squish to test their commercial Qt 4 projects on all
the platforms that they deploy on: Windows, Mac OS X, Linux,
and some Unix variants (Sun, HP, Irix).

Why Squish?

Kitware have been testing their software for over a decade, and as Bill
told us:

Things that are not tested do not work!

They chose Squish for their Qt testing because they consider Squish to
be the only viable tool for Qt GUI testing. Kitware haven't yet taken
advantage of the availability of Squish's source code, but they are glad
that it is available. And regarding licensing, they have found Squish's
flexible licensing to be particularly attractive.

Before choosing Squish, Kitware did consider some other possibilities,
including an in-house developed record/play event system that they had
developed for another project, and several screen click/capture
programs. None of these provided the convenience and power that is
available from Squish. Bill told us that key persuaders were how easy
Squish makes it to add a new test, and the versatility Squish provides
in that tests are recorded using a standard language—Kitware use
Python—and can be edited after they've been recorded.

Kitware make great use of the Squish Spy: this is a tool that can access
a running application. The Spy can be used to navigate through all the
application's objects (menus, widgets, items in lists, tables, trees,
and so on), and to list any particular object's properties—such as
whether it is enabled, its text, font, color, and so on. This tool can
also be used to set up verification points so that at particular points
in a test run the value of specified object properties can be evaluated
to see if they are what they ought to be.

Squish in Practice

One feature that Kitware have found especially beneficial is Squish's
robustness in the face of GUI changes. And the ability to write new
tests from scratch or to edit recorded tests all using an off the shelf
language they are familiar with has proved to be a particular boon.

Bill was straight to the point about froglogic's technical support, when
he told us that it is simply "very good".

The main problem that Kitware have encountered is test robustness in the
face of changes to the application's GUI. Squish's Object Map is
designed specifically to help in such situations since it allows test
engineers to modify how application objects are identified—by
adding/changing property values that have changed as a result of GUI
changes—without needing to change the test code that refers to
these objects. Of course, when an application changes so much that
application objects are added or removed, some amount of test editing
may be necessary to account for the changes, but this is usually both
easy and painless.

Many customers prefer property-based verifications (for example,
comparing numeric or string values), and Kitware is no different in
using them. However, Kitware also make considerable use of Squish's
screenshot verification facility where essentially "photographs" of the
application are taken in the course of recording a test and these are
then compared when the test is run again at a later date.

In addition to Squish and various in-house tools, Kitware use a variety
of other tools as part of their testing effort. These include href="http://www.cdash.org/">CDash, Kitware's own web-based software
testing server. In fact, Kitware developed a Squish integration for
CDash, something that they found quite easy to do.

Conclusion

Kitware have found their use of Squish provides much better test
coverage than they had before. They have also found that Squish testing
has had a direct effect on improving the quality of the applications
they develop and deliver. And they have also found froglogic's technical
support to be a genuine help whenever they've needed it.

froglogic's team would like to thank Bill for taking the time to
share his and his team and company's experience with Squish, and we
look forward to a continued successful relationship.

Get your free and supported evaluation copy of Squish at href="http://www.froglogic.com/evaluate">http://www.froglogic.com/evaluate