Testing a Gui-heavy WPF application

Question

We (my colleagues) have a messy 12 y.o. mature app that is GUI-based, and the current plan is to add new dialogs & other GUI in WPF, as well as replace some of the older dialogs in WPF as well. At the same time we wish to be able to test that Monster - GUI automation in a maintainable way. Some challenges:

1. The application is massive.
2. It constantly gains new features.
3. It is being changed around (bug fixes, patches).
4. It has a back end, and a layer in-between. The state of it can get out of whack if you beat it to death.

What we want is:

• Some tool that can automate testing of WPF.
• auto-discovery of what the inputs and the outputs of the dialog are. An old test should still work if you add a label that does nothing. It should fail, however, if you remove a necessary text field. It would be very nice if the test suite was easy to maintain, if it ran and did not break most of the time.
• Every new dialog should be created with testability in mind.

At this point I do not know exactly what I want, so I am marking this as a community wiki. If having to test a huge GUI-based app rings the bell (even if not in WPF), then please share your good, bad and ugly experiences here.

Answer 1

OK, your app sounds large! I can share my experiences around an application we engineered recently; it was a GUI talking web services to a server that in turn contacted multiple databases and other web services. The client base was around 15,000 users… Either way - this is a lot of work no matter how you approach it; the upside is it will help you not chew your nails off each time you make a release!

MVVM

In general I would also recommend the MVVM pattern and do as much testing as possible without the GUI. GUI testing is just plain hard! I like Josh Smith’s article on MSDN: "WPF Apps With The Model-View-ViewModel Design Pattern" (http://msdn.microsoft.com/en-us/magazine/dd419663.aspx)

Test Scripting

The trick with this app was that we had a lot to test, the guts of it were constantly moving and there were (strangely enough) not enough people to get the testing job done for each iteration.

My solution was to come up with a custom testing tool that leveraged existing libraries. We had a simple script engine that read a file and executed commands. In effect we developed a DSL (http://en.wikipedia.org/wiki/Domain-specific_language) for testing our specific application. The DSL included some simple commands to signal what "window" it was testing, any specific "setup" signals and then a series of commands followed by assertions. It looked something like this:

Test NewXyzItem
Setup Clear_items

Press Some_button
Enter_text_into Name Bobby Joe
(...)
Press Save

Assert_db_has_itemX SomeKey

The format of each line is

"command" "argument" [data]

The scripts go into groups of directories and the "test runner" loads them up, parses them and executes them. Creating logs and reports as you go is useful, I got added in hook for making screen-shots etc which came in handy. If you are interested in implementing something likke this and would like a hand let me know.

The handy thing here was that we could make blanket changes to the test strategy.

Writing the scripts becomes pretty simple which is important because you end up with many, many scripts. The controls are discovered by name so you follow a convention (e.g. “Name” may be "NameTextBox" in code, or “Save” could be "SaveButton").

You can actually harness NUnit etc to be your test runner too.

NOTE - Just run the tests interactively, getting GUI test to work with CI is difficult and problematic...

Data and Testing

One major thing here is that the data management was a huge part of the test problem and cannot be overlooked. Our “fresh deployment” was also very long but some parts were external and we had no control over the freshness of the data. The way we handled the cleaning was to provide hooks through the scripting that allowed us to easily remove objects before tests. Not optimal but was rarely an issue.

Libraries

The library that you may find most useful in "White" (http://white.codeplex.com/) It can test windows apps in general – i.e both WPF and WinForms. Essentially you end up coding things like this:

Button button = window.Get<Button>("SaveButton");
button.Click();

If your app makes async calls you will need to come up with a strategy for the test runner to know when the async call is finished, perhaps though the status bar or something. It depend how you hook in…

Again, a lot of work but it’s worth it.

PK :-)