Automated testing tools

[Tarby777]

Hi all,

I'm looking for an automated testing tool that I can use for regression testing in my 10.2 app. It's a CHUI system running on a Linux server, so the testing tool could either be on a Windows box running a terminal emulator, or on the Linux box itself. I'd like to be able to record the keystrokes involved in various application functions and play them back to an instance of the app, with a view to finding out whether any code delivered that day has screwed anything up. I'd like the tool to alert me if anything on the screen or in the files that are generated is different to what I got when the keystrokes were recorded.

If anyone has a testing tool working in this way (or has tried and failed), I'd be really interested to hear from you.

TIA
Tarby

 

[GregTomkins]

That is (IMHO and with respect) exactly the wrong way to think about testing. Tests built that way tend to be brittle and way more hassle to maintain than you can ever get a payoff from.

The right way (again, IMHO) is to separate the keystroke-capturing part of your application from the number-crunching part, and build non-UI-dependent tests for the latter. The former should be so simple that you can assume they just work, in the same way that you assume when you hit an 'A' on your keyboard, your OS usually detects an 'A' and not a 'Z'.

There are tons of thoughts on testing (actually, too many!) on Stack Overflow. Look for things like unit vs. integration testing, stubs, mocking, EasyMock, Selenium (which does what you are talking about for web apps).

 

[Tarby777]

Point taken, Greg. Unfortunately though, this app is what it is... 20 years old in places, big, mostly written without much attention paid to separation of UI and logic...

Given that a replacement product is on the way, there would be more work involved in refactoring the current app for easy testing than I can justify. However, your comments give me a timely reminder to make sure that we prioritise "testability" in the new app... building it in rather than trying to retro-fit it

Cheers,
Tarby

 

[TomBascom]

There is actually such a feature available with the character client. You can start a session, record the keystrokes and then play them back. PSC uses (used?) it for QA purposes. It isn't supported and it has a few quirks (like being undocumented...) I haven't used it for a long time and I'm not sure if I can find my notes on it. So this might qualify as teasing... don't hold your breath but I'll see what I can dig up when I get back to the office tomorrow.

 

[Tarby777]

Hey Tom, did you have any success in finding your notes? I tried the Knowledgebase and even logged a support call, but couldn't find anyone in PSC who knew about this feature...

TIA
Tarby

 

[TomBascom]

Yes and no...

The details are very sparse. And it is completely unsupported internal stuff that nobody is likely to acknowledge exists -- you are totally on your own if you want to use it. But here is what I know:

-QA RECORD filename1
-QA PLAYBACK filename1 filename2

"RECORD" will capture all of your keystrokes and so forth and put them in "filename1.evt".

i.e:

mpro dbname -p test.p -QA RECORD test1

"PLAYBACK" will play them back and create the 2nd file:

mpro dbname -p test.p -QA PLAYBACK test1 test2

However...

There is no output during the playback. So if you are hoping to compare UI level things you are going to be disappointed. But any files that get created and so forth will be there.

The date when -QA is in use is hard coded to 12/31/1969 at 19:00.

It is very picky about pre-existing files. It will just crash if the filenames already exist.

There is also a "-QA LOG logname" option but, on my system anyway, it doesn't do anything useful. I can run it as:

mpro dbname -p test.p -QA PLAYBACK test1 test2 -QA LOG logname

but it only creates a zero length log. Other cvariations on the command line mostly lead to core dumps.

So, yes, it could be useful for *some* tasks. In PSC's case they probably have a bunch of tests of specific functionality that they use this for. I can see it being a very handy regression testing tool in that way. But as a general purpose test suite? Probably not so much.

 

[GregTomkins]

That's cool! Thanks for vindicating my choice to skim PT every morning