Surprises are always around the corner

by with No Comment Learnings & Experiences

An industry leader providing end-to-end solutions to automate Foreign Exchange trading, our customer provides innovative solutions for financial institutions. Their flagship product, the online FOREX trader, connects to the various Trading Gateways across the world using Adapters. That is no small task. We’re talking millions of transactions at sub 200 ms response time.
When we were called to develop a automation suite for one its components, we didn’t expect anything challenging. Boy, were we in for a surprise or what?

An important middleware Component called as Adapter that links the FX Inside to the Provider. Different providers have their own Adapters. The real work of the Adapter was to direct a homogeneous data sent by the Client, while trading into heterogonous environment to provider and vice versa. These Adapters have to be tested for every release of the core applications. This is a backend non-UI programs that requires scripts to be written to test the functionality at API level.

The objective was to develop automation suite which can be used to test multiple adapters on both Simulator and Live setup. Automation suite should be flexible enough to cater to test new adapters that will be added in the future with minimal changes.

For that we interacted with the developers to understand the functionality of the Adapters and finally we developed a framework which would cater to automating multiple adapters and also add new adapters in the future.

The team started the incremental approach towards Automation of the Adapters by first interacting with the development and QA team, gathering the necessary information by which the common scenarios across the adapters were identified. The critical part of Automation was to develop scripts that can Automatically Restart the Adapters residing on a remote Linux box, send across trading messages to the adapter component, receive them by listening to messaging broker and parse the necessary information.

The result was much better than we anticipated. The execution time of the test scenarios for one Adapter taking two days earlier was reduced to thirty minutes for both live and Simulator environments, which was phenomenal for the client.

STAG developed a test suite to automate tests for every adapter at API level, thus, bringing down System testing effort by 40%.

Smart Test Automation to check product functionality cuts test execution time enabling faster market release

by with No Comment Learnings & Experiences

STAG Software was working on a dashboard product aimed at the mobile telecommunications industry. It was being developed on the LAMP platform, which is a solution stack of free, open source comprising the Linux (operating system), Apache HTTP Server, MySQL (database software), and either Perl, PHP or Python.

The major user interface (UI) component of the product, which was the management UI, had the facility to configure key components, configure handsets, user management (create, modify and delete), upload audio/video clips for video on-demand and live viewing, pinning channels for streaming, display status of streaming servers, streaming sessions, assets as well as generating reports for asset inventory and streaming activity.

The scope of the project and range of features dictated that the project would not only be development intensive, but post-development there would also be an equally intensive testing and debugging stage.

STAG automated the execution of a number of product feature test cases.

400 FUNCTIONALITY TEST CASES AUTOMATED

As some of the product features reached stability, STAG automated the execution of their test cases. Validation of UI-based features was automated using IBM Rational Functional Tester (RFT). The non UI- based server-side features and the validation of the product installation process was automated using Perl.

RFT enabled to automate 400 functionality test cases out of a total of 600 test cases for the management UI. A data driven framework was developed with the ability to take input data for test cases from an Excel sheet. 400+ test cases were managed by developing a catalog of around 40 reusable library functions and 22 main test scripts. These same test scripts could be executed on multiple browsers i.e. Internet Explorer and Mozilla Firefox, which also enabled considerable time and effort savings. Moreover, some of the libraries developed could be used as project assets.

FASTER TIME-TO-MARKET, COST SAVINGS

Benefits of automating the test cases were:

  • Test execution effort was brought down from 17 persons and machine hours to 7 machine hours
  • 42 person days effort was taken to design, develop and test the scripts, which was considerably shorter then anticipated
  • The testing team could focus more on other components/test cases, where manual intervention was essential
  • Cost savings
  • Faster time-to-market

——————
This case study was published in the IBM’s “The Great Mind Challenge for Business, Vol 2, 2011”. . The book recognizes visionary clients who have successfully implemented IBM software (RFT) solutions to create exceptional business value.

Low cost automation challenge

by with No Comment Learnings & Experiences

A New Zealand based customer in the heath care domain embarked on a journey of migrating their Delphi-based products to Microsoft technologies. The products use specialized GUI controls that are not recognized by the typically popular tools. The company was keen to embark on automation right from the early stage of migration. And the budget to develop automation was tight.

We conducted Proof-Of-Concept (POC) to identify the tool that would support automation for both Delphi and VB.Net. We discovered that most popular tools were indeed not compatible with the developed product. The POC concluded that Test Complete did support both Delphi and VB.Net with a few constraints. It was very cost effective however but not user friendly. We convinced the management of our decision. The project started off with us identifying test cases which could be automated. Seven modules were automated and demonstrated.

We developed reusable Keyword Driven Framework for the client. Both individual test case execution and batch run was possible just by choosing the test cases. STAG provided detailed demo of the framework to the in-house QA team.

However some of the test cases chosen for automation were not complete. We validated the test cases, made the necessary changes and then initiated the scripting. The automation work was divided between the STAG and customers team. As we automated the test cases, we guided and trained the customer’s team to automate.

The result – By automating 326 test scenarios, the testing time was cut down from 80 hours to 12 hours! We saved the customer significant money spent on the tool, but more by enabling them to release the product to market ahead of schedule!

We demystified the automation puzzle. Relentless validation tamed!

by with No Comment Learnings & Experiences

A large global provider of BI solutions has a product suite that runs on five platforms supporting thirteen languages with each platform suite requiring multiple machines to deliver the BI solution. The entire multi-platform suite is released on single CD multiple times a year.

The problem that stumped them was “how to automate the final-install validation of multi-platform distributed product”. They had automated the testing of the individual components using SilkTest, but they were challenged with “how to unify this and run off a central console on various platforms at the same time”.

Considering each platform-combination took about a day, this required approximate two months of final installation build validation, and by the time they were done with this release, the next release was waiting! This was a relentless exercise, consuming significant QA bandwidth and time, and did not allow the team to do things more interesting or important.

The senior Management wanted single-push-button automation -identify what platform combination to schedule next, allocate machine automatically from the server farm, install and configure automatically, fire the appropriate Silk scripts and monitor progress to significantly reduce time and cost by lowering QA bandwidth involved in this effort. After deep analysis, in-house QA team decided this was a fairly complex automation puzzle and required a specialist! This is when where we were brought in.

After an intense deep-dive lasting about four weeks, we came up with a custom master-slave based test infrastructure architecture that allowed a central console to schedule various jobs onto the slaves, utilizing a custom developed control & monitoring protocol. The solution was built using Java-Swing, Perl, Expect and adapters to handle Silk scripts. Some parts of the solution where on Windows platform while some on UNIX. This custom infrastructure allowed for scheduling parallel test runs, automatic allocation of machines from a server farm, installing appropriate components on appropriate machines, configuring them and finally monitoring the progress of validation through a web console.

This test infrastructure enabled a significant reduction of the multi-platform configuration validation. The effort reduced from eight weeks to three weeks. We enjoyed this work simply because it was indeed a boutique work fraught with quite a few challenges. We believe that this was possible because we analyzed the challenging problem from wearing a development hat and not the functional test automation hat.