As CLECs take advantage of Internet technology to transition to an e-commerce business model, they are facing a growing need to test the web applications that are becoming increasingly critical to their chances for success.
As a result, demand is growing for website application testing tools and services. According to research by the industry analyst firm IDC (www.idc.com), automated software quality (ASQ) tools represent the fastest-growing segment of the application life-cycle management market. IDC expects revenues from ASQ tools to surpass $2.6 billion in 2004.
IDC analyst Dick Heiman says it's critical to look at the entire web-based application because for any enterprise conducting e-commerce, the website is their business.
"It's the kind of thing where failures don't go to the head of the marketing department, they go to the CEO, because it affects the bottom line of the business," Heiman says.
Which means carriers have to ensure that their sites are not only up, but that they are functioning in a manner that will satisfy the customer.
Rob Cohen, vice president of business development for Segue Software Inc. (www.segue.com), says that availability is generally looked at from a server standpoint. If the server is up and running, it's available.
"The problem is that an application can be available, and therefore have 'five 9s' availability, and yet the performance can still be bad enough that you're going to lose customers," Cohen says. "That Internet-savvy person who is dialing in and trying to buy something on the Internet or get service [from a carrier], even if the server is available, if the [website] performance is bad, they're going to try something else."
Segue, which started out in the client-server space, offers a host of products designed to help customers understand what is going on with their web applications.
Segue's SilkPerformer is a load and performance testing product that emulates multiple scenarios a website might face, including widely fluctuating loads, thousands of simultaneous users, sudden bursts of activity, and users accessing the site with a range of browsers and mobile devices, such as phones, handheld organizers and pagers. SilkPilot is a server-oriented product that performs functional and regression testing of CORBA servers implemented in any programming language, as well as pure Java servers through RMI public interfaces. It also supports the Enterprise JavaBeans (EJB) component model. SilkTest is a functional and regression testing product that recognizes the multiple technologies found in e-business applications, including HTML, JavaScript, ActiveX, Java, Windows 98 controls, Visual Basic and C++.
Segue also has a pair of management products. SilkRadar is a defect-tracking tool and SilkRealizer is a test management and scheduling product.
Function and load testing is performed in the lab while performance monitoring is conducted while the site is operational. Regression testing also is performed to ensure that any changes to the application, such as with a software upgrade, don't cause problems on the website. The ultimate goal is to find problems--and fix them--before website performance problems affect the end user.
Mimicking Many Users
Steve Caplow, director of marketing and business development for RSW Software, a division of newly created Empirix Inc. (www.empirix.com), explains that such tests are designed to mimic what the customer would do when accessing a website.
RSW offers e-TEST Suite, which comprises e-TESTER for functional and regression testing, e-Load for load and scalability testing, and e-Monitor for round-the-clock monitoring of deployed applications.
Caplow explains that RSW captures an assortment of web traffic going through a site, turns it into what the company calls a visual script, then clones that script to make it look as if hundreds or thousands of people are going through the same sequence. RSW can capture and clone a dozen different scripts of users going through different types of operations on the site and play those scripts all at once so it looks like a realistic mix of many different users doing different things on the site at the same time.
Jonathan Rende, director of product marketing for Mercury Interactive Corp. (www.mercuryinteractive.com), says his company also runs tests that try to emulate what a website will actually face from the users.
Mercury Interactive offers testing products and monitoring products as packaged software or hosted services.
Using ActiveTest service, Mercury Interactive customers can load-test their websites and, within 72 hours, receive the results and get information that will allow them to pinpoint where any bottlenecks might be. Customers can also subscribe to Mercury Interactive's ActiveWatch monitoring service, which will monitor the customer's website at any interval specified to provide information on how the site is performing and what the performance problems might be.
"It really acts as an early warning system to production-type problems so [the customer] can be notified before end users pick up the phone and call them," Rende says.
Mercury Interactive also offers two other product lines for testing.
Its enterprise product line offers Test Director, which manages the entire testing process, from the gathering of requirements to the tracking of defects; Load Runner, a load testing tool for any kind of application, whether a web, Java, client/server, ERP, CRM or legacy system; and WinRunner, an automated functional testing tool that supports functional or regression testing of any kind of application.
Mercury Interactive's Astra product line offers Astra QuickTest for functional and regression testing; and Astra LoadTest for load testing.
The company also offers a monitoring tool called Topaz, which can detect and diagnose e-business application performance problems once a site has been rolled into production.
Rende explains that there are two types of monitoring. Active monitoring involves creating a dummy user that performs synthetic transactions at scheduled times. The end-to-end transaction performance results can provide an early warning of performance problems. Passive monitoring gathers information on what the actual users are doing on the site and reports on their performance experience.
Keeping It Real
When testing a website, it's important that the tests emulate as closely as possible what is happening in the real world.
Phil Hollows, vice president of technology for RadView Software Inc. (www.radview.com), says that performance doesn't only depend on how the user accesses the applications, but also on how they connect--whether through a 28kbps or 56kbps modem, DSL, cable modem or a wireless device; or on how many people are using export-grade browsers as opposed to 40-bit or 128-bit encryption browsers.
RadView's WebLoad is a performance testing application designed to see how a web application stands up under a variety of conditions. It records a test agenda in JavaScript and allows for a number of user characteristics, including connection speed, browser types, multithreading, and various levels of SSL (secure sockets layer) encryption strength. Used in concert with RadView's Cruise Control, WebLoad can generate and increase the number of virtual users accessing the application until performance levels can no longer be sustained.
WebLoad Resource Manager offers the same features as WebLoad, but it also allows designers and developers to test and analyze the performance and scalability of core components without the need for a web front end.
Hollows says the need for testing and monitoring is increasing as Internet use skyrockets. He points to some of the more recent web events, such as the presidential election in which MSNBC (www.msnbc.com) experienced twice the load on Election Day as at its previous peak--the Air France Concorde crash in July--and then saw that load double again the next day. Analysts suggest that Internet users on average will leave a website that functions improperly or takes more than eight seconds to perform a function.
If website visitors do, however, face performance problems at a website, Omegon Networks Ltd. (www.omegon.com) tools can allow the individual to tap into a help desk to see where the problem is and whether it's related to the network, the website or elsewhere.
According to Roland Aucoin, Omegon Networks' vice president of sales, performance should be considered to be event related. NetAlly's Traffic Agents can be programmed to create the testing event at specific times, or the Test Center can be commanded to create an event.
