ApTest Manager - Manual software testing management tool

Frequently Asked Questions

Questions that have been asked by people evaluating ApTest Manager.

Answers to General Questions

What type of server does ApTest Manager run on?

ApTest Manager can be hosted on any modern UNIX, Linux, MacOS X, or Windows machine. This server needs to have a web server running on it (Apache, Microsoft IIS), and needs a Perl interpreter (Perl is included with UNIX-based systems, and there is a free one readily available for Windows). With regard to things like memory, processor size, and disk space - it is very hard to estimate this. The software itself does not require much in any of these areas. It all depends on the number of concurrent users, number of test cases and requirements, number of test results, and lots of other factors. As a guideline, consider that our evaluation server, currently hosting hundreds of users with hundreds of test suites and many many thousands of test cases, requirements, and test results, is on a two processor 3GHz Xeon with 2GB of RAM and is using about 2GB of disk space.

What database does ApTest Manager use?

Underlying ApTest Manager is the Berkeley DB from Oracle Corporation, an "open source embeddable database that allows developers to incorporate within their applications a fast, scalable, transactional database engine with industrial grade reliability and availability. As a result, customers and end-users will experience an application that simply works, reliably manages data, can scale under extreme load, but requires no ongoing database administration."

This database is bundled - no additional database software needs to be acquired. As well no user administration of the ApTest Manager database is necessary - database installation, configuration, and maintenance are performed transparently to the user by ApTest Manager.

Optionally, data may also be automatically be mirrored in an external relational database management system so it can be queried with the Structured Query Language (SQL) . Databases supported include SQL Server, MySQL, Postgres, and Oracle.

When we license ApTest Manager do we receive source code?

Yes. ApTest Manager is written in Perl and distributed in source code form. The entire product source code is included, with the exception of a couple of encrypted modules that implement licensing.

ApTest has built a large collection of objects atop its database that provide an abstract layer for accessing ApTest Manager information. These objects and their methods are very efficient, relying upon in memory and on disk caches to speed up access to the most used information. The ApTest Manager GUI is implemented atop this library layer.

There are also a number of scripts included that are also layered on these libraries. The libraries and scripts contain embedded "Perl doc" documentation (a la "javadoc").

We have customers who do modify the source, others who don't but like to be able to refer to it, and others who just feel good about having it.

Note: Perl is an interpreted language. This is why we recommend using the Apache web server. It has a module, named mod-perl, that caches web-based Perl apps and yields performance on a par with compiled languages.

Can I manage my automated testing with ApTest Manager?

Most every QA project entails manual testing. New features, features that change frequently, projects where test automation is too costly or requires too much time to implement; they all call for manual testing. ApTest Manager is unique in providing comprehensive support for manual testing.

As well, you can use ApTest Manager to maintain the specifications for your automated tests, import automated test results, and report on them. Any automated test tool can be used to generate results to be imported into ApTest Manager. Results from automation appear just as if they had been entered manually, so any ApTest Manager report can cover a project with any combination of manual and automated execution and results.

Results can be imported with a utility that comes with ApTest Manager. This script can be run from the command line or as a batch job. It will push your test results into an ApTest Manager Test Session, so for instance your tests could run over night and the results would be there for you in ApTest Manager in the morning.

You could use the script as is, or as a starting point for doing something customized to your needs with ApTest Manager's API, that provides an abstract layer for accessing ApTest Manager's datastore.

In addition to the results importing script (which is obviously not available to users of the eval instance) there is a UI based results importing feature. This is available in the Run subsystem under the Session Management icon with the link Import results into the session. What both the script and UI version do is take a CSV file of results and place them into a Test Session. The script version will also create a session for you automatically.

ApTest Manager does not directly run your automated tests - your existing automation tools will still do that.

I'm trying to figure out how we'll let our testers know that any failed tests have been fixed and are ready for retesting. How about if I add a result code for this?

Yes, that's a great idea, and easily done as ApTest Manager lets you configure your result codes. Further, if you include a link to ApTest Manager in your Problem Reports, your developers can use them to mark an issue as ready for retesting within ApTest Manager.

In this model there would be a a link put into the PR in the bug tracking system to the ApTest Manager run page for the test the PR was filed for. These links can be included automatically in the information ApTest Manager submits when a bug is created. When a bug is fixed the developer would click the link to bring up the run page so they can a) rerun the test to verify the fix and b) change the test result to one indicating it is ready to be verified (and optionally add a comment about the fix). As ApTest Manager keeps a running history of every time a test is run this would record the developer who made the fix, which is a nice info for QA to have.

When QA got a new drop with fixes they could ask ApTest Manager to rerun just the tests with that "ready to retest" result to verify the fixes. Also, if the QA person who originally filed the PR put a link to the bug tracking system into ApTest Manager when they filed it, the QA person verifying the fix could also invoke the bug tracking system to clear the bug there as well.

The ApTest Manager sample Test Suite in the evaluation site has most of this already set up - all you would have to do is add the new result code to the configuration.

Can I manage requirements with ApTest Manager and produce a Requirements Traceability Matrix?

Yes. We provide Requirements Management, including Requirement Traceability and Coverage Reports.

Can I enter information in my local language?

Yes. ApTest Manager supports UTF-8 character encoding, allowing test and requirements information to be entered in local languages (from a localized client system).

Answers to Import/Export Questions

Can existing tests be imported?

Yes. A flexible UI for importing existing tests from applications such as Excel and database programs is included in the product.

Can I import requirements from a Requirements Management tool?

Yes, you can import requirements from CSV files. See for instance Importing and exporting requirements and documents.

How does one export tests (not just the results, but the test case specifications)?

You would generate a Test Specification report using the Customize Report screen to select a CSV (rather than HTML) report be generated.

Answers to Integration Questions

What Revision Control packages does ApTest Manager integrate with?

ApTest Manager can be configured to operate with third party revision control tools that offer a command line interface. Examples are ClearCase, CVS, Perforce, RCS, and Subversion.

If revision control is configured, operations performed on tests by ApTest Manager are also communicated, transparently to the user, to the revision control system. The revision control system's standard UI is used to do things like generate change histories for tests or back out changes to an earlier level.

What Bug Tracking packages does ApTest Manager integrate with?

ApTest Manager can be configured to work with any external bug tracking package that offers a programmatic interface for creating new bugs. We provide example integrations for Bugzilla, JIRA, Mantis, and Test Track Pro. These can be used as-is by people using those tools, or as a starting point for creating integrations with other tools.

Interfaces from CLIs to URLs to SOAP can be supported. If an interface to a bug tracking system is configured, ApTest Manager provides a "Submit a bug report" link that automatically populates a new bug report with information from ApTest Manger about the test that being run and the test environment.

To further integrate the two tools links back to ApTest Manager can be included in the bug report. Links to the bug within the bug tracking package can be stored with test execution results in ApTest Manager. For bug tracking systems with a SOAP interface (e.g. JIRA and Mantis) this is done automatically.

Our on-line evaluation includes an example integration with the Bugzilla bug tracking system that shows some of the powerful things you can do with bug tracking tool integration.

Can graphics files be associated with a test case?

Yes. Any valid HTML, including references to other valid documents, files or web pages, can be included in the information for a test case.

Does ApTest Manager support SSL?

Yes. If your web server is configured to use it and ApTest Manager is accessed via https: instead of http:.

Answers to Licensing Questions

How does licensing work?

You can have as many user accounts, i.e. defined users, as you want. The number of users that can log in simultaneously is limited to the number of licenses you have purchased.

The way licensing works is if a user has no activity for 10 or more minutes the license they were using becomes available to others. If an inactive user later comes back to life the system will automatically require a license for the user. This happens transparently to the user, who is not aware of it unless there is no license available. In that case the user is asked to wait and try again later. That is, licenses are automatically recycled if they are not being used, allowing multiple users to effortlessly share a number of licenses that is less than one per-user.

This also works if a user does not need to contact the server while doing test editing or execution for a sustained period (essentially working locally). Other users can share their license until editing or execution is completed. The user will transparently reacquire a license when next one is needed or, if none are available, have their work held in abeyance until they can retry and complete saving it.

License sharing is normally invisible and license contention only becomes a factor when there are fewer licenses than people wanting to use them. However this would likely be a transient condition that would resolve in short order. You can always add more licenses if the level of contention gets to be a bother :-)

Do you have a scaled back license (in terms of cost) that covers only viewing reports?

We have an access role allows a user to view reports but not do anything else. Such a user would still need a license however, as in our model a license is a license, i.e. there is not a "lite license".

Answers to Performance Questions

My projects are huge, can ApTest Manager support them?

Yes. ApTest Manager is tested by ApTest with Test Suites containing 30,000 Test Cases. We also do automated performance testing using several hundred virtual users.

Can running a test over and over again cause performance to degrade?

The performance of operations that deal with multiple tests and sessions will degrade as the number of tests and sessions they are asked to deal with grows. Otherwise, no - you can have run tests as many times as you like, in as many sessions as you want.

Answers to Reporting Questions

Can Reports be saved in a portable format (e.g. PDF)?

Yes. You can generate a PDF version of any report by using a PDF creation utility (such as PrimoPDF, a free PDF creator).

When I save a report as a pdf I lose the color encoded information.

In Internet Explorer there is an option to print background colors in the Advanced tab of the Internet Options box. In Firefox you can set it under Page Setup.