PHP project build system

#1 Pre-requisites

Before reading this document, it is required to read unit testing guidelines, which is part of this package.

#2 PHPUnit, CVS and Phing

For a typical web based software, our main requirements are to automate several routine tasks like unit testing, updating latest data from version control, loading packages on different servers, cleaning up log files and performing other daily routine task to save manual work.

Hence here we will have brief look at how Phing can be used to automate some of the above mentioned tasks. But before that let's see what Phing is and how it works.

#2.1 What is Phing

Phing (PHing Is Not GNU make) is a project build system based on Apache Ant. In the context of PHP, where it is not required to build and compile sources, the intention of Phing is to ease the packaging, deployment, and testing of applications. For these tasks, Phing provides numerous out-of-the-box operation modules ("tasks") and an easy-to-use, object-oriented model for adding our own custom tasks.
Phing can be installed using the PEAR Installer, as shown in the following command line:

$ pear channel-discover pear.phing.info

$ pear install phing/phing

For other modes of installation and more details read Phing documentation.
Note: Please do not forget to install other dependencies also that Phing might ask during it's installation.

#2.2 How it works

Phing uses XML buildfiles that contain a description of the things to do. The buildfile is structured into targets (groups of task/s) that contain the actual commands to perform (e.g. commands to copy a file, delete a directory, perform a DB query, etc.). So, to use Phing, we would first write our buildfile and then run phing specifying the target in buildfile that we want to execute.

$ phing -f [mybuildfile.xml] [mytarget]

By default Phing will look for a buildfile named build.xml (so you don't have to specify the buildfile name unless it is not build.xml) and if no target is specified Phing will try to execute the default target, as specified in the tag.

A valid Phing buildfile has the following basic structure:

1. The document prolog,
2. Exactly one root element called ,
3. Several Phing type elements (i.e. , , etc.),
4. One or more elements containing built-in or user defined Phing task elements (i.e. , , etc).

It is beyond the scope of this document, to explain above structure. Hence please read phing documentation.

#2.3 Phing and automated testing

Phing has several built in tasks to perform various operations. Some of them, for our purpose, are exec, delete, mkdir, coverage-setup, phpunit, phpunitreport and coverage-report.

1. exec task can be used to execute any command of OS environment and applications;
2. delete can be used to remove files/folders;
3. mkdir to make new directories;
4. phpunit to run unit tests created by developers (as described in this document);
5. phpunitreport to generate report about unit tests;
6. coverage-setup and coverage-report tasks are for providing detailed report of code-coverage analysis.

#2.5 Agile documentation

Continuous testing using Phing is performed everyday on test server and 2 types of documentation is generated at following locations for every project;

1. tests/unittests/reports/index.html and
2. tests/unittests/reports/coverage/index.html (for reporting purpose)

Since this document is generated anytime on the fly and gets updated after every run, it doesn't require to add them into version control.

#2.6 Guidelines about using Phing

1. Phing has vast application areas hence in beginning it is used for limited tasks only like updating local repository from CVS, running unit tests, code analysis, packaging and unpacking files etc. Later it will be expanded to handle more complex tasks.
2. Build management system such as Phing, should not be used by individual developers as it affects whole system. Hence it's care is taken by dedicated test/project/development manager.
3. Phing can be extended to create custom tasks also. But that requires extensive study of it's existing functionality.

#3 Links

http://phing.info/docs/guide/current/
http://www.phpunit.de/manual/3.3/en/build-automation.html#build-automation.phing

Unit testing guidelines

#1 Introduction

This document provides detailed information about how to unit test code of PHP based software. Intended audience of this document consists of developers, testing team, project manager and stack-holders.

#2 What is unit testing

Unit testing is a software verification and validation method in which a programmer tests that individual units of source code are fit for use. A unit is the smallest testable part of an application. In procedural programming a unit may be an individual program, function, procedure, etc., while in object-oriented programming, the smallest unit is a class, which may belong to a base/super class, abstract class or derived/child class.

The goal of unit testing is to isolate each part of the program and to show that the individual parts are correct. Unit testing provides a strict, written contract that the piece of code must satisfy. As a result, it affords several benefits. In this article we will follow PHPUnit as a unit testing software.

#2.1 Advantages of unit testing

#2.1.1 Facilitates change

Unit testing allows the programmer to refactor code at a later date, and make sure the module still works correctly (i.e. regression testing). This provides the benefit of encouraging programmers to make changes to the code since it is easy for the programmer to check if the piece is still working properly. Good unit test design produces test cases that cover all paths through the unit with attention paid to loop conditions. In continuous unit testing environments, through the inherent practice of sustained maintenance, unit tests will continue to accurately reflect the intended use of the executable and code in the face of any change. Depending upon established development practices and unit test coverage, up-to-the-second accuracy can be maintained.

#2.1.2 Simplifies integration

Unit testing helps eliminate uncertainty in the units themselves and can be used in a bottom-up testing style approach. By testing the parts of a program first and then testing the sum of its parts, integration testing becomes much easier.

A heavily debated matter exists in assessing the need to perform manual integration testing. While an elaborate hierarchy of unit tests may seem to have achieved integration testing, this presents a false sense of confidence since integration testing evaluates many other objectives that can only be proved through the human factor. Some argue that given a sufficient variety of test automation systems, integration testing by a human test group is unnecessary. Realistically, the actual need will ultimately depend upon the characteristics of the product being developed and its intended uses.

#2.1.3 Documentation

Unit testing provides a sort of "living document". Clients and other developers looking to learn how to use the module can look at the unit tests to determine how to use the module to fit their needs and gain a basic understanding of the API.

Unit test cases embody characteristics that are critical to the success of the unit. These characteristics can indicate appropriate/inappropriate use of a unit as well as negative behaviors that are to be trapped by the unit. A unit test case, in and of itself, documents these critical characteristics, although many software development environments do not rely solely upon code to document the product in development.

Ordinary documentation, on the other hand, is more susceptible to drifting from the implementation of the program and will thus become outdated (e.g. design changes, feature creep, relaxed practices to keep documents up to date).

#2.1.4 Separation of interface from implementation

Because some classes may have references to other classes, testing a class can frequently spill over into testing another class. A common example of this is classes that depend on a database: in order to test the class, the tester often writes code that interacts with the database. This is a mistake, because a unit test should never go outside of its own class boundary. As a result, the software developer abstracts an interface around the database connection, and then implements that interface with their own mock object. This results in loosely coupled code, minimizing dependencies in the system.

#2.2 Limitations of unit testing

Unit testing will not catch every error in the program. By definition, it only tests the functionality of the units themselves. Therefore, it will not catch integration errors, performance problems or any other system-wide issues. In addition, it may not be easy to anticipate all special cases of input the program unit under study may receive in reality. Unit testing is only effective if it is used in conjunction with other software testing activities.

It is unrealistic to test all possible input combinations for any non-trivial piece of software. A unit test can only show the presence of errors; it cannot show the absence of errors. Though these two limitations apply to any form of software test.

#3 PHPUnit

PHPUnit is an excellent unit testing software for PHP programming language based softwares, which is derived from JUnit software (in Java technology). To learn PHPUnit, well understanding of OOP is must.

Testing with PHPUnit is not a totally different activity from what you should already be doing. It is just a different way of doing it. The difference is between testing, that is, checking that your program behaves as expected, and performing a battery of tests, runnable code-fragments that automatically test the correctness of parts (units) of the software. These runnable code-fragments are called unit tests.

It is to be clearly kept in mind that unit testing should be performed as soon as coding is finished, and not after days and month. When software's code gets changed, corresponding tests should also get changed and tested.

#3.1 How PHPUnit works

PHPUnit API has several built in methods to perform various kind of testing on your given code. Developer first write certain code according to requirement of software. Then he will need to create certain test cases (or suits) using PHPUnit's built in classes and finally to test those test cases (or suits) to be sure that designed code works properly.

1. Test cases (or suits) are nothing but PHP scripts that test code of software using PHPUnit's built in tests. Hence there are 3 components in unit testing:
2. Software's code
3. PHPUnit's built in tests and
4. Test cases (or suits).

#3.2 Installing PHPUnit

PHPUnit should be installed using the PEAR Installer. This installer is the backbone of PEAR, which provides a distribution system for PHP packages, and is shipped with every release of PHP since version 4.3.0.

The PEAR channel (pear.phpunit.de) that is used to distribute PHPUnit needs to be registered with the local PEAR environment:

$ pear channel-discover pear.phpunit.de

This has to be done only once. Now the PEAR Installer can be used to install packages from the PHPUnit channel: 

# pear install phpunit/PHPUnit

After the installation you can find the PHPUnit source files inside your local PEAR directory; the path is usually /usr/lib/php/PHPUnit.

Although using the PEAR Installer is the only supported way to install PHPUnit, you can install PHPUnit manually. For manual installation, do the following:

• Download a release archive from http://pear.phpunit.de/get/ and extract it to a directory that is listed in the include_path of your php.ini configuration file.

• Prepare the phpunit script:

Rename the phpunit.php script to phpunit.
Replace the @php_bin@ string in it with the path to your PHP command-line interpreter (usually /usr/bin/php).
Copy it to a directory that is in your path and make it executable (chmod +x phpunit).

• Prepare the PHPUnit/Util/Fileloader.php script:

Replace the @php_bin@ string in it with the path to your PHP command-line interpreter (usually /usr/bin/php).

After successful installation of PHPUnit, you can use command phpunit /PATH/TO/TESTSCRIPT to run your tests. For all the available options of this command type phpunit -–help.

#3.3 How to learn and use PHPUnit

Road map of learning and using PHPUnit has been shown below.

1. Get good understanding of OOP.
2. Read whole documentation of PHPUnit software to understand what it is and how it works.
3. Install PHPUnit to build up testing environment.
4. Study code of all scripts of PHPUnit software even if at first time, you don't understand much.
5. Having look at the testing examples provided by PHPUnit, understand how unit testing works.
6. Now try to build your own test cases of software on which you are working.
7. After having enough experience of building test cases, try to build test suit that can perform all test at once (which is the goal of the unit testing).
8. Try to build more complex and fully automated test suits using PHPUnit's extensions and other utilities or by building your own custom extensions according to requirements of your software.

To create test cases in your live projects, you may require to reorganize your classes into packages and sub packages. For test cases, you may create new folder test/unittest in your project's main directory and then create test cases (scripts) according to packages and sub packages defined above.

#4 Guidelines and principles

Here are guidelines and principles of implementing unit testing to be followed by developer tram.

#4.1 How to write test cases 

1. Best way to write test cases is to study an example given here and some more from PHPUnit documentation.
2. 1st part of test script contains code to include necessary files/fixtures, while 2nd part contain test case class.
3. In beginning develop simple tests for single method or collection of methods. After having certain experience, build more complex test cases. Finally go to suite level which provides fully automated testing environment.
4. Create readable tests - write comments in Asserts and in UT. Write descriptive method names (even very long). Use local variables in Asserts. Use constants. UT should be readable like a book.
5. Test cases must cover at least 75% of code. However this will be shown in reports generated by Phing when such tests are run from there. The danger of not implementing a unit test on every method is that the coverage may be incomplete. Just because we don't test every method explicitly doesn't mean that methods can get away with not being tested.
6. You may require configuration parameters for database, paths and locations of other dependent software to run unit tests. For that there has been created a common file tests/unittests/config.inc.php which would be included in your test cases. Since it is common file or all tests, it is normally managed test/build manager. Hence if it is required to update that file, test manager must be consulted.
7. Moreover this file should be included in your test cases as include_once or require_once constructs only otherwise there will be warnings/errors of duplication of code while running all tests together as suite from build system.
8. If you find difficulties in creating/running tests cases for any of your class method etc., it means it requires regression that is to re-organize structure of class/class package to be able to test them in proper way.
9. Since private methods of class can not be tested directly, it's corresponding public method should be tested. However every private method always has a parent public method which uses that private method. For more information visit Links sections.
10. The concepts that you may need to be consider while building test cases (or suits) are Test-first programming, Code-coverage analysis, Refactoring, Incomplete tests, Agile documentation, Debugging Tests etc.

#4.2 How to organize and run them

1. Developers should maintain hierarchy of classes as well as their equivalent test cases in same way. So for example if class file is grouped as Validate/Numeric.php then it's corresponding test case should also be grouped as Validate/NumericTest.php inside folder tests/unittests. Moreover name of test case class should also follow name of class that is to be tested. Same is true for phpDocumentator tags in test script. That is; tests scripts must be maintained as any other code- like Keep It Simple, Stupid (KISS)
2. Fixtures (such as images, data files etc.) of test cases should be kept in same folder where test case/s reside/s. However for larger contents, they can be put in separate folder.
3. All test cases are to be run from project's base directory only as shown in earlier example. This is necessary because it is required to generate agile documentation of unit tests cases and their classes.

#4.3 Testing instructions

1. Test cases should not be designed in such a way that they require specific order of execution; that is they must run independent of other implementation.
2. Test cases like to test Mail system, Database, File-system etc. requires additional setup apart from test script. Instructions about such setup and dependencies should clearly be mentioned in text file (like README.txt) inside corresponding test case's package. Such instructions can also contain information about how to run tests cases, other prerequisites, dependencies etc. wherever applicable.
3. While overall instructions are to be kept at tests/unittests/README.txt file only.

#4.4 Scope of unit testing

1. The crucial issue in constructing a unit test is scope. If the scope is too narrow, then the tests will be trivial and the objects might pass the tests, but there will be no design of their interactions.
2. Likewise, if the scope is too broad, then there is a high chance that not every component of the new code will get tested. The programmer is then reduced to testing-by-poking-around, which is not an effective test strategy.
3. It is recommended to write test cases for logical part only i.e static code/data need not to get tested. For complex test cases such as for Database, Mailing etc., it may require certain environmental settings. For that, concerned higher positioned person can be contacted for availability of it.

Bottom line of unit testing is that: it all depends upon the test case that you create. The better the cases, that covers all possible areas to be tested, the more worth the test is. If your test is poor, you wont yield advantages of unit testing.

#4.5 What tests should be written

While doing unit testing, it is important to know that what kind of tests should be done. There has been provided some insight about type of some tests.

1. All positive test: This set of tests ensures that everything works as expected.
2. All failure test: Use these tests on a one-by-one basis to ensure that every failure or exception case works.
3. Positive sequence tests: This set of tests ensures that calls in the correct order work as expected.
4. Negative sequence tests: This set of tests ensures that when calls are made out of order, they fail.
5. Load tests: When appropriate, you can perform a small set of tests to determine that the performance of those tests is within an expected range. For example; 500 mails should be sent within 10-15 seconds.
6. Resource tests: These tests ensure that the application program interface (API) properly allocates and frees resources-- for example, opening, writing, and closing a file-based API several times in a row to ensure that no files remain open.
7. Callback tests: For APIs that have callback methods, these tests ensure that the code runs properly if callbacks are not defined. In addition, these tests ensure that the code runs properly when callbacks are defined but behave inappropriately or generate exceptions.

First 4 types of tests are must for every test-case. Rest can be implemented according to requirements in code.

#4.6 Unit testing in the context of team

1. Developers should do unit testing of their developed libraries and dependent libraries only, i.e they don't require to do unit testing of whole library since other code might have other requirement/dependency about which they are not aware of. That means test-suite should be run by Test/Project/Development manager only.
2. Above positioned persons can use build tools like phing to automate unit testing process, whenever there are changes in code, without manually running every time. Procedure of using Phing in unit testing and other tasks has been mentioned in this document.
3. No untested code should be committed to version control, that is each package should have corresponding test cases.

#4.7 Re-usability of test cases/data

1. Although it may seem like a good idea to throw random data at an interface, try to avoid it because the data is hard to debug. If data is generated randomly on each invocation, you may get an error on one pass that you don't get on another. If your test requires random data, generate the data in a file, then use that file on every run. In this way, you can have noisy data, but still be able to debug errors.
2. In unit tests since each test case is tightly attached to particular piece of code, it is difficult to write re-usable test cases. However whenever possible, test cases/data should be made re-usable and hence should be kept in test library in order to re-use them for other projects.
3. To create such library, Version control system can be used to store them in organized way under namespace like unit_test_library. And underneath that, there can be created several pages to describe nature of test case/data and how to use it in other projects. Similar practice can be used for functional test cases.

#5 Links

http://www.phpunit.de/manual/3.3/en/index.html
http://www.acm.org/ubiquity/views/t_burns_1.html
http://www.simplifytheweb.com/files/presentation.ppt
http://readyset.tigris.org/nonav/templates/test-suite.html
http://stackoverflow.com/questions/34571/whats-the-best-way-of-unit-testing-private-methods
http://stackoverflow.com/questions/643898/mocking-methods-used-in-static-methods-phpunit
http://stackoverflow.com/questions/243274/best-practice-unit-testing-abstract-classes
http://www.artima.com/weblogs/viewpost.jsp?thread=203994

Functional testing guidelines

#1 Introduction

This document provides detailed information about how to do functional testing of a web based software. Intended audience of this document consists of testing team, developers, project manager and stack-holders.

Please read document quality assurance guidelines before reading this document.

#2 What is Functional Testing?

Functional testing is a testing strategy, which needs little or no need of internal design or code etc. The types of testing under this strategy are totally based/focused on the testing for requirements and functionality of the work product/software application. Functionality testing (FT) is sometimes also called as Black Box testing, Opaque Testing, Behavioral Testing, Closed Box Testing etc.

The base of the FT strategy lies in the selection of appropriate data as per functionality and testing it against the functional specifications in order to check for normal and abnormal behavior of the software.

In order to implement FT Strategy, the tester is needed to be thorough with the requirement specifications of the system and as a user, should know, how the system should behave in response to the particular action.

FT works on TDD methodology, so It should be created once the testers exercised the code for completed stories (functionalities) to verify that functional requirements had been met. It should be synchronized with functionality of the project.

#2.1 Advantages

1. Tester needs little or no knowledge of implementation, including specific programming languages.
2. Tester and programmer are independent of each other.
3. Tests are done from a user's point of view.
4. Will help to expose any ambiguities or inconsistencies in the specifications.
5. Test cases can be designed as soon as the specifications are complete.

#2.2 Disadvantages

1. Only a small number of possible inputs can actually be tested, to test every possible input stream would take nearly forever.
2. Without clear and concise functional requirement specifications, test cases are hard to design.
3. There may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried.
4. May leave many program paths untested.
5. Cannot be directed toward specific segments of code which may be very complex (and therefore more error prone).

#3 Implementation

For web based software, test cases are first designed into HTML format for reporting, documentation, verification then their corresponding test cases are recorded and executed using selenium software.

Selenium is an open source tool to do automated functional tests for web-based applications. It is based on java scripting to a large extent. It’s simplicity and robustness makes it an excellent candidate for introducing automated functional testing in our project.

While Selenium-IDE operates as a Firefox add-on and provides an easy-to-use interface for developing and running individual test cases or entire test suites. It has a recording feature, which will keep account of user actions as they are performed and store them as a reusable script to play back. It also has a context menu (right-click) integrated with the Firefox browser, which allows the user to pick from a list of assertions and verifications for the selected location. Selenium-IDE also offers full editing of test cases for more precision and control.

Hence there will be 2 types of artifacts for functional test cases:

1. Designed test cases in HTML format.
2. Corresponding Selenium test case.

#4 Guidelines and principles

QA team should follow below guidelines and principles for designing, executing and documenting functional test cases.

#4.1 Designing test cases

1. Test cases are directly mapped Functional Requirement Specification. Hence each FRS should have corresponding test case.
2. Test cases should be written in enough detail that they could be given to a new team member who would be able to quickly start to carry out the tests and find defects.
3. Test cases should be first designed in HTML format as shown in following table. That is; there have to be sections; Unique ID, Purpose, URL, Prerequisites (optional), Test data, Steps, Notes/Question (optional). After that each test case will be recorded and executed using Selenium IDE.
4. The crucial issue in constructing functional test case is selection of range of input data. If range is too narrow, then the tests will be trivial and the objects might pass the tests.

#4.1.1 Test case format

UniqueTestCaseID: Test Case Title
Purpose:	Short sentence or two about the aspect of the system is being tested. If this gets too long, break the test case up or put more information into the feature descriptions.
URL:	URL where test is to be performed.
Prerequisites:	Assumptions that must be met before the test case can be run. e.g., logged in, guest login allowed, user testuser exists.
Test Data:	List of variables and their possible values used in the test case. You can list specific values or describe value ranges. The test case should be performed once for each combination of values. These values are written in set notation, one per line, like; loginID = {Valid loginID, invalid loginID, valid email, invalid email, empty} password = {valid, invalid, empty}
Steps:	Steps to carry out the test. See step formatting rules below. visit LoginPage enter userID enter password click login see the terms of use page click agree radio button at page bottom click submit button see PersonalPage verify that welcome message is correct username
Notes and Questions:	If any

Each step can be written very tersely using the following keywords:

login [as ROLE-OR-USER]

Log into the system with a given user or a user of the given type. Usually only stated explicitly when the test case depends on the permissions of a particular role or involves a workflow between different users.

visit LOCATION

Visit a page or screen. For web applications, LOCATION may be a hyperlink. The location should be a well-known starting point (e.g., the Login screen), drilling down to specific pages should be part of the test.

enter FIELD-NAME [as VALUE] [in SCREEN-LOCATION]

Fill in a named form field. VALUE can be a literal value or the name of a variable defined in the Test Data section. The FIELD-NAME itself can be a variable name when the UI field for that value is clear from context, e.g., enter password.

enter FIELDS

Fill in all fields in a form when their values are clear from context or when their specific values are not important in this test case.

click "LINK-LABEL" [in SCREEN-LOCATION]

Follow a labeled link or press a button. The screen location can be a predefined panel name or English phrase. Predefined panel names are based on GUI class names, master template names, or titles of boxes on the page.

click BUTTON-NAME [in SCREEN-LOCATION]

Press a named button. This step should always be followed by a see step to check the results.

see SCREEN-OR-PAGE

The tester should see the named GUI screen or web page. The general correctness of the page should be testable based on the feature description.

verify CONDITION

The tester should see that the condition has been satisfied. This type of step usually follows a see step at the end of the test case.

verify CONTENT [is VALUE]

The tester should see the named content on the current page, the correct values should be clear from the test data, or given explicitly. This type of step usually follows a see step at the end of the test case.

perform TEST-CASE-NAME

This is like a subroutine call. The tester should perform all the steps of the named test case and then continue on to the next step of this test case.

Every test case must include a verify step at the end so that the expected output is very clear. A test case can have multiple verify steps in the middle or at the end. Having multiple verify steps can be useful if you want a smaller number of long tests rather than a large number of short tests.

#4.2 Recording and Executing test cases

Once test cases are designed, they are be recorded and executed using Selenium. Selenium IDE is the tool to develop Selenium test cases which is an easy-to-use Firefox plug-in and is generally the most efficient way to develop test cases.

While recording and executing Selenium test cases, following checklist is to be kept in mind.

• Links

1. Check that the link takes you to the page it said it would.
2. Ensure to have no orphan pages normally (a page that has no links to it).
3. Are all referenced web sites or email addresses hyperlinked if not generated by image?
4. Check all of your links to other websites.
5. If we have removed some of the pages from our own site, set up a custom 404 page that redirects your visitors to your home page (or a search page) when the user try to access a page that no longer exists.
6. Check all mailto links and whether it reaches properly.

• Forms

1. Acceptance of invalid input.
2. Optional versus mandatory fields.
3. Input longer than field allows.
4. Default values on page load/reload.
5. Is Command Button can be used for hyperLinks and Continue Links?
6. Is all the data inside combo/list box are arranged in required order?
7. Are all of the parts of a table or form present? Correctly laid out? Can you confirm that selected texts are in the right place?
8. Does a scroll-bar appear if required? 

• Data verification and validation

1. Is the Privacy Policy clearly defined and available for user access?
2. At no point of time the system should behave awkwardly when an invalid data is fed.
3. Check to see what happens if a user deletes cookies while in site.
4. Check to see what happens if a user deletes cookies after visiting a site.

• Data integration

1. Check the maximum field lengths to ensure that there are no truncated characters?
2. If numeric fields accept negative values can these be stored correctly on the database and does it make sense for the field to accept negative numbers?
3. If a particular set of data is saved to the database check that each value gets saved fully to the database. (i.e.) Beware of truncation (of strings) and rounding of numeric values.

• Date field checks

1. Assure that leap years are validated correctly & do not cause errors/miscalculations.
2. Assure that Feb. 28, 29, 30 are validated correctly & do not cause errors/ miscalculations.

• Numeric fields

1. Assure that lowest and highest values are handled correctly.
2. Assure that numeric fields with a blank in position 1 are processed or reported as an error.
3. Assure that fields with a blank in the last position are processed or reported as an error an error.
4. Assure that both + and - values are correctly processed.
5. Assure that division by zero does not occur.
6. Include value zero in all calculations.
7. Assure that upper and lower values in ranges are handled correctly.
8. Alphanumeric field checks
9. Use blank and non-blank data.
10. Include lowest and highest values.
11. Include invalid characters & symbols.
12. Include valid characters.
13. Include data items with first position blank.
14. Include data items with last position blank.

#4.3 Organizing test cases

While test cases are successfully designed, recorded and executed they should be stored on file-system to re-run them for regression testing, documentation, demonstration etc. There have been designed certain standards and conventions for that, as shown below;

#4.3.1 HTML test cases

1. HTML test case should be named as ModuleAction.html. For example if module is User and action is Login then test case name would be UserLogin.html. Some more examples are UserAdd.html, ImageAttach.html etc.
2. These test cases are to be organized into single test suite to view them at same time. For that name TestSuite.html is to be used. A test suite document is an organized table of contents for our test cases: it simply lists the names of all test cases that we intend to write. The suite can be organized in several ways. For example, we can list all the system components, and then list test cases under each. Or, we could list major product features, and then list test cases for each of those.
3. Test suite is organized in a grid where the rows are types of business objects or module names and the columns are types of operations or actions. Each cell in the grid lists test cases that test one type of operation on one type of object. Each individual operations/action should also contain hyperlink to corresponding test case document. Format of TestSuite.html should be as follows:

Test cases by modules and actions or business objects and operations
Modules/Actions BOs/Operations	add	list/browse	edit	delete	search
User	UserAdd	UserList	UserEdit	UserDelete	UserSearch
Ad	AdAdd	AdList	AdEdit	AdDelete	AdSearch
Club	ClubAdd	ClubList	ClubEdit	ClubDelete	ClubSearch
Element	ElementAdd	ElementList	ElementEdit	ElementDelete	ElementSearch

Once all these objects are created they should be stored at location; tests/functionaltests/html/ location for concerned project. Other artifacts like documents, instructions etc. also should be put at same location.

Since these objects are for internal purpose; they may not be publicly visible. However separate/sub domain can be created to view them from web if needed.

#4.3.2 Selenium test cases

1. Selenium test case should also be named as ModuleAction.html. For example if module is User and action is Login then test case name would be UserLogin.html. Some more examples are UserAdd.html, ImageAttach.html etc.
2. These test cases are to be organized into single test suite HTML file to re-run them at same time. For that name TestSuite.html is to be used.
3. Once all these objects are created they should be stored at location; tests/functionaltests/selenium/ location for concerned project. Other artifacts like documents, instructions etc. also should be put at same location.
4. Since these objects are for internal purpose; they may not be publicly visible. However separate/sub domain can be created to re-run them from web on other servers if needed.

#4.4 Scope of testing

1. Minimum scope of overall testing procedure is to test all features in all popular browsers and platforms.
2. Stress testing should be performed (try to overload the program with inputs to see where it reaches its maximum capacity), especially with real time systems.
3. Crash testing should be performed to see what it takes to bring the system down.
4. Other scope can be defined according to FRS when test plan is defined.

#4.5 What test should be written

While doing functional testing, it is important to know that what kind of tests should be done. There has been provided some insight about type of some tests.

1. All positive tests: This set of tests ensures that everything works as expected.
2. All failure tests: Use these tests on a one-by-one basis to ensure that every failure or exception case works.
3. Positive sequence tests: This set of tests ensures that calls in the correct order work as expected.
4. Negative sequence tests:This set of tests ensures that when calls are made out of order, they fail.
5. Load tests: When appropriate, you can perform a small set of tests to determine that the performance of those tests is within an expected range. For example; while uploading image, how large file software is able to handle.

First 4 types of tests are must for every test case. Rest can be implemented according to FRS.

#4.6 Functional testing in the context of team

Functional testing team is also part of development team, hence they should be included in every meeting, discussion, planning etc. to make them aware about functionalities are to be developed.

Testing strategy described here is to be performed by dedicated testers only. However standard practice is that developers do functional testing immediately after they develop required functionality. Then testing related to that functionality can be given to dedicated testers to test it from user point of view.

FT team may not always have daily tasks of testing, hence in spare time they can also assist developers in developing new functionality if they wish.

#5 Links

http://www.w3.org/TR/2004/WD-qaframe-test-20040225/
http://seleniumhq.org/
http://seleniumhq.org/projects/ide/