Functional testing guidelines

#1 Introduction

This document provides detailed information about how to do functional testing of a web based software. Intended audience of this document consists of testing team, developers, project manager and stack-holders.

Please read document quality assurance guidelines before reading this document.

#2 What is Functional Testing?

Functional testing is a testing strategy, which needs little or no need of internal design or code etc. The types of testing under this strategy are totally based/focused on the testing for requirements and functionality of the work product/software application. Functionality testing (FT) is sometimes also called as Black Box testing, Opaque Testing, Behavioral Testing, Closed Box Testing etc.

The base of the FT strategy lies in the selection of appropriate data as per functionality and testing it against the functional specifications in order to check for normal and abnormal behavior of the software.

In order to implement FT Strategy, the tester is needed to be thorough with the requirement specifications of the system and as a user, should know, how the system should behave in response to the particular action.

FT works on TDD methodology, so It should be created once the testers exercised the code for completed stories (functionalities) to verify that functional requirements had been met. It should be synchronized with functionality of the project.

#2.1 Advantages

1. Tester needs little or no knowledge of implementation, including specific programming languages.
2. Tester and programmer are independent of each other.
3. Tests are done from a user's point of view.
4. Will help to expose any ambiguities or inconsistencies in the specifications.
5. Test cases can be designed as soon as the specifications are complete.

#2.2 Disadvantages

1. Only a small number of possible inputs can actually be tested, to test every possible input stream would take nearly forever.
2. Without clear and concise functional requirement specifications, test cases are hard to design.
3. There may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried.
4. May leave many program paths untested.
5. Cannot be directed toward specific segments of code which may be very complex (and therefore more error prone).

#3 Implementation

For web based software, test cases are first designed into HTML format for reporting, documentation, verification then their corresponding test cases are recorded and executed using selenium software.

Selenium is an open source tool to do automated functional tests for web-based applications. It is based on java scripting to a large extent. It’s simplicity and robustness makes it an excellent candidate for introducing automated functional testing in our project.

While Selenium-IDE operates as a Firefox add-on and provides an easy-to-use interface for developing and running individual test cases or entire test suites. It has a recording feature, which will keep account of user actions as they are performed and store them as a reusable script to play back. It also has a context menu (right-click) integrated with the Firefox browser, which allows the user to pick from a list of assertions and verifications for the selected location. Selenium-IDE also offers full editing of test cases for more precision and control.

Hence there will be 2 types of artifacts for functional test cases:

1. Designed test cases in HTML format.
2. Corresponding Selenium test case.

#4 Guidelines and principles

QA team should follow below guidelines and principles for designing, executing and documenting functional test cases.

#4.1 Designing test cases

1. Test cases are directly mapped Functional Requirement Specification. Hence each FRS should have corresponding test case.
2. Test cases should be written in enough detail that they could be given to a new team member who would be able to quickly start to carry out the tests and find defects.
3. Test cases should be first designed in HTML format as shown in following table. That is; there have to be sections; Unique ID, Purpose, URL, Prerequisites (optional), Test data, Steps, Notes/Question (optional). After that each test case will be recorded and executed using Selenium IDE.
4. The crucial issue in constructing functional test case is selection of range of input data. If range is too narrow, then the tests will be trivial and the objects might pass the tests.

#4.1.1 Test case format

UniqueTestCaseID: Test Case Title
Purpose:	Short sentence or two about the aspect of the system is being tested. If this gets too long, break the test case up or put more information into the feature descriptions.
URL:	URL where test is to be performed.
Prerequisites:	Assumptions that must be met before the test case can be run. e.g., logged in, guest login allowed, user testuser exists.
Test Data:	List of variables and their possible values used in the test case. You can list specific values or describe value ranges. The test case should be performed once for each combination of values. These values are written in set notation, one per line, like; loginID = {Valid loginID, invalid loginID, valid email, invalid email, empty} password = {valid, invalid, empty}
Steps:	Steps to carry out the test. See step formatting rules below. visit LoginPage enter userID enter password click login see the terms of use page click agree radio button at page bottom click submit button see PersonalPage verify that welcome message is correct username
Notes and Questions:	If any

Each step can be written very tersely using the following keywords:

login [as ROLE-OR-USER]

Log into the system with a given user or a user of the given type. Usually only stated explicitly when the test case depends on the permissions of a particular role or involves a workflow between different users.

visit LOCATION

Visit a page or screen. For web applications, LOCATION may be a hyperlink. The location should be a well-known starting point (e.g., the Login screen), drilling down to specific pages should be part of the test.

enter FIELD-NAME [as VALUE] [in SCREEN-LOCATION]

Fill in a named form field. VALUE can be a literal value or the name of a variable defined in the Test Data section. The FIELD-NAME itself can be a variable name when the UI field for that value is clear from context, e.g., enter password.

enter FIELDS

Fill in all fields in a form when their values are clear from context or when their specific values are not important in this test case.

click "LINK-LABEL" [in SCREEN-LOCATION]

Follow a labeled link or press a button. The screen location can be a predefined panel name or English phrase. Predefined panel names are based on GUI class names, master template names, or titles of boxes on the page.

click BUTTON-NAME [in SCREEN-LOCATION]

Press a named button. This step should always be followed by a see step to check the results.

see SCREEN-OR-PAGE

The tester should see the named GUI screen or web page. The general correctness of the page should be testable based on the feature description.

verify CONDITION

The tester should see that the condition has been satisfied. This type of step usually follows a see step at the end of the test case.

verify CONTENT [is VALUE]

The tester should see the named content on the current page, the correct values should be clear from the test data, or given explicitly. This type of step usually follows a see step at the end of the test case.

perform TEST-CASE-NAME

This is like a subroutine call. The tester should perform all the steps of the named test case and then continue on to the next step of this test case.

Every test case must include a verify step at the end so that the expected output is very clear. A test case can have multiple verify steps in the middle or at the end. Having multiple verify steps can be useful if you want a smaller number of long tests rather than a large number of short tests.

#4.2 Recording and Executing test cases

Once test cases are designed, they are be recorded and executed using Selenium. Selenium IDE is the tool to develop Selenium test cases which is an easy-to-use Firefox plug-in and is generally the most efficient way to develop test cases.

While recording and executing Selenium test cases, following checklist is to be kept in mind.

• Links

1. Check that the link takes you to the page it said it would.
2. Ensure to have no orphan pages normally (a page that has no links to it).
3. Are all referenced web sites or email addresses hyperlinked if not generated by image?
4. Check all of your links to other websites.
5. If we have removed some of the pages from our own site, set up a custom 404 page that redirects your visitors to your home page (or a search page) when the user try to access a page that no longer exists.
6. Check all mailto links and whether it reaches properly.

• Forms

1. Acceptance of invalid input.
2. Optional versus mandatory fields.
3. Input longer than field allows.
4. Default values on page load/reload.
5. Is Command Button can be used for hyperLinks and Continue Links?
6. Is all the data inside combo/list box are arranged in required order?
7. Are all of the parts of a table or form present? Correctly laid out? Can you confirm that selected texts are in the right place?
8. Does a scroll-bar appear if required? 

• Data verification and validation

1. Is the Privacy Policy clearly defined and available for user access?
2. At no point of time the system should behave awkwardly when an invalid data is fed.
3. Check to see what happens if a user deletes cookies while in site.
4. Check to see what happens if a user deletes cookies after visiting a site.

• Data integration

1. Check the maximum field lengths to ensure that there are no truncated characters?
2. If numeric fields accept negative values can these be stored correctly on the database and does it make sense for the field to accept negative numbers?
3. If a particular set of data is saved to the database check that each value gets saved fully to the database. (i.e.) Beware of truncation (of strings) and rounding of numeric values.

• Date field checks

1. Assure that leap years are validated correctly & do not cause errors/miscalculations.
2. Assure that Feb. 28, 29, 30 are validated correctly & do not cause errors/ miscalculations.

• Numeric fields

1. Assure that lowest and highest values are handled correctly.
2. Assure that numeric fields with a blank in position 1 are processed or reported as an error.
3. Assure that fields with a blank in the last position are processed or reported as an error an error.
4. Assure that both + and - values are correctly processed.
5. Assure that division by zero does not occur.
6. Include value zero in all calculations.
7. Assure that upper and lower values in ranges are handled correctly.
8. Alphanumeric field checks
9. Use blank and non-blank data.
10. Include lowest and highest values.
11. Include invalid characters & symbols.
12. Include valid characters.
13. Include data items with first position blank.
14. Include data items with last position blank.

#4.3 Organizing test cases

While test cases are successfully designed, recorded and executed they should be stored on file-system to re-run them for regression testing, documentation, demonstration etc. There have been designed certain standards and conventions for that, as shown below;

#4.3.1 HTML test cases

1. HTML test case should be named as ModuleAction.html. For example if module is User and action is Login then test case name would be UserLogin.html. Some more examples are UserAdd.html, ImageAttach.html etc.
2. These test cases are to be organized into single test suite to view them at same time. For that name TestSuite.html is to be used. A test suite document is an organized table of contents for our test cases: it simply lists the names of all test cases that we intend to write. The suite can be organized in several ways. For example, we can list all the system components, and then list test cases under each. Or, we could list major product features, and then list test cases for each of those.
3. Test suite is organized in a grid where the rows are types of business objects or module names and the columns are types of operations or actions. Each cell in the grid lists test cases that test one type of operation on one type of object. Each individual operations/action should also contain hyperlink to corresponding test case document. Format of TestSuite.html should be as follows:

Test cases by modules and actions or business objects and operations
Modules/Actions BOs/Operations	add	list/browse	edit	delete	search
User	UserAdd	UserList	UserEdit	UserDelete	UserSearch
Ad	AdAdd	AdList	AdEdit	AdDelete	AdSearch
Club	ClubAdd	ClubList	ClubEdit	ClubDelete	ClubSearch
Element	ElementAdd	ElementList	ElementEdit	ElementDelete	ElementSearch

Once all these objects are created they should be stored at location; tests/functionaltests/html/ location for concerned project. Other artifacts like documents, instructions etc. also should be put at same location.

Since these objects are for internal purpose; they may not be publicly visible. However separate/sub domain can be created to view them from web if needed.

#4.3.2 Selenium test cases

1. Selenium test case should also be named as ModuleAction.html. For example if module is User and action is Login then test case name would be UserLogin.html. Some more examples are UserAdd.html, ImageAttach.html etc.
2. These test cases are to be organized into single test suite HTML file to re-run them at same time. For that name TestSuite.html is to be used.
3. Once all these objects are created they should be stored at location; tests/functionaltests/selenium/ location for concerned project. Other artifacts like documents, instructions etc. also should be put at same location.
4. Since these objects are for internal purpose; they may not be publicly visible. However separate/sub domain can be created to re-run them from web on other servers if needed.

#4.4 Scope of testing

1. Minimum scope of overall testing procedure is to test all features in all popular browsers and platforms.
2. Stress testing should be performed (try to overload the program with inputs to see where it reaches its maximum capacity), especially with real time systems.
3. Crash testing should be performed to see what it takes to bring the system down.
4. Other scope can be defined according to FRS when test plan is defined.

#4.5 What test should be written

While doing functional testing, it is important to know that what kind of tests should be done. There has been provided some insight about type of some tests.

1. All positive tests: This set of tests ensures that everything works as expected.
2. All failure tests: Use these tests on a one-by-one basis to ensure that every failure or exception case works.
3. Positive sequence tests: This set of tests ensures that calls in the correct order work as expected.
4. Negative sequence tests:This set of tests ensures that when calls are made out of order, they fail.
5. Load tests: When appropriate, you can perform a small set of tests to determine that the performance of those tests is within an expected range. For example; while uploading image, how large file software is able to handle.

First 4 types of tests are must for every test case. Rest can be implemented according to FRS.

#4.6 Functional testing in the context of team

Functional testing team is also part of development team, hence they should be included in every meeting, discussion, planning etc. to make them aware about functionalities are to be developed.

Testing strategy described here is to be performed by dedicated testers only. However standard practice is that developers do functional testing immediately after they develop required functionality. Then testing related to that functionality can be given to dedicated testers to test it from user point of view.

FT team may not always have daily tasks of testing, hence in spare time they can also assist developers in developing new functionality if they wish.

#5 Links

http://www.w3.org/TR/2004/WD-qaframe-test-20040225/
http://seleniumhq.org/
http://seleniumhq.org/projects/ide/

SEO guidelines

#1 About this document

This document aims to defines generic search engine optimization requirements for various projects.

At this moment this document contains general guidelines of SEO. In future, at the time of taking training session, this document will be expanded further in order to be used as perfect resource for almost all SEO requirements.

#2 General requirements

#2.1 Server location

The server should be located in same country from where it will be mostly accessed. Moreover If the service will have it’s own domain then it should reside on a dedicated server. Wildcard DNS should not be allowed as well as all sub domains, if any, should be activated separately.

#2.2 Robots.txt

The robots.txt file is to be placed in the root (value of DocumentRoot directive in case web server is Apache) directory of the software. It should allow the search engines to crawl all directories where information related to various entities will be shown.

Personal pages such as listing owner's entities, posting/editing entities that require login should be blocked. Most search engines now a days are able to find out this behavior hence you may omit such entries into robots.txt file.

#2.3 Encoding

If there are will be used special characters in language of website they will need to get encoded in URLs (maybe using PHP function like urlencode()) and Filenames using UTF-8 encoding. For full documentation of encoding of such characters, please visit http://www1.tip.nl/~t876506/utf8tbl.html). As practically all browsers supports the Unicode UTF-8 standard, it should not be important to encode the characters in the actual content. The suitable HTML entities can be taken from this address: http://leftlogic.com/lounge/articles/entity-lookup/ anyhow.

There should be a 301 redirect from any page with special characters in the URL where someone writes the URL using the special characters and not the encoded ones if that user has a browser that over writes the UTF-8 character set with some other character set. See how Wikipedia functions for an example. This prevents links with the wrong character set to be used on external pages.

#2.4 Header responses

#2.4.1 Page not found (404 error)

Entities that are removed from database/software should not be shown. When someone accesses the removed listings page the server should respond with a 404 header response (and not a 200 response) and show an error message (or optionally a separate page) saying that the entity is already deleted/expired/sold etc. Furthermore the relevant listing page should be shown.

#2.4.2 Redirects (301 error)

As a general rule of thumb all redirects should be done using the 301 permanently moved response. All sub domains should be redirected this way (example.com -> 301 -> www.example.com) and also all other domains that contain the same information, as shown below;

www.example.net -> 301 -> www.example.com

www.example.in -> 301 -> www.example.com

Assure also that only the specified URLs work and make a 301 redirect rule for all non-specified URL’s when called missing.

#3 General page requirements

#3.1 Using standards

The site should comply with the World Wide Web consortium’s (http://www.w3.org/) recommendations for creating web pages (XHTML 1.0 Transitional should be enough) and also comply with the Americans with disabilities act (http://www.ada.gov/) if required.

#3.2 Page design

The pages should be designed with CSS positioning and the content part of the page should appear in the source code as early as possible preferably before other body content such as navigational blocks.

The navigation should be implemented with anchor tags and text and the links should not redirect.

Breadcrumb navigation would increase SEO with internal back links and usability in a sense that the visitor would see their location on site. Example of the breadcrumb navigation: Home => List furniture items => View table => ...

Scripts and other elements (CSS) should be put in external files. The source code should be kept clean with little or no unused code. The preferred maximum file size for HTML code is 100 KB.

#3.3 Elements of a page

The following elements should always be included (and be editable somehow) on a page which is to be indexed by search engine:

• Page title ([title]-element in the header)
• Meta description, robots and keywords (in the header)
• Page heading (one [h1] per page)

#3.3.1 Page title ([title]-element in the header)

A page title should be as specific and concise as possible with respect to the document. This will insure its uniqueness and click-through in Search Engine Result Pages. A structure similar to "Page name | Section name | Site title - Tagline" is encouraged for clarity, uniqueness and better usability for the visitor. Focus on delivering a title that spans from specific (closer to the beginning) to general keywords. The length of the title needs no more then 80 characters.

#3.3.2 Meta description, robots and keywords (in the header)

HTML meta description around 150 characters should be sufficient. Although it doesn't hurt to be a little more, this data should contain the most concise information about the document. The uniqueness of this information also plays a fair role as far as Search Engine Result Pages are concerned.

Meta keywords on the other hand are not quite necessary since it is the responsibility of the search engine indexers to determine the nature and the relevancy of the document. For the purposes of accuracy, they can't rely on what the document claims it to be. There comes a transition on the Web which provides this sort of meta information about the document. Today, the results gained from meta keywords are negligible. See below some examples of well written meta tags;

<meta name="description" content="Suppliers of quality office furniture and accessories at discount prices.">
<meta name="keywords" content="furniture, office, store, shop, retail, discount">

#3.3.3 Page heading (one <h1> per page)

A proper structured document will consist of headings, paragraphs, lists, tables, and forms, and use an external stylesheet to style them. Many search engines place more emphasis on text within heading tags (and not just on keywords provided in meta elements), so make sure they use keywords. Use one <h1> tag per page with the most important keywords. You can also use other head tags ( <h2>, <h3> etc.) to provide variations and support the main heading.

Some example of tags are;

<h1>Tables</h1>
<h2>Round tables</h2>
<p>... information about round tables ...</p>
<h2>Square Desks</h2>
<p>... information about square desks, etc.</p>

#3.3.4 Body text

Make sure the text of your web pages contain keywords and common phrases which people might search for. Be careful with the frequency of your keywords - you want to have them occur at least a few times if possible, but don't repeat yourself so much that the copy becomes unnatural. The idea is to discretely spread keywords around without making it obvious.

A well written document will naturally use keywords that are appropriate and in proportion. Search engine algorithms essentially compare similar documents to get a better understanding of the nature of the document. If a document is not well written and gives off-balanced scores then it will raise flags and possibly mark it as not relevant as it indicates a document that is written for the machine and not for the human reader. Keep in mind that indexing is in place to assist human searches. An example of good body text could be like;

[p]Buy office furniture at affordable prices from any of our retail stores.[/p]

#3.3.5 Images and Pictures

When pictures, that are not part of the page template, are used they should always include an ALT description. This description should either be automated or editable (This is partly already a requirement of the Americans with disabilities act).

#3.4 Automation

The title element and the meta description and keywords need to be automatically generated according to different templates. These templates will include page- and directory specific elements as well as generic elements. An example of a template for the title element for a page called Search results page could be:

[Results] - [category] - Search results – My furniture example.com

• Different elements that could be included are
• Results = Search results pages (New, Old, All)
• Category name = Such as Wood tables, Wood chairs, Metal chairs
• Area = Can represents location of entity.
• Page number in a Search results, if applicable
• The category name or area might not be in basic form – different grammatical forms might be needed.

In the title, meta information and headings the keywords or key phrases are added as is or in another grammatical form but when automating (URL rewriting) the URL, it may need some encoding if other language has been used:

• Non-ASCII keywords (and phrases) included in URLs need to be encoded in hex values (maybe using PHP function like urlencode()) like:
• www.example.com/product/table/એપલ => www.example.com/product/table/%E0%AA%8F%E0%AA%AA%E0%AA%B2

#4 Index page

Index page of your website is the most likely to get the highest number of inbound links since it is entry point of your website. Hence linking other pages of website from this page becomes very important. By theory this page should host almost links of all pages that starts from here.

However number of links in such page should be around 100, in many projects it may not be possible to display all links. In such cases most important links should be made visible from here. And remaining pages could be linked from there because our purpose is to chain all important pages to be get indexed.

To make this working it becomes important to identify those important links. For example if you are selling something then this home page can have link of those pages that display list of items per category of products. Similarly if they are bound to certain geographical location and if you website displays list of selling items per province/city/area then links to those pages could be placed on this page.

#5 Search pages

Search pages whether simple or extended, may not be indexed as they are not containing, be default, any information to be searched for.

However for usability point of view, their URLs, page design and on-page information should be properly designed and implemented.

#6 Listing pages

Listing pages are the 2nd most important pages for any website as they display information about entities for which website is created. Listings entities can include various types of stuffs ranging from selling items, ads, jobs etc.

Such listing may contain pagination and sorting links depending upon results and interest of users. It is recommended to keep pagination links in text mode so that search engine can crawl through all available pages and can index those pages. However sorting links may be implemented using JS (Ajax) etc. so that additional query to server can be minimized. From search engine point of view, it doesn't matter in what order information displays.

If possible, URL scheme of such pages can be made self-informative. For example for furniture selling website URLs can be designed like below; 

www.myfurniture.com/tables/round
www.myfurniture.com/tables/square
www.myfurniture.com/tables/plastic
www.myfurniture.com/chairs/rocking
www.myfurniture.com/chairs/revolving

Text appearing on such pages should be as informative as possible and number of entity per list should be kept around 10 to 30 entities. Listing pages may also contain links to other important pages which are to be indexed.

#7 View pages

View entity pages shows detailed information about entities listed in listing pages. Title, Meta description, Meta keywords, H1 tag should contain information about entity that is expected to be viewed.

#8 Other pages

Other pages may include pages like Login pages, Posting/Editing entity pages etc.

#8.1 Login pages

Such pages should not get indexed as they don't contain any public searchable information.

#8.2 Posting/Editing entity pages

Any page that contains forms to be submitted are not normally indexed as they don't display any searchable information to general public.

General rule of thumb is that those pages which changes stat of the server (like data is inserted/updated, file is created/delete etc.) or those pages which are personal to users are not indexed as they are tightly integrated with data of the website.

#9 Resources

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=35769
http://help.yahoo.com/l/us/yahoo/search/basics/basics-18.html