Software Testing

Software testing life cycle (STLC) identifies what test activities to carry out and when (what is the best time) to accomplish those test activities. Even though testing differs between organizations, there is a testing life cycle.
Software Testing Life Cycle consists of six (generic) phases:

• Test Planning
• Test Analysis,
• Test Design,
• Construction and verification,
• Testing Cycles,
• Final Testing and  and Implementation
• Post Implementation.

Software testing has its own life cycle that intersects with every stage of the SDLC. The basic requirements in software testing life cycle is to control/deal with software testing –  Manual,Automated and Performance.

Test Planning

This is the phase where Project Manager  has to decide what things need to be tested,do I have the appropriate  budget etc. Naturally proper planning at this stage would greatly reduce the risk of low quality software. This planning will be an ongoing process with no end point.
Activities at this stage would include preparation of high level test plan-(according to  IEEE test plan template The Software Test Plan (STP) is designed to prescribe the scope, approach, resources, and schedule of all testing activities. The plan must identify the items to be tested, the features to be tested, the types of testing to be performed, the  personnel responsible for testing, the resources and schedule required to complete testing, and the risks associated with the plan.). Almost all of the activities done during this stage are included in this software test plan and revolve around a test plan.
In Test Planning following are the major tasks:
 1. Defining scope of testing
 2. Identification of approaches
 3. Defining risk
 4. Identifying resources
 5. Defining Time Schedule

Test Analysis

Once test plan is made and decided upon, next step is to deal a little more into the project and decide what types of testing should be carried out at different stages of  SDLC, do we need or plan to automate, if yes then when the appropriate time to automate is, what type of specific  documentation. I need for testing,;,
Proper and regular meetings should be held between testing teams, project managers, development teams,  BA  to check the progress of things which will give a fair idea of the movement of the project and ensure the completeness of the test plan created in the planning phase, which will further help in enhancing the right testing strategy created earlier. We will start creating test case formats and test cases itself. In this stage we need to develop Functional validation matrix based on Business Requirements to ensure that all system requirements are covered by one or more test cases, identify which test cases to automate, begin review of documentation, i.e. Functional Design, Business Requirements, Product Specifications, Product Externals etc. We also have to define areas for  Stress and Performance Testing.

Test Design

Test plans and cases which were developed in the  analysis  phase are revised. Functional validation matrix is also revised and finalized. In this stage risk assessment criteria is developed. If you have thought of automation then you have to select which test cases to automate and begin writing scripts for them. Test data is prepared. Standards for unit testing and pass / fail criteria are defined here. Schedule for testing is revised (if necessary) & finalized and test environment is prepared.

An approach for Security Testing of Web Applications

Introduction

As more and more vital data is stored in web applications and the number of transactions on the web increases, proper security testing of web applications is becoming very important. Security testing is the process that determines that confidential data stays confidential (i.e. it is not exposed to individuals/ entities for which it is not meant) and users can perform only those tasks that they are authorized to perform (e.g. a user should not be able to deny the functionality of the web site to other users, a user should not be able to change the functionality of the web application in an unintended way etc.).

Some key terms used in security testing

Before we go further, it will be useful to be aware of a few terms that are frequently used in web application security testing:

What is “Vulnerability”?
This is a weakness in the web application. The cause of such a “weakness” can be bugs in the application, an injection (SQL/ script code) or the presence of viruses.

What is “URL manipulation”?
Some web applications communicate additional information between the client (browser) and the server in the URL. Changing some information in the URL may sometimes lead to unintended behavior by the server.

What is “SQL injection”?
This is the process of inserting SQL statements through the web application user interface into some query that is then executed by the server.

What is “XSS (Cross Site Scripting)”?
When a user inserts HTML/ client-side script in the user interface of a web application and this insertion is visible to other users, it is called XSS.

What is “Spoofing”?
The creation of hoax look-alike websites or emails is called spoofing.
Security testing approach:

In order to perform a useful security test of a web application, the security tester should have good knowledge of the HTTP protocol. It is important to have an understanding of how the client (browser) and the server communicate using HTTP. Additionally, the tester should at least know the basics of SQL injection and XSS. Hopefully, the number of security defects present in the web application will not be high. However, being able to accurately describe the security defects with all the required details to all concerned will definitely help.

Security Testing Techniques:

1) Access to Application:

Whether it is a desktop application of website, access security is implemented by ‘Roles and Rights Management’. It is often done implicitly while covering functionality, e.g.in a Hospital Management System a receptionist is least concerned about the laboratory tests as his job is to just register the patients and schedule their appointments with doctors. So, all the menus, forms and screen related to lab tests will not be available to the Role of ‘Receptionist’. Hence, the proper implementation of roles and rights will guarantee the security of access.

How to Test: In order to test this, thorough testing of all roles and rights should be performed. Tester should create several user accounts with different as well multiple roles. Then he should use the application with the help of these accounts and should verify that every role has access to its own modules, screens, forms and menus only. If tester finds any conflict, he should log a security issue with complete confidence.

2. Data Protection:

There are further three aspects of data security. First one is that a user can view or utilize only the data which he is supposed to use. This is also ensured by roles and rights e.g. a TSR (telesales representative) of a company can view the data of available stock, but cannot see how much raw material was purchased for production.

So, testing of this aspect is already explained above. The second aspect of data protection is related to how that data is stored in the DB. All the sensitive data must be encrypted to make it secure. Encryption should be strong especially for sensitive data like passwords of user accounts, credit card numbers or other business critical information. Third and last aspect is extension of this second aspect. Proper security measures must be adopted when flow of sensitive or business critical data occurs. Whether this data floats between different modules of same application, or is transmitted to different applications it must be encrypted to make it safe.

How to Test Data Protection: The tester should query the database for ‘passwords’ of user account, billing information of clients, other business critical and sensitive data and should verify that all such data is saved in encrypted form in the DB. Similarly (s)he must verify that between different forms or screens, data is transmitted after proper encryption. Moreover, tester should ensure that the encrypted data is properly decrypted at the destination. Special attention should be paid on different ‘submit’ actions. The tester must verify that when the information is being transmitted between client and server, it is not displayed in the address bar of web browser in understandable format. If any of these verifications fail, the application definitely has security flaw.

3. Brute-Force Attack:

Brute Force Attack is mostly done by some software tools. The concept is that using a valid user ID, software attempts to guess the associated password by trying to login again and again. A simple example of security against such attack is account suspension for a short period of time as all the mailing applications like ‘Yahoo’ and ‘Hotmail’ do. If, a specific number of consecutive attempts (mostly 3) fail to login successfully, then that account is blocked for some time (30 minutes to 24 hrs).

How to test Brute-Force Attack: The tester must verify that some mechanism of account suspension is available and is working accurately. (S)He must attempt to login with invalid user IDs and Passwords alternatively to make sure that software application blocks the accounts that continuously attempt login with invalid information. If the application is doing so, it is secure against brute-force attack. Otherwise, this security vulnerability must be reported by the tester.

The above three security aspects should be taken into account for both web and desktop applications while, the following points are related with web based applications only.

4. SQL Injection and XSS (cross site scripting):

Conceptually speaking, the theme of both these hacking attempts is similar, so these are discussed together. In this approach, malicious script is used by the hackers in order to manipulate a website. There are several ways to immune against such attempts. For all input fields of the website, field lengths should be defined small enough to restrict input of any script e.g. Last Name should have field length 30 instead of 255. There may be some input fields where large data input is necessary, for such fields proper validation of input should be performed prior to saving that data in the application. Moreover, in such fields any html tags or script tag input must be prohibited. In order to provoke XSS attacks, the application should discard script redirects from unknown or untrusted applications.

How to test SQL Injection and XSS: Tester must ensure that maximum lengths of all input fields are defined and implemented. (S)He should also ensure that defined length of input fields does not accommodate any script input as well as tag input. Both these can be easily tested e.g. if 20 is the maximum length specified for ‘Name’ field; and input string “

thequickbrownfoxjumpsoverthelazydog” can verify both these constraints. It should also be verified by the tester that application does not support anonymous access methods. In case any of these vulnerabilities exists, the application is in danger.

5. Service Access Points (Sealed and Secure Open)

Today, businesses depend and collaborate with each other, same holds good for applications especially websites. In such case, both the collaborators should define and publish some access points for each other. So far the scenario seems quite simple and straightforward but, for some web based product like stock trading, things are not so simple and easy. When there is large number of target audience, the access points should be open enough to facilitate all users, accommodating enough to fulfill all users’ requests and secure enough to cope with any security-trial.

How to Test Service Access Points: Let me explain it with the example of stock trading web application; an investor (who wants to purchase the shares) should have access to current and historical data of stock prices. User should be given the facility to download this historical data. This demands that application should be open enough. By accommodating and secure, I mean that application should facilitate investors to trade freely (under the legislative regulations). They may purchase or sale 24/7 and the data of transactions must be immune to any hacking attack. Moreover, a large number of users will be interacting with application simultaneously, so the application should provide enough number access point to entertain all the users.

In some cases these access points can be sealed for unwanted applications or people. This depends upon the business domain of application and its users, e.g. a custom web based Office Management System may recognize its users on the basis of IP Addresses and denies to establish a connection with all other systems (applications) that do not lie in the range of valid IPs for that application.

Tester must ensure that all the inter-network and intra-network access to the application is from trusted applications, machines (IPs) and users. In order to verify that an open access point is secure enough, tester must try to access it from different machines having both trusted and untrusted IP addresses. Different sort of real-time transactions should be tried in a bulk to have a good confidence of application’s performance. By doing so, the capacity of access points of the application will also be observed clearly.

Tester must ensure that the application entertains all the communication requests from trusted IPs and applications only while all the other request are rejected. Similarly, if the application has some open access point, then tester should ensure that it allows (if required) uploading of data by users in secure way. By this secure way I mean, the file size limit, file type restriction and scanning of uploaded file for viruses or other security threats. This is all how a tester can verify the security of an application with respect to its access points.

Test data preparation techniques:Test data preparation techniques:

We have briefly discussed the important properties of test data and it also elaborates how test data selection is important while database testing. Now let’s discuss the ‘techniques to prepare test data’.

There are only two ways to prepare test data:

Method 1. Insert New Data:

Get a clean DB and insert all the data as specified in your test cases. Once, all your required and desired data has been entered, start executing your test cases and fill ‘Pass/Fail’ columns by comparing the ‘Actual Output’ with ‘Expected Output’. Sounds simple, right? But wait, it’s not that simple.

Few essential and critical concerns are as follows:

1. Empty instance of database may not be available
2. Inserted test data may be insufficient for testing some cases like performance and load testing.
3. Inserting the required test data into blank DB is not an easy job due to the database table dependencies. Because of this inevitable restriction, data insertion can become difficult task for tester.
4. Insertion of limited test data (just according to the test cases needs) may hide some issues that could be found only with the large data set.
5. For data insertion, complex queries and/or procedures may be required, and for this sufficient assistance or help from the DB developer(s) would be necessary.

Above mentioned five issues are the most important and the most obvious drawbacks of this technique for test data preparation. But if there are some advantages as well:

1. Execution of TCs becomes more efficient as the DB has the required data only.
2. Bugs isolation requires no time as only the data specified in test cases present in the DB.
3. Less time required for testing and results comparison.
4. Clutter-free test process

Method 2. Choose sample data subset from actual DB data:

This is the feasible and more practical technique for test data preparation. However it requires sound technical skills and demands detailed knowledge of DB Schema and SQL. In this method you need to copy and use production data by replacing some field values by dummy values. This is the best data subset for your testing as it represents the production data. But this may not be feasible all the time due to data security and privacy issues.

This strategy deserves one separate post which we’ll discuss in next article ‘Database gray-box testing’ and precautions to take while testing database.

Database Testing – Properties of a Good Test Data and Test Data Preparation Techniques

A couple of months ago, I wrote about database testing strategies. It covered the aspect that is entirely related to the execution of test cases. It was all about black-box testing of a database. There is another important aspect of DB testing activity which we will cover in this article.

As a tester, you have to test the ‘Examination Results’ module of the website of a university. Consider the whole application has been integrated and it is in ‘Ready for Testing’ state. ‘Examination Module’ is linked with ‘Registration’, ‘Courses’ and ‘Finance’ modules. Assume that you have adequate information of the application and you created a comprehensive list of test scenarios. Now you have to design, document and execute these test cases. In ‘Actions/Steps’ section of the test cases, you must mention the acceptable data as input for the test. The data mentioned in test cases must be selected properly. The accuracy of ‘Actual Results’ column of TC Document is primarily dependent upon the test data. So, step to prepare the input test data is significantly important. Thus, here is my rundown on ”DB Testing – Test Data Preparation Strategies”.

Properties of Test Data:

The test data should be selected precisely and it must possess the following four qualities:

1. Realistic: By realistic, it means the data should be accurate in the context of real life e.g. in order to test ‘Age’ field, all the values should be positive and 18 or above. It is quite obvious that the candidates for an admission in the university are usually 18 years old (this might be defined in requirements).

2. Practically valid: This is similar to realistic but not the same. This property is more related to the business logic of AUT e.g. value 60 is realistic in age field but practically invalid for a candidate of Graduation or even Masters Programs. In this case, a valid range would be 18-25 years (this might be defined in requirements).

3. Versatile to cover scenarios: There may be several subsequent conditions in a single scenario, so choose the data shrewdly to cover maximum aspects of a single scenario with minimum set of data, e.g. while creating test data for result module, do not only consider the case of regular students who are smoothly completing their program. Give attention to the students who are repeating the same course and belong to different semesters or even different programs. The data set may look like this:

Sr#	Student_ID	Program_ID	Course_ID	Grade
1	BCS-Fall2011-Morning-01	BCS-F11	CS-401	A
2	BCS-Spring2011-Evening-14	BCS-S11	CS-401	B+
3	MIT-Fall2010-Afternoon-09	MIT-F10	CS-401	A-
…	…	…	…	…

There might be several other interesting and tricky sub-conditions. E.g. the limitation of years to complete a degree program, passing a prerequisite course for registering a course, maximum no. of courses a student may enroll in a single semester etc. etc. Make sure to cover all these scenarios wisely with finite set of data.

4. Exceptional data (if applicable/required): There may be certain exceptional scenarios that are less frequent but demand high importance when occur, e.g. disabled students related issues.

How to Achieve Level 5 Maturity for QA and Testing Process

This article explains what are CMM levels and how to achieve these CMM levels for QA processes, explained with best examples.

For any process whether it is a QA process, development process or any non-technical process, there are levels of its maturity. By levels of maturity we mean that the level of formality and processes improvement, like ad-hoc processes – to formally defined steps – to managed result metrics – to optimization of the processes.

CMM (Capability Maturity Model) is process based model which is used to assess the maturity of an organization for different domains. Although this model is normally termed as the software development model but eventually it was used for other processes as well like QA and testing.

It has 5 different levels of maturity from 1 to 5. As we go towards level 5 from 1, variability and inconsistency reduces. Below are the details of 5 levels. Here we will go through the 5 CMM levels with respect to QA process and what all output/result is expected for each level to mature a QA/testing process and reach up to level 5.

Level 1 – Ad-Hoc: Unplanned, unsystematic, and inconsistent

As the word ‘Ad-Hoc’ states: unplanned, unprepared, at this level significance is not given to planning, following processes, guidelines and standards. There is no standardized & consistent way of doing any task. The only thing which is important at this level is meeting the timelines, irrespective of the quality of the end product and deliverables.

As there are no pre-defined standards and processes, same task is done in different ways by different people.

And this becomes even more unsystematic and inconsistent if same task is done differently next time.

Example -
QA – The example would be that in an organization although QA is 1 of the phases in a product life cycle but there are not any standard & no process defined, no templates for QA deliverables like plan, strategy, scenarios, and cases are standardized. Even if these are documented then all team members have their own way of doing it and not consistent at all.

Level 2 – Control: initiate defining processes at high level:

Solution to the problem which we saw at Level 1 of unavailability of QA processes, methodology & standards would be to have all these in place. The standards and processes are not only finalized but also are well documented, so that those can be re-used by any one for similar task.

Example -
QA – Define overall QA process and methodology for different types of testing like functional, data, performance etc. Define the role of a QA engineer in project’s life cycle and prepare templates for deliverables in each phase. Not only define and prepare rather share within team

Physical and Thermal Testing

Tests to determine a wide range of physical properties on raw materials, semi-finished or finished products are available. This may range from UKAS ISO 17025:2005 accredited (UKAS accredited testing laboratory No. 0013) to documented methods, national or international standards through to industrial or bespoke procedures. Whatever the specification, the testing is to the highest standards using transparent, traceable, calibrated approaches by highly qualified and experienced technical staff.

Thermal testing can range from measuring expansion at modest temperatures through to testing at high temperature to show fitness for purpose in harsh and corrosive environments. Bespoke testing regimes can be developed to proof test products in a wide range of industrial applications.

Examples of some of Ceram’s physical tests and thermal tests are shown below.

Physical Properties

Whilst many of the tests conducted are specific to particular materials and applications, others are more general in nature. Most of the tests are conducted to specific testing regimes under ISO, EN, BS, ASTM and DIN methodologies.

Refractories and Raw Materials

• Abrasion resistance
• Carbon monoxide attack
• Crushing strength
• Dimensions
• Fired shrinkage
• Modulus of Rupture
• Permanent Linear Change
• Plasticity
• Porosity
• Refractoriness
• Sieve grading
• Thermal shock
• Density

• Change in dimensions
• Creep
• Erosion of fibres
• Flexural strength
• Particle size
• Permeability
• Pore Size Distribution
• Pyrometric Cone Equivalent (PCE)
• Refractoriness-under-load
• Surface area
• Vitrification.

Building Products

• Bending strength
• Capillary absorption test
• Density
• Drying shrinkage
• Flexural strength
• Water absorption
• Moisture movement
• Reaction to fire
• Voids
• Thermal properties
• Vapour permeability

• Bond strength
• Compressive strength
• Dimensions
• Efflorescence
• Freeze thaw
• Water suction
• Nib support
• Sand grading
• Tensile splitting
• Transverse strength.

Tiles, Adhesives and Grouts

• Breaking load
• Colour differences
• Abrasion resistance
• Flexural strength
• Impact resistance
• Moisture expansion
• Rectangularity
• Shear strength
• Slip resistance
• Tensile strength
• Thermal shock
• Water absorption

• Chemical resistance
• Crazing resistance
• Dimensions
• Frost resistance
• Modulus of Rupture
• Adhesive open time
• Staining
• Slip
• Surface quality
• Thermal expansion
• Transverse strength.

Testing of Thermal Properties

Whilst many of the tests conducted are specific to particular materials and applications, others are more general in nature. Most of the tests are conducted to specific testing regimes under ISO, EN, BS, ASTM and DIN methodologies. Some of the available tests are listed below:

• Specific heat
• Specific heat capacity fibres
• Thermal conductivity
• Thermal expansion.

Environmental Analysis

As a UKAS accredited testing laboratory (No. 0013) to ISO 17025:2005, we offer a wide range of environmental tests to national and international standards. With experience of working with geotechnical companies, developers and regulators involved in redeveloping contaminated sites, our knowledge of environmental analysis is vast. Our services include:

Biogas Analysis

Once alternative fuels have been produced or sourced, stringent tests must be carried out, both to satisfy legislative demands and to assure users that the fuel is both safe to use and is not damaging to their processes.

Landfill Gas Analysis

Offering a broad range of chemical analysis, Ceram is also accredited to conduct the sampling and testing of gas following the Environment Agency Technical Guidance Note LFTGN04.

Alternative and Solid Fuels Analysis

Whether you need calorific values, legislative advice, or finished product evaluation, Ceram is well positioned to supply end users of alternative fuels and waste processors or handlers with all the essential support and analysis required.

Air Quality Monitoring

We offer both monitoring of the quality of both indoor and outdoor air, for example to test material emissions or occupational exposure limits in the workplace.