Introduction

In this white paper we will discuss some of the best practices applied at Belatrix when developing and performing Quality Assurance on Applications using the Cloud.

Belatrix Software is a leading Software Services Outsourcing company with its main offices and delivery centers based in Mendoza, Argentina and Lima, Peru. We provide Software Development, Quality Assurance, Localization and Graphics Design Services using the “Agile Near-Shore Model”.

Cloud Computing

Cloud computing is a means of using the Internet and remote servers for software applications, data access, data management and storage resources. Cloud computing allows consumers and businesses to use applications without installing software or accessing local files on their computers and hence can be used on any computer with Internet access.

Cloud Testing

Cloud Testing is a means of testing cloud-based applications that use resources found in the cloud. By resources, we mean any element (hardware, software and infrastructure) necessary to carry out the tests. Cloud testing provides an end-to-end solution that transforms the way testing is done and can help an organization boost its competitiveness by reducing the cost of testing without negatively impacting mission-critical production applications. By leveraging a cloud computing solution for testing, organizations can shorten provisioning time because the cloud enables provisioning of test servers on demand. This helps ensure unused servers are not sitting idle.

How and Where to Test?

Cloud testing services provided by Belatrix assure a high level of coverage and they also represent end-user environment of the application. This is due to the fact that devices which the application is expected to be run on, whether they are computers, tablets, smart phones or other devices are easily available or easily simulated.

Cloud Testing Benefits

It’s easy to see why Cloud Computing solutions appeal more and more to businesses globally.

• Location independent access – The use of thin clients or virtualization significantly reduces hardware requirements, making it possible to test the services anywhere.
• Reduced cost of ownership – Using service provided by existing Cloud deployments removes the need to install and support hardware. The use of Cloud solutions makes it possible to centralize all IT components. Any maintenance steps will need to be executed only once, centrally, and they will be mirrored on all end user instances.
• Green IT – The reduced requirement for hardware, implementations and location dependence enables businesses to reduce the carbon footprint of its IT infrastructure.
• Good Cloud Testing Practices
• It is important to take into consideration the additional time and/or personnel needed to perform exhaustive tests on all the devices.
• It is essential to have an explicit list of devices on which quality assurance needs to be performed. Hence, the company -should commit to providing support for all target devices and operating systems

Unlike the traditional performance testing techniques, where scalability is typically limited to a certain number of users within the network, in the cloud, the scope of the applications scalability is virtually unlimited.

Performance Testing

This is used to measure response times and isolate issues related to specific steps or actions while system is subjected to increasing load from different locations and multi user operations.

Load Testing

Application / system stability is a major factor when considering user count in multiples of hundreds.

Stress Testing

Due to the cloud characteristics, it is imperative to identify issues as the system is tested to its breaking points at often beyond 2x, 3x or more maximum expected usage.

Capacity Testing

Being hosted in a cloud environment it is prudent to determine maximum capacity for current or future hardware, bandwidth or other needs as well as to validate installed hardware and network for expected usage scenarios

Fail-over Testing

This is set of tests which are conducted under anticipated load with component failure (simulated) during the course of a test.  For example, in a web environment, fail-over testing determines what will happen when multiple web servers are being used under peak anticipated load, and one of them unexpectedly dies.

Application security testing

This type of testing is done to secure application software that is running on or being developed in the cloud. This includes testing whether it’s appropriate to migrate or design an application to run in the cloud, and if so, what type of cloud platform is most appropriate. The fact that most applications have dependencies on various other systems should be taken into consideration.

Governance Risk Compliance (GRC) testing

Its main focus is to list threats, vulnerabilities and risks that are associated to all three parts of Cloud Computing – Infrastructure as a Service (IaaS), Platform as a Service (PaaS) & Software as a Service (SaaS) and suggest controls which have been assimilated from the best practices prevailing in the Industry.

Latency Testing

Cloud testing is utilized to measure the latency between the action and the corresponding response for any application after deploying it in the cloud.

Browser testing

Browsers are not necessary in cloud-based software applications if thin-client software is used. However, in most cases, a variety of popular browsers are used, all in almost equal measure. It is imperative to list at least the top four or five browsers by global usage, and ensure that the application test plans include testing the applications with these browsers. Scripting at the browser end usually causes browser incompatibilities, and cloud-based applications need to be designed with minimal or no browser scripting if you expect a variety of browsers to be used.

Commonly Used Tools

Computing Load Test and Performance Monitoring Tools such as:

• Perfecto Mobile’s
• Keynote (Test Center Enterprise)
• Monitis
• Cloudsleuth
• BrowserMob
• CloudTools
• GFI
• LoadStorm
• CloudHarmony
• InterMapper
• BlazeMeter

Web Functional/Regression Test Tools:

• Windmill
• QEngine
• Soasta
• Selenium

If accelerating time-to-market for new applications and services is a priority, test-and-development strategies that meet this criterion justify the expense of using a cloud platform. It also warrants investment because it replaces long-term capital and operating expense with a more efficient and flexible option.

Depending on the volume of development projects across the enterprise and the percent that fit with the cloud, it can also reduce need for internal lab resources which can be repurposed more efficiently.

About Belatrix’s QA Services.

Belatrix’s Quality Assurance department provides testing and quality assurance services for projects developed within the company (internal projects) or by other companies (external projects). It also supplies mechanisms to improve existing practices by developing standardized processes for the entire company and by applying ISO™ standards, international standards, and lessons learned from CMMI™-SEI.

Belatrix’s QA team specializes in high value-added QA services such as Load Testing, Scripted Testing, etc. Because of this we are extremely selective in our recruitment process and only hire top software engineers that have a passion for Quality Assurance.

Besides all the above, we must measure the quality of the applications developed regardless of the platform on which they will be executed. For this we may follow some or all of the following strategies:

• Functional Testing
• Graphic User Interface Testing
• Security
• Compatibility, and multi-device environments
• Concurrence testing: multi-user
• Risk-Based Testing
• Acceptance Testing
• Performance
• Accessibility Testing
• Regression
• Installation and Configuration
• Installation Qualification Testing
• Integration Test
• API Testing and Unit Test
• Navigability testing
• Automation
• Mutation test
• DB testing
• Smoke Test
• Webservices
• Web Compatibility

We invite you to learn how a relationship with Belatrix will give your company a distinctive advantage through low cost, disciplined, and high quality software development and quality assurance services.