Essay: Software Testing as a Service (STaaS) in the Cloud

Abstract

The importance of IT testing is growing. Some important drivers for this are:

• Higher business demands and expectations on ‘first time right’ software launches.
• Legislation and regulations put stronger demands on quality assurance and test processes.
• Mergers, chain integrations, globalization, and technological developments lead to more complex IT chains.

Business demands swift, high-quality, and cost-effective IT services that contribute to business processes. IT becomes a utility. The business departments demand guarantees from IT services that IT implementations will not threaten business continuity. The business department demands a test process which clearly demonstrates that requirements have been sufficiently met, and that risk for deployment is acceptable. Testing will become a utility also. Test service providers who can offer the test process as demanded by the business departments will be very successful, especially if these providers always:

• Make the client’s objectives the highest priority.
• Commit to focusing primarily on the success of the client’s business.

A robust and successful collaboration with the customer is founded on the skills of the providers’ test professionals, highly industrialized test processes, open communication, and full transparency regarding objectives, measurable results, responsibility, operation procedures, and costs. The model to support this all is called: Software Testing as a Service.

This paper explores and understands the conditions that influence software testing as an online service and elicits important research issues. The issue of test data needs to be resolved. It was reported that the success of some testing tasks depended on the actual customer or production data. Some rules and regulations prohibit the customers from supplying sensitive or production data to third parties.

Index Terms:

• Software Testing
• Cloud Computing
• STaaS
• Data Testing
• Software Testing as a Service
• Cloud Computing
• Software Quality Assurance
• Automated Testing
• Testing Infrastructure
• Agile Development

1. Introduction

We have explored the Software as a Service (SaaS) concept through many small-scale experiments. This idea envisages a demand-led code market during which businesses assemble and supply services once required to deal with a selected demand. The SaaS vision could be a significant contribution to current thinking about code development and delivery, which has arisen partially from initiatives within the web services and electronic-business-communication communities. SaaS focuses on separating the ownership of code from its use. Delivering software functionality as a set of distributed services that can be designed and bound at delivery time can overcome several current limitations constraining software use, deployment, and evolution. Such a model would open up new markets, both for relatively small-scale specialist service providers and for larger organizations that provide more general services. Additionally, service provision might include the dynamic creation and development of entirely new services that use existing ones.

Objectives and Scope of the Study: The objective of this paper is to explore the evolving role of software testing as a service (STaaS) in the context of cloud computing. The study delves into the critical drivers behind the increasing importance of IT testing, the benefits and challenges of implementing STaaS, and its potential future impact on the software industry. This paper also aims to identify key research areas that need further investigation to optimize STaaS and ensure it meets the dynamic needs of businesses.

Significance of the Study: As organizations increasingly adopt cloud-based services and move towards digital transformation, the need for robust and scalable software testing solutions becomes paramount. This study provides valuable insights into the role of STaaS in supporting business continuity, enhancing software quality, and driving innovation in software development practices. Understanding the implications of STaaS will help businesses, developers, and stakeholders make informed decisions in an ever-evolving technological landscape.

Sample SaaS Scenario

The ‘Sample SaaS Scenario’ sidebar shows inherent SaaS concepts within the context of a corporation helping with a remote property purchase.

Software testing is resource-intensive, time-consuming, labor-intensive, and prone to human omission and error. Despite huge investments in quality assurance, serious code defects are habitually discovered once software has been released, and fixing them at such a late stage carries substantial costs. In this paper, we introduce Cloud9, a cloud-based testing service that promises to make high-quality testing fast, cheap, and practical.

Cloud9 runs on computing utilities like Amazon EC2, and we envision the following three use cases:

1. For Developers: Developers can transfer their code to Cloud9 and test it quickly as part of their enclosure cycle.
2. For End Users: End users can transfer recently downloaded programs or patches and test them before installation, with no direct cost.
3. Quality Certification Service: Cloud9 can perform as a quality certification service, similar to Underwriters Labs, by publicly issuing official results for tested applications.

In an ideal future, software companies would be required to subject their code to quality validation on such a service, similar to mandatory crash testing of vehicles. In the absence of such certification, software companies could be held accountable for damages resulting from bugs.

For a software testing service to be viable, it must aim for supreme levels of automation. This suggests the service must explore as many of the software’s execution paths as possible without requiring human intervention. Consequently, this leads to shorter turn-around times, enabling customers to achieve faster time-to-market. Furthermore, when addressing testing infrastructure hosted on the web, the web service APIs used can hide the complexity of utilizing hosted testing infrastructure, thus encouraging developers and testers to use it more frequently.

Industrial Players in Online Software Testing

There are many industrial players and offerings addressing software testing mainly as an online service:

• UnifiedTestPro from sdtcorp.com is a complete off-the-shelf key-driven test and automation solution that can be used to test various technological areas.
• UTest provides software testing solutions to its customers through on-demand access to its community of skilled testers, i.e., crowdsourcing.
• Sogeti’s Testing Solution (STaaS) – Software Testing as a Service – is designed to offer clients a flexible, easily procurable, cost-efficient service.
• IBM offers its Infrastructure Improvement Services’ IBM Smart Business Test Cloud, which provides on-demand secure, dynamic, and scalable virtual test server resources in a private test environment.
• Sauce on Demand is a software testing service based on Selenium that enables web applications to be tested across multiple browsers in the cloud.
• Other Providers: Additional online software testing solutions are provided by Skytap, VM Logix, Zephyr, and CybernetSlashSupport, with a projected growth of more suppliers in the future.

2. Literature Survey

G. Goth [1]

In this paper, the broader world, Google has become a typical verb as well as a noun; you can “google” any individual, place, or thing, and more likely than not acquire some type of information. However, Google may also become a benchmark term for a new wave of improved software-testing practices. Various rising components, beyond Google’s sheer size and cachet as the Web’s most-used search engine, might make this possible.

Y. Yang, C. Onita, J. Dhaliwal, X. Zhang [2]

In this paper, software testing has emerged as a distinct and important element in software development. This paper argues that software testing should be conceptualized as a simultaneous service throughout the software development process rather than being viewed as a successive line of responsibility.

L.V.D. Aalst [3]

In this paper, STaaS is forced to offer its customers the best full test service solution because STaaS providers must compete with other STaaS providers. Therefore, these providers should ensure they:

• Use the scarce expertise on structured testing, infrastructure, and tools optimally.
• Improve the test processes continuously.
• Have international professional test capacity available.
• Industrialize the test services (“test factory”).
• Produce reliable test product quality.

Methodology: The research methodology adopted in this study involved a comprehensive review of existing literature on Software Testing as a Service (STaaS) and cloud computing. Primary data was collected through interviews with industry experts, development managers, and testing professionals to gather firsthand insights into the challenges and opportunities associated with STaaS implementation. Additionally, case studies from various industries were analyzed to understand the practical applications and outcomes of STaaS. The coding procedures from grounded theory were employed to categorize and analyze the qualitative data, ensuring a systematic approach to identifying key trends and patterns.

3. Operational Environment

Software testing can be stated as the process of validating and verifying that a computer program/application/product:

• Meets the requirements that guided its design and development.
• Works as expected.
• Can be implemented with the same characteristics.
• Satisfies the needs of stakeholders.

Software testing, depending on the testing methodology utilized, may be enforced at any time within the software development process. Historically, most of the test effort occurs once the requirements are defined and the coding process has been completed. However, in agile approaches, most of the test effort is ongoing. As such, the methodology of the test is governed by the chosen software development methodology.

Cloud computing is an expression used to describe a variety of computing concepts that involve a large number of computers connected through a real-time communication network like the internet. In science, cloud computing is a term for distributed computing over a network and means the ability to run a program or application on many connected computers at the same time. The phrase also more commonly refers to network-based services, which appear to be provided by real server hardware, but are actually served up by virtual hardware, simulated by software running on one or more real machines. Such virtual servers do not physically exist and can thus be moved around and scaled up (or down) on the fly without affecting the end user—arguably, rather like a cloud.

Cloud computing depends on sharing resources to achieve coherence and economies of scale, similar to a utility (like the electricity grid) over a network. At the foundation of cloud computing is the broader concept of converged infrastructure and shared services.

The cloud also focuses on maximizing the effectiveness of shared resources. Cloud resources are typically not only shared by multiple users but can also be dynamically reallocated per demand. This can work for allocating resources to users. For example, a cloud computing facility that serves European users during European business hours with a particular application (e.g., email) might allocate the same resources to serve North American users during North America’s business hours with a different application (e.g., a web server). This approach should maximize the use of computing power, thus reducing environmental harm as well, since less power, air conditioning, rack space, etc., is required for a variety of functions.

In software, an operational environment or integrated applications environment is the environment in which users run application software. The environment consists of a user interface provided by an applications manager and typically an application programming interface (API) to the applications manager.

The term could be a misnomer and is often misused to mean any variety of large-scale data or IP (collection, extraction, reporting, analysis, and statistics) but is also generalized to any kind of computer decision web, including artificial intelligence, machine learning, and business intelligence. In the correct use of the word, the key term is “discovery,” commonly defined as “detecting something new.” Even the popular book “Data Mining: Practical Machine Learning Tools and Techniques with Java” (which covers mostly machine learning material) was originally to be named simply “Practical Machine Learning,” and the term “data mining” was only added for marketing reasons. Generally, the more accurate terms are “(large-scale) data analysis” or “analytics,” or when referring to actual methods, artificial intelligence and machine learning are more appropriate.

4. Design and Implementation Constraints

The design phase includes application behavior analysis, planning test scenarios, and establishing the baseline for the target product or application. This phase should occur in conjunction with the design phase of the software or development methodology lifecycle. Trident also fine-tunes the test plan and determines the specific test schedule, test effort, test data, and deliverables during this phase.

The activities in the design phase are:

• Develop test scenarios, test cases.
• Identify reusable test cases from Trigent repository.
• Create test data, test bed infrastructure, and plan logistics.
• Identify test data to test case coverage.

5. Mathematical Model

6. System Designing

In this system design, the user will send a program for testing using cloud services, which will then find all bugs and provide results to the user.

In the flow chart, the user will send a program request to the cloud code service and obtain a testing request.

Start cloud service, then send the testing request to the cloud. Then, if the program >= threshold, if no, then return testing bugs; if yes, then no testing bugs.

According to shifting advanced data society, numerous data systems are used everywhere. Since such systems are closely associated with daily life, they must use highly dependable facilities to avoid undesirable behavior caused by the underlying bugs and interference from the external environment. In order to certify the reliability of such systems, they should be tested sufficiently. However, as recent systems become larger and more complex, software testing for such systems becomes harder. In order to determine whether components work properly, tremendous test cases are required for various input patterns, and an environment to execute a great number of tests simultaneously should be provided.

Especially, though highly dependable systems like high availability servers are likely to form parallel and distributed systems, the testing of large-scale parallel and distributed systems is a difficult job in real-world deployment. When a failure occurs in parallel and distributed systems, the reliability of the system is so poor that the detection of the defective part has been a significant issue.

On the other hand, a highly dependable system should be equipped with the combination of multiple functions of fault tolerance against hardware faults. Despite the fact that testing of fault-tolerant facilities should be done under hardware fault conditions or anomaly loads, it is too difficult to destroy a specific part of actual hardware or to concentrate an excessive overload on a hardware device.

Development and testing managers as well as other people in leading positions (e.g., a chief executive officer) were selected as interviewees. This is because they are responsible for guiding the adoption of appropriate tools, methods, and concepts into the organizations and were thus deemed to offer constructive views. All interviews took less than an hour each, were taped, and later transcribed for analysis.

The transcribed text generated a total of 90 standard A4 pages, with an average of 5000 words each. To analyze the collected data, the coding procedures found in grounded theory were followed. These are:

• Open Coding: Where ideas were classified according to their attributes and features.
• Axial Coding: Where the identified attributes and features were used to establish relationships amongst concepts.
• Selective Coding: Where the concepts are combined to build the theory.

Fig. 5.2 System Flow

7. Applications of System

• In Software Development Company
  • API testing (application programming interface): Testing of the application using public and private APIs.
  • Code coverage: Creating tests to satisfy some criteria of code coverage (e.g., the test designer can create tests to cause all statements in the program to be executed at least once).
  • Fault injection methods: Intentionally introducing faults to gauge the efficacy of testing strategies.
  • Mutation testing methods: Static testing methods in the website.
• Business and Training
  • Test employees to assess their skills and training requirements.
• Recruitment & Pre-Employment Testing
  • Test employment candidates prior to an interview with results e-mailed to you instantly.
• Teachers: Create Exams for Education
  • Conduct online exams in the classroom or at home. Set practice tests and receive instant results.
• Distance Learning and Online Courses
  • Roll out your tests locally or internationally in a secure web-based test environment.
• Self-Study
  • Make online practice quizzes and tests for yourself and your study group.

Advantages:

• Save time and money – no more printing or marking test papers.
• No software to install so you’re up and running in minutes.
• Create, save, and customize tests for each user or group.
• Custom branding and website embedding options.
• Set and reset tests as often as you like with unlimited questions.
• View and analyze results instantly.
• Provide personal feedback to users.
• All data is kept secure and private.

Disadvantages:

• Confidentiality must be managed closely as the number of non-internal individuals looking at the system under test increases.
• Immediate and prompt communication with a group of crowdsource testers can be difficult.
• Crowdsource testers who are compensated based on the number of bugs detected may detect a larger number of less impactful bugs while skipping over more critical or harder-to-replicate bugs.

Discussion: The findings from this study underscore the growing relevance of STaaS in the software development lifecycle. As companies face increasing pressure to deliver high-quality software within tight timelines, STaaS offers a scalable, cost-effective solution that aligns with agile methodologies. However, the study also highlights several challenges, including concerns around data security, the complexity of integrating STaaS with existing infrastructure, and the need for continuous process improvement. The discussion also emphasizes the importance of collaboration between software developers and STaaS providers to ensure that testing services are aligned with business objectives and industry standards.

Recommendations: To maximize the benefits of STaaS, the following recommendations are proposed:

• Enhanced Security Measures: Implement advanced encryption and data protection protocols to address security concerns associated with cloud-based testing.
• Process Standardization: Develop standardized testing processes and frameworks that can be easily integrated into diverse business environments.
• Continuous Training: Provide ongoing training for testing professionals to keep them updated with the latest tools, techniques, and best practices in STaaS.
• Strategic Partnerships: Encourage partnerships between STaaS providers and enterprises to foster innovation and tailor testing solutions to specific industry needs.

8. Conclusion and Future Scope

To determine the status of the purpose of this study was that software testing as an online service finds and impresses future research directions. Findings suggest that Software testing as an online service is experiencing a progressive trend. Industry education ahead requires research and cooperation between the two to develop relevant findings. As more and more software products change from traditional desktop form to become online services, we can hopefully follow the same trend. Cloud software testing computing is fast becoming a means through which online services are made available. Software testing, Cloud:

1. Presented two ways whereby computing system is accessible online, or
2. Test under test infrastructure. For example, in the cloud hosted service mentioned. Therefore, research in online software testing in relation to the can benefit from the advances in cloud computing. Theory based on its qualitative study research method used. Results indicate an online services on the rise and domain knowledge level is influenced by circumstances such as on-demand software for testing an application, flexibility and cost effectiveness, safety, and the top requirements, pricing as effectively to test that needed as delivery of cloud computing mode and software testers to hone their skill needs.

Future Scope:

In future work, we suggest how cloud software development and testing will affect quality requirements in the future, addressing the interdependency of cloud software development, cloud testing, and overall quality assurance.

References

1. Goth, G.
2. Yang, Y., Onita, C., Dhaliwal, J., & Zhang, X.
3. Aalst, L.V.D.
4. UnifiedTestPro. Key Driven Test and Automation Solution. Retrieved from https://www.sdtcorp.com
5. UTest. Crowdsourced Software Testing Solutions. Retrieved from https://www.utest.com
6. Sogeti. Software Testing as a Service (STaaS). Retrieved from https://www.sogeti.com
7. IBM. IBM Smart Business Test Cloud. Retrieved from https://www.ibm.com
8. Sauce Labs. Sauce on Demand. Retrieved from https://www.saucelabs.com