To meet the growing demand for Oracle Container Engine for Kubernetes (OKE) with global organizations, Oracle Cloud Infrastructure (OCI) is introducing new capabilities that can boost the reliability and efficiency of large-scale Kubernetes environments while simplifying operations and reducing costs.
For software teams, testing makes sense, applications should be screened for bugs. But why is testing important for your business and how does it fit into DevOps?
Testing is part of Continuous Delivery that assures quality at each stage of the delivery pipeline before moving on to the next stage. DevOps is an iterative cycle of building, testing, and releasing software in short iterations. A comprehensive testing environment helps each iteration of the DevOps loop strengthen the quality of the product. A weak testing phase can mean defects progress to release, and developers need to fix bugs while the product is live. Development teams fall on both sides of the testing spectrum.
A surveyby Mabel on the state of testing in DevOps indicates that automated testing (at least 4–5 different types of tests) is key to customer happiness. The 2021 State of DevOps DORA Report reveals that continuous testing is an indicator of success, with elite performers who meet their reliability targets being 3.7 times more likely to use continuous testing.
In this post, I discuss automated and manual tests, and 2 common types of tests: functional and non-functional.
Manual and automated tests
Software tests can be manual or automatic. If you've used an app on your device and reported a bug, you've carried out a manual test. Automated tests are scripted in advance and executed by a machine, they compare an expected result with the actual result. Both methods of testing have their place in a software application, however manuals tests are slower and require an environment for the testers.
As developers write automated tests in advance, the errors found in manual testing can inform automated tests to strengthen the test suite. Manual tests are suitable when opinion and nuance play a role, like user experience. There is no pre-determined result for an automated test to check in these cases.
Automated tests are near-instant and execute in the hundreds or thousands at runtime. Automated tests check for functionality and make sure every line of code and feature works as intended. In a DevOps process, automated tests enable Continuous Delivery by giving a test coverage of the application. If you want to set up your application with a test coverage, you install automated tests for every component of the application.
When you add new features to a release, you can run the tests to identify whether test coverage has decreased. You can use the results to identify bugs for the new release. Automated tests complement a Continuous Delivery DevOps strategy. The more tests you automate, the faster your application can iterate and cycle through the DevOps loop of building, testing, and releasing.
Functional and non-functional tests
There are many tests you can perform on your application. One way to categorize tests is functional and non-functional.
Functional tests ask questions like:
■ Does this button work?
■ Does one module work with another module?
■ Does the user journey work from the start of the experience to the end?
Maintenance testing checks if the application has retained all functionality from version to version. It asks whether any functionality in the application has regressed between versions. I include maintenance testing under functional tests because it relates to functionality, though some sources list it as a third type.
Non-functional tests check the way a system operates rather than the functions of the system. Non-functional tests asks questions like:
■ How secure is the application?
■ How much load can the application handle?
■ Can the application scale if needed?
Examples of functional tests
Unit tests test an individual unit of code for functionality. In the example below, we test a function, which is a block of code that does one task.
1. Test if a function works. This function averages the weather for the previous 14 days.
2. Give the function the weather inputs for the previous 14 days.
3. Check whether the expected output matches the actual output.
Integration tests verify the functionality between 2 or more modules. This example tests the integration between the e-commerce storefront and the shopping cart module.
1. Load the e-commerce store and add some items to the cart, and go to checkout.
2. Check that the correct number of items are in the cart and the listed price is correct.
Smoke testing is preliminary testing to reveal failures that can result in rejection of a release.
1. Does the web server return a 200 OK response?
2. Can I ping the database?
Acceptance testing confirms your application is working according to a requirements specification. In this example, there's a requirement for a rewards system to work with an application. The tests check for the expected behavior of a rewards system.
1. If a user tries to purchase a product with rewards points and they have enough points, the purchase price should be $0.
2. If a user tries to purchase a product with rewards points and they do not have enough points, the purchase price should be $20.
Examples of non-functional tests
Load and performance tests
Load and performance tests check metrics like speed, response time, and resource usage of the application.
1. Measure the loading time of the home page and flag if it exceeds a threshold value.
2. Measure database response time when handling 100 or more concurrent requests. Flag if response time exceeds threshold value.
3. Under increasing load, test how many nodes an application needs to recover.
Security tests check for security related weaknesses in the system.
1. Scan log files for sensitive information. eg. credit card numbers or email addresses.
2. Scanning for long running sessions as a sign that session handling isn't working as expected.
Scalability tests test for issues related to scaling the application.
1. Under increasing load, test how many nodes an application needs to recover.
2. Test the length of time required for more nodes to be added and the application to recover. Chart how quickly you can expect 90% of scale up events to complete in.
Testing is essential to DevOps processes, making sure your software meets quality requirements before moving to the next stage. Research has shown that automated testing is a strong indicator of customer happiness and successful teams.
Tests can be run manually or be automated, and there are 2 main types of tests: functional and non-functional.
A robust testing environment fits well with Octopus Deploy. Octopus takes care of the release and deployment, and automates sections of the DevOps lifecycle, making complex deployments easier. To learn more about the importance of testing, read our post about why you should track vulnerabilities after deployment.
Perforce Software joined the Amazon Web Services (AWS) Independent Software Vendor (ISV) Accelerate Program and listed its free Enhanced Studio Pack (ESP) in AWS Marketplace.
Aembit, an identity platform that lets DevOps and Security teams discover, manage, enforce, and audit access between federated workloads, announced its official launch alongside $16.6M in seed financing from cybersecurity specialist investors Ballistic Ventures and Ten Eleven Ventures.
Hyland released Alfresco Content Services 7.0 – a cloud-native content services platform, optimized for content model flexibility and performance at scale.
CAST AI has announced the closing of a $20M investment round.
Check Point® Software Technologies introduced Infinity Global Services, an all-encompassing security solution that will empower organizations of all sizes to fortify their systems, from cloud to network to endpoint.
OpsCruise's Kubernetes and Cloud Service observability platform is certified to run on the Red Hat OpenShift Kubernetes platform.
DataOps.live released an update to the DataOps.live platform, delivering productivity for data teams.
CoreStack and Zensar announced a strategic global partnership. CoreStack will provide its AI-powered NextGen cloud governance and FinOps capabilities, complementing Zensar’s composable cloud operations offering.
Delinea introduced the Delinea Platform, a cloud-native foundation for Delinea's PAM solutions that empowers end-to-end visibility, dynamic privilege controls, and adaptive security.
Sysdig announced a new foundation that will serve as the long-term custodian of the Wireshark open source project.
Talend announced the latest update to Talend Data Fabric, its end-to-end platform for data discovery, transformation, governance, and sharing.
Descope has raised $53M in seed funding and emerged from stealth to launch a frictionless, secure, and developer-friendly authentication and user management platform.
Loft Labs announced Loft v3 with new capabilities and flexibility for platform teams to build and enable their development teams with a self-service Kubernetes.
AWS Application Composer is now generally available.