What's the difference?
Test Automation Engineer works on automating the Regression Test Suite.
Test Engineer ensures the product can be released to the customer.
With my industry experience, I have realized, that, as part of automation engineer your job duties don't stop with creating Automation tests, Frameworks and running them. With more tests being automated, your responsibilities not only include running the complete Regression suite but occasionally validation and verification of the product.
There are different documents/sites defining different types of testing and where their usage, however,but finding bugs is a different skill. Though we are testers by role, our analytical minds work like developers as we develop scripts too. We could be biased. When something doesn't work, we look for a workaround and think it's no big deal. This attitude is a deal breaker!Instead use your skills to analyse and break the code.
Here I am going to list some of my ideas to ensure the product works as expected by trying to break it.
Before getting into functional testing(Black Box Testing) we can ensure White Box Testing is done to an adequate level. This is not always possible but if the development team works with you, you can ensure these are done so that there is better test coverage.
In White Box testing, the best way to ensure coverage is Path Testing. Path Testing can be done in three ways.
Line Coverage - Making sure all lines are executed at least once
Branch Coverage - Making sure during the control flow(if) all possibilities are executed (true/false)
Condition Coverage - Making sure every condition that leads to true or false is covered.
Apart from this we can check Data Integrity i.e. every function that modifies data is thoroughly checked.
Assuming the above coverage, we can step into Black Box Testing.
The first set of test cases should focus on verifying if the functionality is working as expected. Compare the program's behavior to every word of the specification document. If the functionality itself is not working as expected, then further testing is not required. A clear report is given when creating a bug for this.
Note: Every time a bug is fixed, there are side effects. So regression is very important apart from testing the bug fix.
The second cycle is boundary testing. Every function has a minimum value it can take and max value it can take. These include URL lengths for web pages, Cart Sizes in eCommerce, User ids in forms and min and max input values,negative values and zero for arithmetic operations.
After this, the most important part is to test is State Transitions. How the system behaves when moving from one state to another. What information is transferred or left behind when moving between these states and what is done with the information. How much data can be carried between these states and can it carry if there is an information overload. What happens when you try to go back and forth in a web page, try to access every option you have to go to a different state and make sure its working. These tests should be done like monkey testing, the more innovative you get the better.
Race Conditions are a good scenario to find bugs. This is very important for a system that is going to be used by multiple users
Error Recovery: When you test with invalid inputs, the system should recover gracefully. How a system recovers from a crash is very important for a pleasant customer experience. Sometimes when data is not configured properly in the environment, the user interface can hang. This can be handled by taking care of all exceptions possible.
Integration Testing: When testing with third parties, verify all kinds of inputs are tested. You can send information by missing some fields, including all fields and see how the system is behaving. Negative testing is equally important. Example for this kind of testing is to verify how the system behaves when there is one faulty record in a file of multiple records. The system should be able to report that record but process the remaining ones.
Compatibility and Configuration Testing: This is verifying if how your system behaves after updating to a newer version and how it works in a new environment. This could be different Operating Systems, Browsers etc.
System Testing: This is similar to Integration testing the only difference being there are no stubs or mocking of data. All modules are up and running.
Usability Testing: Here you will be checking the user experience. This could be alignment issues, how text is displayed, is it readable etc.
Smoke Testing: After each build, a set of test cases are executed to make sure all major functionalities are working as expected.
Types of Errors generally found:
User Interface Errors - Button disabled etc
Control Flow Errors(What comes next)
Functionality Errors
Boundary Related Errors
Calculation Errors (Arithmetic)
Race Conditions
Load Handling (Found in Performance Testing)
Output Display like Graphs, Tables, Results
Error Handling
Source and version control(When fixed code is missed to be deployed in subsequent release)
This is my strategy to find bugs.
The most important thing to remember is the earlier a bug is found the lower the cost to fix it.
Complete Test Coverage is not possible. No matter how much testing we do, there will be some scenarios that will be found after release. These customer incidents are added to regression test suite after fix.
Test Automation Engineer works on automating the Regression Test Suite.
Test Engineer ensures the product can be released to the customer.
With my industry experience, I have realized, that, as part of automation engineer your job duties don't stop with creating Automation tests, Frameworks and running them. With more tests being automated, your responsibilities not only include running the complete Regression suite but occasionally validation and verification of the product.
There are different documents/sites defining different types of testing and where their usage, however,but finding bugs is a different skill. Though we are testers by role, our analytical minds work like developers as we develop scripts too. We could be biased. When something doesn't work, we look for a workaround and think it's no big deal. This attitude is a deal breaker!Instead use your skills to analyse and break the code.
Here I am going to list some of my ideas to ensure the product works as expected by trying to break it.
Before getting into functional testing(Black Box Testing) we can ensure White Box Testing is done to an adequate level. This is not always possible but if the development team works with you, you can ensure these are done so that there is better test coverage.
In White Box testing, the best way to ensure coverage is Path Testing. Path Testing can be done in three ways.
Line Coverage - Making sure all lines are executed at least once
Branch Coverage - Making sure during the control flow(if) all possibilities are executed (true/false)
Condition Coverage - Making sure every condition that leads to true or false is covered.
Apart from this we can check Data Integrity i.e. every function that modifies data is thoroughly checked.
Assuming the above coverage, we can step into Black Box Testing.
The first set of test cases should focus on verifying if the functionality is working as expected. Compare the program's behavior to every word of the specification document. If the functionality itself is not working as expected, then further testing is not required. A clear report is given when creating a bug for this.
Note: Every time a bug is fixed, there are side effects. So regression is very important apart from testing the bug fix.
The second cycle is boundary testing. Every function has a minimum value it can take and max value it can take. These include URL lengths for web pages, Cart Sizes in eCommerce, User ids in forms and min and max input values,negative values and zero for arithmetic operations.
After this, the most important part is to test is State Transitions. How the system behaves when moving from one state to another. What information is transferred or left behind when moving between these states and what is done with the information. How much data can be carried between these states and can it carry if there is an information overload. What happens when you try to go back and forth in a web page, try to access every option you have to go to a different state and make sure its working. These tests should be done like monkey testing, the more innovative you get the better.
Race Conditions are a good scenario to find bugs. This is very important for a system that is going to be used by multiple users
Error Recovery: When you test with invalid inputs, the system should recover gracefully. How a system recovers from a crash is very important for a pleasant customer experience. Sometimes when data is not configured properly in the environment, the user interface can hang. This can be handled by taking care of all exceptions possible.
Integration Testing: When testing with third parties, verify all kinds of inputs are tested. You can send information by missing some fields, including all fields and see how the system is behaving. Negative testing is equally important. Example for this kind of testing is to verify how the system behaves when there is one faulty record in a file of multiple records. The system should be able to report that record but process the remaining ones.
Compatibility and Configuration Testing: This is verifying if how your system behaves after updating to a newer version and how it works in a new environment. This could be different Operating Systems, Browsers etc.
System Testing: This is similar to Integration testing the only difference being there are no stubs or mocking of data. All modules are up and running.
Usability Testing: Here you will be checking the user experience. This could be alignment issues, how text is displayed, is it readable etc.
Smoke Testing: After each build, a set of test cases are executed to make sure all major functionalities are working as expected.
Types of Errors generally found:
User Interface Errors - Button disabled etc
Control Flow Errors(What comes next)
Functionality Errors
Boundary Related Errors
Calculation Errors (Arithmetic)
Race Conditions
Load Handling (Found in Performance Testing)
Output Display like Graphs, Tables, Results
Error Handling
Source and version control(When fixed code is missed to be deployed in subsequent release)
This is my strategy to find bugs.
The most important thing to remember is the earlier a bug is found the lower the cost to fix it.
Complete Test Coverage is not possible. No matter how much testing we do, there will be some scenarios that will be found after release. These customer incidents are added to regression test suite after fix.
Selenium |Training|Job Support|+91-741-626-7887 Selenium a Web based automation tool that automates anything and everything available on a Web page. http://laymanlearning.com/selenium-training/ --- Send Enquiry --- hr@laymanlearning.com
ReplyDeleteNice post! definitively I will come back to update me on this technology Thanks for the informative post. Keep doing.
ReplyDeletesoftware testing training institute chennai
testing training
Nice post! definitively I will come back to update me on this technology Thanks for the informative post. Keep doing.
ReplyDeleteSelenium training institute in Chennai
Selenium Training Chennai