Test data identification, manipulation, handling, and reporting are major parts of ETL testing. If we do any test data mistake here then it will end up in a critical issue. I would like to share my experience one of the scenario where I missed to communicate the test data unavailability instead I took my own action on that.
Experience of mistake in Test data:
I was working on a project which has the below data flow,
Source -> Stage1 -> Stage -> Datamart
The scope of the testing for us was Stage -> Datamart. And a separate testing team was doing the Source-> Stage1 part.
While I was doing performing the data purging testing on Datamart, there was not enough data to test the feature. Without intimating/requesting to a vendor, I myself took the option of updating the data in staging table such that the data would satisfy the purge criteria.
I executed the data purge test scenario and passed the test case after verifying the functionality.
The code has been deployed into UAT environment and the team has started UAT phase. UAT team member created test records in source application to test the same feature (purging) which I was tested in integration. The created record was rejected in Stage1-Stage ETL job itself due to incorrect Join condition implemented by the development team.
On next day call, the question was to the testing team who was tested Stage1->Stage section, it seems that the team missed capturing the specific test scenario itself.
Then question on me, “Do you have a test case to cover this?” I said yes and explained as we didn’t have the required test data I modified the data in the staging table. Everyone was shocked and I heard one hard voice that, you should have told about the unavailability of test data?
Be careful in case of test data unavailability and don’t make the same test data mistake in your ETL testing.
Never ever wait for anyone or don’t consult with anyone, just drop a mail to appropriate group with clear explanation. Open it up in your daily call and confirm on the further step.