Lead Test Engineer
As a Lead Test Data Engineer, you will play a critical role in ensuring the success of our migration projects from legacy systems to the Snowflake data warehouse, followed by reporting on PowerBI, Sigma, or Tableau dashboards. You will be responsible for leading and managing the testing efforts across the entire data pipeline, from ingestion, consumption, and validation, to transformation using DBT and final visualization. Your focus will be on ensuring data integrity, accuracy, and quality throughout the migration and reporting processes.
Have a question?
Our data experts are ready to help. Just enter your info, and we'll direct you to the right person.
Our data experts are ready to help. Just enter your info, and we'll direct you to the right person.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Key Responsibilities:
- Test Strategy Development:
- Lead the design and implementation of comprehensive test strategies for migration projects involving legacy systems to Snowflake.
- Define test plans, test cases, and test scripts for validating data ingestion, transformation, and consumption processes.
- Develop and maintain automated testing frameworks for continuous integration and delivery, ensuring compatibility with DBT and other ETL tools.
- Data Quality Assurance:
- Ensure data accuracy and integrity across all stages of the data pipeline, from ingestion to visualization.
- Perform thorough testing of data transformations, aggregations, and loading processes in Snowflake.
- Validate the accuracy of data models and transformations executed using DBT.
- Identify and document data quality issues, working closely with data engineers to resolve them.
- Collaboration and Communication:
- Collaborate with data engineers, analysts, and visualization experts to ensure alignment on testing objectives and data quality standards.
- Provide technical guidance and mentorship to junior test engineers and data engineers on testing best practices.
- Communicate test results, issues, and risks to project teams and management in a clear and concise manner.
- Test Automation:
- Lead the development and maintenance of automated data testing scripts and tools, focusing on Snowflake and DBT-based pipelines.
- Implement automated regression testing for data pipelines to ensure ongoing data quality throughout the migration process.
- Integrate test automation into the CI/CD pipeline for seamless and efficient testing processes.
- Visualization Validation:
- Validate data accuracy and consistency in PowerBI, Sigma, and Tableau dashboards, ensuring that the visualizations reflect the correct underlying data.
- Perform end-to-end testing from data ingestion to final visualization, ensuring that all data transformations and visualizations are accurate and meet business requirements.
- Work closely with dashboard developers to ensure that test results are accurately represented in the visualizations.
- Data Governance and Compliance:
- Ensure that testing processes adhere to data governance policies and regulatory requirements, particularly in the context of data migration.
- Validate data security, privacy, and compliance measures during the testing process.
- Contribute to the development and enforcement of data testing standards and best practices.
- Continuous Improvement:
- Stay up-to-date with industry trends and advancements in data engineering, migration, and testing methodologies.
- Identify opportunities for process improvements and automation to enhance the efficiency and effectiveness of the testing process.
- Lead post-mortem reviews of testing activities, identifying lessons learned and areas for improvement.
Qualifications:
- Education:
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related field.
- Experience:
- 7+ years of experience in data engineering or data testing, with at least 3 years in a leadership role.
- Proven experience in testing migration projects, particularly from legacy systems to Snowflake.
- Hands-on experience with data testing tools and methodologies, specifically in the context of Snowflake and DBT.
- Technical Skills:
- Strong knowledge of SQL and experience with relational databases (e.g., SQL Server, Oracle, Snowflake).
- Proficiency in data pipeline testing, ETL testing, and data validation methodologies, with a focus on DBT.
- Experience with test automation tools and frameworks (e.g., Selenium, JUnit, PyTest).
- Familiarity with data visualization tools such as PowerBI, Sigma, and Tableau.
- Strong scripting skills in Python, Shell, or a similar language.
- Soft Skills:
- Excellent problem-solving and analytical skills with a strong attention to detail.
- Strong leadership and mentoring abilities, with a focus on team development.
- Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.