EV – Production Support Sr. Data Engineer B. – Job2237

Job Summary

We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will be responsible for developing enhancements, optimizing, and maintaining data integration processes and ETL workflows using MS SQL Server and SSIS. You will also support database development and maintenance tasks, including schema implementation, indexing strategies, query optimization, and data validation.

If you have strong experience in data integration, ETL development, MS SQL Server, SSIS, database development, performance monitoring, and incident management, we would love to hear from you.

Job Responsibilities

As a Data Engineer specializing in production support for our Microsoft SQL Server-based Enterprise Data Warehouse (EDW), and in future migrations to Snowflake and backend Redshift-based EDW, you will play a vital role in ensuring the reliability, availability, and performance of our data infrastructure. 

You will collaborate closely with cross-functional teams to diagnose and resolve issues, implement enhancements, and optimize processes to meet the evolving needs of our organization, including the transition to new data warehousing technologies:

• Production Support for Data Integration and ETL Processes:
  1. Collaborate with the on-premises team to improve, optimize, and sustain data integration processes and ETL workflows utilizing MS SSIS and, in the future, Snowflake.
  2. Modify and optimize complex ETL transformations, data validation, and error handling mechanisms to ensure data accuracy and integrity.
  3. Troubleshoot and resolve data integration failures, ETL errors, and data quality issues within the EDW environment, working closely with data engineers, data analysts, and business stakeholders to identify root causes and implement corrective actions.
  4. Provide timely support for critical production systems, responding to emergencies and escalations as needed to maintain service availability and data integrity.
• Database Job Management and Maintenance:
  1. Provide proactive and reactive production support for the Microsoft SQL Server-based Enterprise Data Warehouse (EDW), and in the future, for Snowflake.
  2. Ensure continuous monitoring of system health, diagnose performance issues, and resolve incidents promptly to minimize downtime.
  3. Implement and maintain monitoring tools, alerts, and dashboards to track system performance metrics, data processing status, and data quality trends. Proactively address potential issues before they impact business operations.
  4. Provision database access and facilitate BI security for visualization reports as necessary.
• Support for SSAS, Multidimensional Modeling, and future Semantic Layer:
  1. Collaborate with data modelers, architects, and developers to implement changes, enhancements, and upgrades to the EDW infrastructure, ensuring seamless integration and minimal disruption to ongoing operations, including the future support for semantic layer.
  2. Assist in defining and configuring measures, dimensions, hierarchies, and aggregations to support OLAP queries, data analysis, and the development of the semantic layer.
  3. Participate in optimizing SSAS cubes for performance and efficient data retrieval, aligning with the requirements of the semantic layer and ensuring compatibility with future data warehousing technologies.
• Data Quality Assurance:
  1. Work closely with the data governance team to analyze data sources, identify potential data quality issues, and assess data quality dimensions.
  2. Develop data quality rules and metrics in Collibra based on business requirements and industry best practices.
  3. Configure and customize data quality checks in Collibra’s data quality module to enforce rules, validate data, and automate monitoring processes.
  4. Document data quality rules, definitions, and thresholds in Collibra’s data governance platform to facilitate transparency and accountability.
• Documentation and Collaboration:
  1. Create and maintain technical documentation, including system configurations, data integration workflows, and troubleshooting guides.
  2. Collaborate with the on-premises team and stakeholders to understand requirements, provide technical recommendations, and drive improvements in data engineering processes.
  3. Participate in cross-functional meetings, communicate progress, and share insights to align teams and drive collaboration.
• Incident Management and Support:
  1. Provide timely response and support for incidents, data issues, and user inquiries related to the EDW.
  2. Manage the ticket queue, ensuring timely resolution of tickets related to data integration, ETL processes, and database access provisioning.
  3. Document incident reports, root cause analyses, and resolution procedures to facilitate continuous improvement within the team.
• Month-End Reconciliation:
  1. Perform month-end reconciliation by comparing data from each billing system with reports supplied by source system providers.
  2. Investigate any variances identified during reconciliation and provide updates on findings.
  3. Send daily email updates until all billing systems are reconciled.

Basic Qualifications

• Minimum of 5 years of experience as a Data Engineer with real experience working with data integration and ETL development
• Experience in writing and optimizing SQL queries for data extraction, transformation, and analysis.
• Proficiency in developing ETL workflows using MS SQL Server and SSIS.
• Experience with database development using MS SQL Server for the Enterprise Data Warehouse (EDW).
• Strong expertise in SSAS development and multidimensional modeling.
• Knowledge of data quality assurance techniques.
• Performance monitoring and optimization experience with EDW, MS SQL Server, SSIS packages, and SSAS models.
• Creation and maintenance of technical documentation related to system configurations, data integration workflows, troubleshooting guides.
• Familiarity with SSAS, multidimensional modeling, and semantic layer.
• Knowledge of data quality assurance using Collibra or similar tools.
• Experience in incident management and resolution related to EDW.

Nice to Have

• Experience with Snowflake and backend Redshift-based EDW.
• Knowledge of migrations to Snowflake and backend Redshift-based EDW.
• Experience in managing and maintaining Snowflake databases.

Target Start Date: 1/6/2025
Engagement Length: Long term.
Time Zone: CST
Working Hours: 8:00 am to 5:00 pm