Senior Data Engineer (Trading Data & Analytics)

Job Description – Senior Data Engineer (Trading Data & Analytics)

Position: Senior Data Engineer

Experience: 7+ Years (Preferred)

Domain: Trading Data & Analytics (TD&A)

Employment Type: Contract

Work Mode: Remote

Position Overview

We are looking for an experienced Senior Data Engineer to support Trading Data & Analytics (TD&A) shared data capabilities. The candidate will be responsible for designing, developing, enhancing, and maintaining reusable data products, data pipelines, integrations, and cloud-based data platform capabilities.

The ideal candidate will have strong hands-on experience with Snowflake, DBT, Confluent Cloud, Fivetran, and Apache NiFi, along with expertise in data ingestion, transformation, data integration, data governance, and production support.

The role requires close collaboration with product owners, architects, developers, data engineers, and business stakeholders to deliver scalable, secure, reliable, and reusable enterprise data solutions across multiple trading commodities and programs.

Key Responsibilities

1. Data Engineering & Pipeline Development

• Design, develop, and maintain reusable data pipelines, shared datasets, and enterprise data products.
• Build and support data ingestion, transformation, integration, and data delivery solutions.
• Develop data pipelines integrating ETRM (Energy Trading and Risk Management) systems and internal enterprise applications.
• Implement scalable data engineering solutions across cloud-based platforms.
• Support cross-commodity data capabilities and global trading data products.

2. Data Platforms & Technologies

• Develop and maintain data solutions using the following technologies:
• Snowflake
• DBT (Data Build Tool)
• Confluent Cloud / Kafka
• Fivetran
• Apache NiFi
• Power BI
• Manage data orchestration, workflow automation, scheduling, and dependency management.
• Configure, optimize, and maintain data product platforms.
• Manage Snowflake structures, access controls, security, and data provisioning.

3. Data Quality & Governance

• Develop and implement data quality controls, validation processes, and monitoring capabilities.
• Ensure data accuracy, consistency, reliability, and availability.
• Implement reusable and governed data products following enterprise data standards.
• Design scalable, secure, and maintainable data architecture patterns.
• Ensure compliance with established data governance and security requirements.

4. Production Support & Observability

• Monitor and support production data services, pipelines, integrations, and platform capabilities.
• Implement monitoring, telemetry, observability, and alerting solutions using Datadog and other approved tools.
• Troubleshoot data processing failures, integration issues, and platform defects.
• Participate in incident management, root-cause analysis, and service restoration.
• Improve platform reliability, performance, and operational efficiency.
• Support cost optimization, technical debt reduction, and continuous improvement initiatives.

5. DevOps & Engineering Best Practices

• Support DevOps activities, release management, and deployment processes.
• Participate in technical design discussions, code reviews, and peer reviews.
• Follow engineering standards and best practices.
• Maintain technical documentation and knowledge-sharing materials.
• Collaborate with architecture, infrastructure, development, and business teams to ensure successful delivery.

Required Technical Skills

• Strong experience in Data Engineering and enterprise data pipeline development.
• Hands-on experience with Snowflake and Snowflake data architecture.
• Strong experience with DBT for data transformation and modeling.
• Experience with Confluent Cloud / Apache Kafka for data streaming and integration.
• Experience with Fivetran for data ingestion and integration.
• Experience with Apache NiFi for data flow management and orchestration.
• Experience with Datadog for monitoring, observability, and alerting.
• Knowledge of data orchestration, workflow automation, scheduling, and dependency management.
• Experience with data quality, validation, governance, and security.
• Understanding of DevOps, CI/CD, release management, and production support.
• Experience supporting enterprise-scale data platforms and integrations.
• Familiarity with Power BI and enterprise reporting solutions.

Preferred Qualifications

• Experience working with Trading Data & Analytics platforms.
• Knowledge of Energy Trading and Risk Management (ETRM) systems.
• Experience supporting global trading data products across multiple commodities.
• Understanding of cloud-based enterprise data architecture.
• Experience working in large-scale, cross-functional engineering teams.
• Strong troubleshooting, analytical, and problem-solving skills.
• Excellent communication and collaboration skills.

Key Deliverables

• Development and management of global trading data products.
• Data product platform configuration a