Python Data Engineer

Overview:
Seeking a Python-centric Data Engineer with commodity trading system experience. Key responsibilities include the development, maintenance, and optimization of data warehouses, ETL pipelines, and data lakes, primarily using Python. Proficiency in MDM systems and data lineage tools is also desirable. This role is critical for designing and executing data solutions that enhance our commodity trading capabilities.

Responsibilities:
- Lead the development, maintenance, and optimization of data pipelines, ETL processes, and data warehouses using Python, tailored to commodity trading needs.
- Collaborate with teams to capture requirements and develop scalable data solutions.
- Ensure high data quality and integrity across systems with Python-based tools.
- Optimize data processing and storage for enhanced performance and cost-efficiency.
- Develop and implement data governance policies, focusing on Python environments.
- Troubleshoot and support data-related issues, with a strong emphasis on Python solutions.

Requirements:
- Advanced proficiency in Python 3.8 for data manipulation, scripting, and pipeline development.
- Experience with AWS (S3, Glue) for data storage and processing, using Python SDKs.
- Knowledge of Kafka for real-time data streaming, with a focus on Python integration.
- Skilled in Python-based data pipeline tools like PySpark, Apache Beam, and Airflow.
- Strong SQL skills; familiarity with databases such as SQL Server, Redshift, and Mongo DB through Python.
- Experience with BI tools (Tableau, Qlik Sense, Power BI) and their integration with Python.
- Exceptional problem-solving skills, attention to detail, and ability to work collaboratively.

Preferred Skills:
- Java 11+ knowledge for developing custom data applications, though Python remains the primary focus.
- Experience in commodity trading or the financial services industry, with a strong preference for Python-based projects.
- Understanding of data governance principles, especially as they apply to Python environments.

Education & Experience:
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- 3+ years of experience in data engineering, with significant Python projects.
- Prior involvement with commodity trading systems, especially with Python, is highly regarded.