Data Engineer

Responsibilities:

• Design and develop scalable Data Hubs to meet the data requirements of the entire organization.
• Create and refine ETL/ELT pipelines to streamline data ingestion, transformation, and storage.
• Organize Data Hubs in a way that promotes efficient data access and accommodates various use cases.
• Implement real-time data ingestion and update models to support streaming data and real-time analytics.
• Build data validation, anomaly detection, and monitoring features to ensure data integrity and reliability.
• Optimize data processing and storage to enhance performance with large-scale datasets.
• Set up CI/CD pipelines to automate the flow of data and deployment processes.
• Collaborate with data architects, AI engineers, and business teams to ensure data solutions align with enterprise goals.
• Continuously enhance the data infrastructure, focusing on scalability and reliability.
• Engage in Scrum/Agile processes to deliver robust, high-quality data solutions.
• Develop and maintain comprehensive documentation for data models, pipelines, and technical decisions.

Qualifications:

• Minimum 4 years of experience
• Strong proficiency in Python (or another relevant programming language) for data engineering tasks.
• Experience with Azure Data Factory, ADLS, and Azure SQL for data integration and storage solutions.
• Practical experience in creating scalable and optimized data pipelines.
• Familiarity with real-time data ingestion, streaming, and dynamic model updates.
• Knowledge of data preparation processes for AI/ML and Generative AI applications.
• Proven experience in developing automated data validation and monitoring systems.
• Expertise in SQL for data transformation and analytics tasks.
• Experience with CI/CD processes, infrastructure-as-code (Terraform), and containerization technologies (Docker, Kubernetes/AKS).
• Solid understanding of data security, compliance standards, and best practices.
• Ability to collaborate effectively within cross-functional teams and deliver results in an Agile setting.
• Strong ability to create high-quality technical documentation for data structures, processing logic, and best practices.
• Proficiency in English (both written and spoken), with a minimum of a B2 level.

Perks:

• Private medical care
• Multi-sport card
• Annual bonus
• Trainings and certificates