Job Summary:

We are currently seeking a motivated Data Engineer to join our clients' growing data engineering team. The ideal candidate will have a strong foundation in Python, SQL, Cloud technologies, advanced warehouses like Snowflake/Redshift/Athena, Kubernetes/microservices architecture, Kafka and big data technologies like Hadoop/spark or their cloud equivalent. They will be responsible for architecting, developing, maintaining, and optimizing data solutions/ data-pipelines / ETL processes to support our cloud-based initiatives. The candidate should have sufficient experience to understand the overall architecture and complex intricacies of our product and accordingly direct the team in development/migration activities. This is an excellent opportunity for a data enthusiast to gain hands-on experience in a dynamic, data-driven environment.

Key Responsibilities:

• Own the technical delivery of part of a project
• Take ownership of deliverables and direct the team in choosing the best possible approach
• Engage with different cross-functional agile teams and stakeholders to understand business and technical requirements
• Lead technical teams of 5-7 members and mentor junior members.
• Define robust architectures using Cloud services and direct the team accordingly
• Be a subject matter expert for any Data Engineering tool/technology
• Independently perform technical responsibilities and oversee the work of other team members
• Develop, construct, test, and maintain solutions architecture
• Identify ways to improve data reliability, efficiency, and quality
• Build and automate data systems and pipelines
• Improve data reliability, efficiency, and quality
• Participate in code reviews, peer feedback sessions, and continuous improvement initiatives
• Assist in the development and maintenance of data process documentation, including data flow diagrams, data mapping documents, and test cases
• Continuously learn and stay current with industry trends, emerging technologies, and best practices in data engineering and cloud-based data warehousing

Requirements:

• Bachelors or Masters or PhD degree required (in data science, data analytics, Artificial Intelligence, Cloud Computing, Computer Science, Physics, Mathematics or related quantitative disciplines)
• Be an excellent Python and SQL developer
• Minimum 5+ years of experience in Big data tools like Hadoop/Spark or their cloud equivalent.
• Minimum 5+ years experience in Cloud technologies like AWS or GCP. Cloud Certifications, especially in GCP or AWS preferred
• Experience with advanced Warehouses like Snowflake, Athena, and/or Redshift. Snowflake Certification preferred
• Experience with Kubernetes and Microservices architecture
• Expreience with Kafka producers, consumers services
• Familiarity with Jira & Agile frameworks like SAFE is a plus
• Experience in owning the technical delivery of part of a project
• Ability to deliver to short timelines and under pressure
• Desire to become a technical SME
• 8+ years of industry experience
• Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams
• Eagerness to learn and stay current with industry trends, emerging technologies, and best practices in data engineering and cloud-based data warehousing

Contact