Exploreture
Job Overview
Associate Data Engineer
We are looking for an Associate Data Engineer to join our team who will work on various data engineering projects. Your responsibilities include, but are not limited to choosing optimal solutions, then implementing, maintaining, and monitoring them. You will also be responsible for integrating them with the organization’s infrastructure or other 3rd party client infrastructures.
Responsibilities
- Design and implement scalable, reliable distributed data processing frameworks and analytical infrastructure.
- Be part of a team to define, design, and implement data integration, management, storage, consumption, backup, and recovery solutions that ensure the high performance of the organization’s enterprise data.
- Develop Structured Query Language (SQL), Data Definition Language (DDL), and Python or equivalent programming scripts to support data pipeline development, problem-solving, data validation, and performance tuning.
- Work with software engineers, DevOps, ML engineers, and data scientists to achieve the organization’s goals.
Requirements
- Broad knowledge of programming languages is required (Python/Scala).
- Understanding data warehousing solutions, relational database theories, and no-SQL databases.
- Good knowledge of new and emerging tools for extracting, ingesting, and processing large datasets (Kafka, Spark, Hadoop, DataBricks, or equivalent).
- Familiarity with Azure Data Factory and Azure Logic Apps is a big plus.
- Hands-on experience with Amazon Web Services is a big plus, and knowledge about other cloud systems (Azure/GCP).
- Understanding Dockerization and Kubernetes is a big plus.
- Understanding data modeling and design patterns is a big plus.
- Knowledge of web scraping technologies is a big plus (Selenium, Beautifulsoup etc).
- Familiarity with Linux.
- Excellent interpersonal, communication, and organizational skills are required.
Job Detail
-
Offered SalaryNot Specified
-
Career LevelNot Specified
-
ExperienceNot Specified
-
GenderBoth
-
INDUSTRYComputer and technology
-
QualificationNot Specified