Data Engineer - Quezon City
Quezon City, National Capital Region
Posted over 30 days ago
- Company:
- Dempsey Resource Management , Inc.
- Company Description:
- We are an executive search company engaged in sourcing and referral of professionals in various fields. We do assist our client companies in the sourcing of competent and qualified candidates to fill up various job positions in their organization for direct hiring.
- Contract Type:
- Contract
- Experience Required:
- 3 to 4 years
- Education Level:
- Bachelor’s Degree
- Number of vacancies:
- 1
Job Description
-Experience: At least 3 years in a data engineering role working on data integration, processing, and transformation use cases with open-source languages (i.e. Python) and cloud technologies.
-Strong programming skills in Python specifically for API integration and data libraries, with emphasis on quality and test-driven development.
-Demonstrated proficiency with big data storage formats (Apache Parquet, ORC) and practical knowledge of pitfalls and optimization strategies.
-Demonstrated proficiency with SQL
-Experience with data modeling:
-Relational modeling
-Dimensional modeling
-NoSQL modeling
-Working knowledge of IaC on AWS (CloudFormation or CDK)
-Working knowledge of AWS Services:
-Required: Glue, IAM, Lambda, DynamoDb, Step Functions, S3, CloudFormation or CDK
-Nice-to-have: Athena, Kinesis, MSK, MWAA, SQS
-Experience with orchestration of data flows/pipelines: Apache Airflow or Dagster
-Nice-to-have: Experience with data streaming (Kinesis, Kafka)
-Experience with Apache Spark
-Client-facing experience, multi-cultural team experience, technical leadership, team leadership
Key Responsibilities:
-Design, build, and maintain cloud-based data pipelines and workflows that support analytics and operational systems.
-Integrate data from various sources using APIs and cloud services.
-Develop clean, efficient, and test-driven code in Python for data ingestion and processing.
-Optimize data storage and retrieval using big data formats like Apache Parquet and ORC.
-Implement robust data models, including relational, dimensional, and NoSQL models.
-Collaborate with cross-functional teams to gather and refine requirements and deliver high-quality solutions.
-Deploy infrastructure using Infrastructure as Code (IaC) tools such as AWS CloudFormation or CDK.
-Monitor and orchestrate workflows using Apache Airflow or Dagster.
-Follow best practices in data governance, quality, and security.
- Salary:
- ₱70,000.00 Monthly
