Director - Data Engineering Lead – Financial Services
-
Location
Hong Kong
-
Sector:
-
Job type:
-
Contact:
Warwick Pearmund
-
Contact email:
w.pearmund@hamlynwilliams.com
-
Salary high:
0
-
Salary low:
0
-
Published:
9 months ago
-
Expiry date:
2020-08-12
As a Data Engineering Lead, you will develop, maintain, test and evaluate Big Data solutions within our cloud-based Big Data Platform.
You should have 15+ years’ software development experience with agile methodology, continuous integration and automated releases.
Experience in engineering (commercial or open source) software platforms and large-scale data infrastructures (preferably on cloud) is absolutely essential.
The role:
- Building and Architecting a distributed and highly parallelised Big Data ingestion and processing pipeline which is able to process massive amounts of data (both structured and unstructured data) in near real-time.
- Evaluate and deploy data quality frameworks to measure and improve data quality across the
- Organisation
- Engage with internal and external stakeholders, both producers and consumers of data, to design data pipelines.
- Lead a team of Data Engineers and Site Reliability Engineers to deliver according to platform product roadmaps in a continuous delivery environment
Job Requirements:
Essential:
- 15+ years in Data engineering / Platform Engineering
- Demonstrable ability to write production code in Scalar Python
- Good understanding of, and hands on experience with, the latest data technologies
- Experience with container technologies (OpenShift, Docker or k8s)
- Comfortable in environments with strong Site Reliability engineering culture
- Proven leadership experience from within a financial services or pure technology company
- Knowledge in design and implementation of data infrastructure on AWS or GCP
Preferred competencies:
- Apache Spark and Apache Nifi
- Delivering data platforms
- In-memory data processing
- Implementation of data security capabilities such as encryption and anonymisation
- SQL