Big Data Engineer
Big Data Engineer is required to join one of the largest sectors in Malta, Financial Services. You will be at the core of the data eco-system identifying unstructured data sources, building map-reduce processes and making the data available for consumption through the data lake, both within the department as well as by other data consumers.
- Creating, optimizing and maintaining distributed, fault-tolerant and scalable data pipelines
- Testing and deploying new solutions to existing systems under established change control procedures
- Improving the Data processes and procedures continuously to accelerate delivery whilst raising quality standards. Conducting design and code review to ensure compliance with standards for production deployments
- Maintaining source control, records and documentation as appropriate in a regulated bank
- Providing support for the full stack of technologies and systems within the department to ensure SLAs are kept and working towards attaining expert level knowledge of core technologies used by the department
- Responding to production issues or incidents during and off business hours and providing data integration and expertise to the organisation as needed
- The role requires flexibility to work on an on-call basis as needed for production support.
Skills and Experience required:
- A minimum of 3 years’ work experience in application development or production support
- Working experience or good understanding of Data Lake concepts and technologies
- Practical experience using Big Data tools and technologies such as Talend for BD, Hive, Spark, Hadoop, HBase, Hive, and Pig
- Strong documentation skills
- Willing to conduct research and learn new technologies