Big Data Developer

Role overview: The developer will work with the team to research and develop server side components to solve complex data cleansing, entity resolution, network graph building and graph querying problems on a very large (Big Data) scale in a highly performant way. Key Accountabilities: • Designing and contributing to the implementation of complex algorithms, • Responsible for implementing Scala solutions on Hadoop (Spark) and GraphX (Spark GraphDB), development of Elastic Search components may also be required. • Complex data engineering pipelines, • Integrate solution with other components of the wider programme, • Identify and develop the Technical Design documents, • Provide prioritization, planning, estimating, work assignments, quality assurance, • Working closely with Project Managers, Technical Leads, Data analysts, server and web developers, QA engineers, and Business Analysts throughout the project lifecycle. Requirements: • Development experience in Scala; desired – Hadoop & Spark, • Strong development experience in big data processing systems, • Overall a good general technical knowledge, • Experience in social network analysis, • Experience on Google Cloud Platform – nice to have, • Skilled at generating actionable insights from data, • Strong mathematical background, • Developing, iterating and production sing complex data models, • Elastic Search/Lucene • Hive/SparkSQL. Key Accountabilities: • Designing and contributing to the implementation of complex algorithms, • Responsible for implementing Scala solutions on Hadoop (Spark) and GraphX (Spark GraphDB), development of Elastic Search components may also be required. • Complex data engineering pipelines, • Integrate solution with other components of the wider programme, • Identify and develop the Technical Design documents, • Provide prioritization, planning, estimating, work assignments, quality assurance, • Working closely with Project Managers, Technical Leads, Data analysts, server and web developers, QA engineers, and Business Analysts throughout the project lifecycle.