Job description:

Duties and responsibilities:

• Responsible for Hadoop development
• Implementation including loading from disparate data sets, pre-processing using Hive and Pig.
• Scope and deliver various Big Data solutions
• Ability to design solutions independently based on high-level architecture.
• Manage the technical communication between the survey vendor and internal systems
• Maintain the production systems (Kafka, Hadoop, Cassandra, Elasticsearch)
• Collaborate with other development and research teams
• Building a cloud based platform that allows easy development of new applications

Big Data Engineer job description: Job requirements and qualifications

• 5+ years of experience as a Big Data Engineer or similar role
• 3+ years experience programming and/or architecting a back end language (Java, J2EE, Core Java)
• University degree in Computer Science, Engineering or equivalent preferred)
• Experience with Java oriented technologies (JBoss, Spring, Spring, Hibernate, REST/SOAP)
• Experience with non-relational & relational databases (SQL, MySQL, NoSQL, Hadoop, MongoDB, etc.)
• Experience with Spark, or the Hadoop ecosystem and similar frameworks
• Familiarity with various tools such as AWS, Mesos or Docker and an instinct for automation
• Creative and innovative approach to problem-solving