Essential skills and experience
- Proven experience as a Data Engineer
- BSc/BA in Computer Science, Engineering or relevant field; graduate degree in Data Engineering or other numerical field is preferred
- Experience in data mining
- Awareness of machine-learning and operations research, statistical data analysis and data visualisation
- Familiarity with Scala, Java or C++ is an asset
- Experience using business intelligence tools (e.g. Tableau) and data frameworks (e.g. Hadoop)
- Experience of Agile development CI/CD pipeline
- Excellent communication and presentation skills and ability to communicate effectively with both technical and non-technical audiences
- Understanding of data connectivity to visualisation tools such as ODBC, JDBC, OLEDB etc.
- Comprehensive understanding of Regex Syntax, use of capture groups and experience of creating Regex pattern matching syntax
Nice-to-have skills and experience
- Understanding of computer networking (WAN), including network design, security and logging
- Experience of open source data processing/management tools such as NiFi, Spark, Kafka, Druid, Oozie Storm, Hive, HDFS etc.
- Understanding of Regex Syntax, use of capture groups and experience of creating Regex pattern matching syntax
- Experience of working with network security data sets, including Syslog, CEF, IDS/IPS Log etc.
- Familiarity with Red Hat Linux Operating System
- Able to write scripts using Python or Bash