London, United Kingdom
Harrington Starr is partnering with a leading Financial Services company who are seeking an Elastic Stack SME to join their data services team in Poland, Krakow.
Successful candidates would be able to demonstrate experience setting up Elastic Stack environments around 40+ nodes, 5TB+ daily ingestion and 100+ users. Design and implementation experience is essential.
Things you will work on
- Building robust, reliable, automated data pipelines
- Programming in Java, Scala or Python
- Writing complex SQL on Hive, SparkSQL
- Work on Java Data Services to extract data from HBase, Elastic and serve in JSON
- Working on real-time pipelines, Kafka, Kinesis, Spark streaming, Apache Flink
- Real-time analytics on Elastic search
- Working with large data volumes, including processing, transforming and transporting large scale data using big data stack: M/R, Hive SQL, Spark etc
- Building custom data loads using scripting language Python or Shell script.
- Experience with Big Data stack of technologies, including Hadoop, HDFS, Hive, and Hbase.
- Design, build and launch new data models in production.
- Design, build and launch new data extraction, transformation and loading processes in production.
- Define and manage SLA for all data sets in allocated areas of ownership.
- Work with data infrastructure to triage infra issues and drive to resolution.