Data Engineer

Data Engineer

Toronto, Ontario, Canada -Permanent

Job Description

Our client is a division of one of the worlds largest mobile commerce platforms which produces more than 30 million transactions each month, the majority of these transactions occurring on mobile apps. They started out by offering mobile recharge and utility bill payments, and today they offer a marketplace to consumers on their mobile apps.

Their scale offers a very unique set of challenges and they are innovation around data/capabilities essential to scale even higher. They work with much longer horizons and accept trial and failure as part of building the right solution.

They are about solving problems that have little or no precedent. These challenges require lots creative thinking based upon a very deep understanding of how software works.

The ideal candidate is an ace problem solver and a learner at heart. You should have a very high degree of comfort in a programming language of your choice.

We offer flexible work schedule and you get your choice of tools (both hardware and software) and most importantly the opportunity to work on tough problems with the brightest minds around you.

What you are expected to do:

-Work directly with Data Analysts and Platform Engineering Team to create reusable experimental and production data pipelines
-Understand, tune, and master the processing engines (like Spark, Hive, Cascading, etc) used day-to-day
-Keep the data whole, safe, and flowing with expertise on high volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc)
-Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit
-Explore as many new technology options for data processing, storage, and share them with the team
-Develop tools and contribute to open source wherever possible
-Adopt problem solving as a way of life always go to root cause!

Must Have Skills:

Proficient in Java and/or Scala
Fluidly switch mindsets between Java/Scala and Python
Implement data platform components such as RESTful APIs, Pub/Sub Systems, Database Clients
AWS experience a plus


Application of toolsets in the Apache Hadoop ecosystem
Assembly and deployment of JVM applications
Database experience is essential.
Familiarity with reactive platforms and micro-services
Experience with R, Apache Spark, or Akka a plus


Starting: ASAP
Travel: 0%
Dress Code: Casual

Similar jobs in Toronto:

Similar jobs in other locations: