Data Engineer



Data Engineer

Toronto,†Ontario †-†Permanent


Job Description

Our client is a custom consulting firm that designs and builds high-performance, enterprise solutions with accentuation on robust architecture, and a perfect user experience.​ They are looking for a Data (hadoop) engineers to add to their fast- growing 150 person team here in Toronto. If being part of a fast-paced agile environment is important to you, then you are in the right place!


-Provide expertise in a wide range of technical areas, including but not limited to Cloudera Hadoop ecosystem, configuration management, hardware and software configuration and tuning, software design, and development
-Design and development of data ingestion pipelines. Perform data migration and conversion activities.
-Develop and integrate software applications using relevant development methodologies and standards applying standard architectural patterns in consideration of critical performance characteristics and security measures
-Document and maintain project artifacts along with collaborating with Business Analysts, Architects, and Senior Developers to establish the physical application framework (e.g. libraries, modules, execution environments)
-Suggest best practices and implementation strategies using Hadoop, Python, Java, ETL tools
-Maintain comprehensive knowledge of industry standards, methodologies, processes, and best practices


Must Have Skills:

-A Bachelorís degree in Computer Science or a related IT discipline
-5+ years of experience in IT development
-Must have 3+ years relevant professional experience working with Hadoop (HBase, Hive, MapReduce, Sqoop, Flume) Java, JavaScript, .Net, SQL, PERL, Python or equivalent scripting language
-A track record of great teamwork and the ability to work independently and drive projects to completion
-In-depth knowledge of SCALA and Spark programming
-Demonstrable experience with ETL tools
-Knowledge of implementing and integrating web services
-Well-versed in software development methodologies (agile, waterfall)


Details:

Starting: ASAP