Senior Data Engineer
Senior Data Engineer
Montreal, Quebec, Canada
This job allows you to work remotely
Our client is one of the first networks devoted to helping B2B enterprise. Our first-of-its-kind Sales Analytics platform combines a proprietary, self-learning network with applications that is ready to use, data backed, and built on predictive analyses.
We are looking for a Senior Data Engineer to help the data engineering team with implementing automated processes to support new data sources integrations, expanding data transformation capabilities, enabling to programmatically monitor processes and data quality across multiple systems as well as facilitating data management reporting automation.
•Integrating data from various data sources (e.g CRMs, Messaging, Video conference, Telephony, Group Document management, Contract signature/management, etc)
•Writing in-house data back into source systems.
•Reporting data quality and processes execution alerts.
•Performing data extraction, transformation, cleansing and loading between different data layers and environments.
•Detecting customers settings and metadata at various source and target systems.
•Consolidating and loading relevant data for data quality, metadata management, data consumption, UI usage and processes performance related dashboards.
•Data archival, removal and reprocessing.
Must Have Skills:
•B.S. in Computer Science or related field and industry experience.
•5+ years of experience developing in object oriented programming languages, preferably in Scala.
•5 to 6 years of experience with Open Source, Hadoop ecosystem and Big Data technologies with Linux
•Experience with Python or Java is ideal
•Hands-on experience with the Hadoop Cloudera distribution
•Hands on experience with messaging system Kafka
•Experience with development best practices under continuous integration, testing and deployment in an AGILE environment involving source control tools such as GIT.
•Familiarity with database tools, Integration Architecture, Data Integration, ETL, Business Intelligence concepts and Big-Data solutions.
•Hands on experience with developing and supporting complex SQL and NoSQL
•Ability to maintain, refactor, improve, and test existing code to reduce technical debt.
•Knowledge and experience in integrations involving a variety of data providers.