Senior Data Engineer

Senior Data Engineer

Remote/Telecommute JobREMOTE / Toronto, Ontario, Canada -Permanent
This job allows you to work remotely

Job Description

As a Senior Data Engineer, you work closely with cross-functional technical teams to architect and build AI software solutions and the necessary infrastructure required for customizable and efficient Data Pipelines to supply the IoT ML automation solution. In addition, you maintain, optimize, and deploy new data ingestion, transformation, and ETL pipelines and strive for the long term, holistic data strategy of the business.

Key Responsibilities:

Build scalable, customizable, and efficient Data Pipelines on Distributed frameworks such as AWS/Azure/Spark/Databricks
Create data acquisition and preparation tools for data scientist team that assist them in building and optimizing our product
Build and/or configure the necessary infrastructure required for optimal Data Pipelines
Troubleshooting, tuning and optimizing Apache Spark applications
Build, maintain, optimize, and deploy new and existing data pipelines including ingestion, transformation, and other ETL solutions
Build analytics tools and reports that utilize the data pipeline to provide actionable insights other key business performance metrics

Must Have Skills:

Strong experience with Python, Pandas, PostgreSQL, Jenkins, and Docker
Proficient with end-to-end Jenkins automate CI/CD pipeline deploying to docker swarm or kubernetes
Solid experience with data modeling, data warehousing, and building ETL pipelines
Excellent SQL skills and experience working with relational databases
Experience with cloud services/frameworks such as Azure, AWS, Spark, Databricks
Experience with cloud orchestration: e.g., docker containers and kubernetes
ETL Experience with time-series and operations data

Key Competencies:

* Self-starter
* Experienced
* Strong oral and written communication
* Detail oriented
* Team player
* Intelligent
* Excellent analytical skills
* Pragmatic
* Customer focused


Starting: ASAP
Travel: 0%
Dress Code: Casual

Similar jobs in Toronto:

Similar jobs in other locations: