Senior Data Engineer
Senior Data Engineer
New York City, New York, USA - Permanent
Job Description
Our client is an early stage startup with the aim of making a big impact in the Financial Services AI space. They are building systems that can automate the most complex knowledge work in the world, e.g., financial analysis, research, due diligence, and more.
They are already working with some of the largest and well established financial institutions and hedge funds around the world. With a waiting list of clients ready to jump on their platform.
They are attempting to solve problems like:
•Creating financial research that's worth paying attention to: aggregating, analyzing, and producing insights from real-time information. Say goodbye to equity research.
•Dealing with the most sensitive data in the world: client data from the largest financial services companies on earth.
•Working past the edge of published AI research: tackling problems beyond the complexity of existing AI benchmarks.
•Unsolved product, architectural, and business problems: natural language interfaces, prohibitively expensive evaluation of models, massive marginal costs, versioning/training/segregating models per task, client, and so on.
What you will do:
As a Data Engineer, you will help build out real-time data pipelines for millions of unstructured financial documents to feed the financial LLM.
You will work closely with the AI and Product engineers to build scalable and reliable data pipelines and models. It’s cutting-edge data engineering at the AI frontier.
Please Note - This role requires you to be in the office 5x/week.
Must Have Skills:
•5+ years of industry experience as a data engineer
•Highly proficient with Python and SQL, and an intuitive understanding of multi-threading, multi-processing, asyncio, and other concurrency primitives
•Experience with at least one of: Postgres, Snowflake or Elasticsearch
•Experience deploying and monitoring mission-critical ETL pipelines with large and heterogenous datasources
•Experience working with Apache Airflow
•Experience with AWS or other cloud environment
Nice to Have Skills:
•Experience at a hyper growth startup
•Financial Services work experience