Senior Data Engineer

Earnest Research

Manhattan., NY, US

Earnest is seeking a data engineer to help scale up our data infrastructure. You will be part of a data-driven decision-making culture and collaborate with software engineers in building out the data tools and processes to support the creation of insights that will drive our business. The role will involve design and implementation of the entire data pipeline, from capturing and storing disparate data sources to processing that data and making that data available to other team members. You will be working across the company to understand their data needs, and creating systems that provide consistent and complete information to help solve various business problems.


Maintaining and implementing tools and systems that ingest, transform, organize, and expose data insights
Collaborating with other engineers to help implement and design our next generation data warehouse system
Working closely with our data analyst team to gather technical requirements and provide support on analytics processes
Develop and maintain data pipelines, with a focus on writing scalable, clean, and fault-tolerant code to handle disparate data sources
Implement new product features and performance improvements to existing products
Help drive optimization, testing and tooling to improve data quality across the product line

Required skills:

2+ years of experience in data engineering or a related field
Proficiency in one of Python, Java, Scala, or a similar programming language
Experience with Hadoop and related technologies (Hive, Pig, Spark, Presto, Impala)
Strong SQL experience (MySQL, Redshift/Postgres)
Comfortable with source control (GitHub) and working in a Linux environment
Experience with handling and processing large data sets in a business environment
Understanding of structured and unstructured data design/modeling
Strong analytical, quantitative, problem-solving, and critical thinking skills
Excellent verbal and written communication skills
Additional preferred skills:

Experience with AWS tools, i.e. especially EMR, Redshift, Data Pipeline
Experience working with large volumes of time series financial data
Exposure to Data Science
Knowledge of machine learning and natural language processing
NoSQL experience: HBase, MongoDB
Familiarity with BI and analytics tools (e.g. Looker, Tableau)

More Careers at Earnest Research