Date Posted: 2021-10-08

Job ID: EB-9276945213
Category: Analytics & Data Sciences
Location: Houston, Boston, or Pittsburgh, 

Senior Big Data Engineer
150K Base Salary + 20% Bonus + 20% RSUs

Direct-Hire Full-Time (No C2C or C2H)
Boston, Houston, or Pittsburgh Preferred
60 to 70% Remote Option

Position Summary

Senior Big Data Engineer manages the uninterrupted flow of info by designing & maintaining data pipelines delivering data across our org. Building the automated data pipelines to ingest & prepare the data to meet our reporting & analytics needs. This includes building & maintaining the data structures & architectures for data ingestion, processing & deployment for large-scale, data-intensive apps. This individual must ensure optimal ETL (Extract, Transformation, & Load) solutions are developed by applying best practices to the data modeling, code development & automation.

Essential Job Functions

• As part of an agile team, design, develop & maintain an optimal data pipeline architecture using both structured data sources & big data for both on-premise & cloud-based environments.
• Develop & automate ETL code using scripting languages, ETL tools & job scheduling software to support all reporting & analytical data needs.
• Design & build dimensional data models to support the data warehouse initiatives.
• Assemble large, complex data sets that meet the analytical needs of the data scientist teams.
• Assess new data sources to better understand availability & quality of data.
• Identify, design, & implement internal process improvements: automating manual processes, optimizing data pipeline performance, re-designing infrastructure for greater scalability & access to information.
• Participate in requirements gathering sessions to distill technical requirements from business requests.
• Collaborate with business partners to productionize, optimize, & scale enterprise analytics.
• Collaborate with data architects & modelers on data store designs & best practices
• Provide off-hours support for all developed data pipelines in an on-call rotation.


• Bachelor’s degree in Computer Science, Engineering, Information Science, Math or related discipline
• Data engineering, data management or cloud certification is a plus

Experience/Minimum Requirements

• At least six (6) to eight (8) years of experience in a data engineering role or related specialty with demonstrated ability in data modeling
• At least two (2) years Data engineering experience on the Microsoft Azure, Amazon Web Services (AWS), or Snowflake
• At least (4) years of development experience on Spark platform
• Experience with Python programming. Additional programming experience with Scala is a plus.
• Experience using Extract, Transformation & Load (ETL) tools to build automated data pipelines, preferably Informatica (IICS)

Other Skills/Abilities

• Thorough understanding of relational, columnar & NoSQL database architectures & industry best practices for development
• Understanding of dimensional data modeling for designing & building data warehouses
• Excellent advanced SQL coding & performance tuning skills
• Experience with big data streaming technologies such as Kafka
• Experience with parsing data formats such as XML/JSON & leveraging external APIs
• Understanding of agile development methodologies
• Ability to work in a team-oriented, collaborative environment; good interpersonal skills
• Strong analytical & problem-solving skills; ability to weigh various suggested technical solutions against the original business needs & choose the most cost-effective solution
• Keen attention to detail & ability to access impact of design changes prior to implementation
• Self-driven, highly motivated & ability to learn quick
• Ability to effectively prioritize & execute tasks in a high-pressure environment
• Strong customer service orientation
• Ability to present & explain technical information to diverse types of audiences in a way that establishes rapport & gains understanding
• Work experience with geospatial data & spatial analytics is preferred

BrainWorks, founded in 1991, is a leading executive search firm that prides itself on delivering superior talent to drive business performance. Our consultants are experts in their practice areas, which include CRM, Direct / Database Marketing and Customer Sciences / Advanced Analytics.

The Relationship Marketing and Decision Sciences Practice Group places top performing candidates, from mid-level managers to C-level executives, with companies spanning from entrepreneurial start-ups to Fortune 500’s. Some of the business areas in which these candidates specialize include the following: Statistical Modeling, Advanced Analytics, Web Analytics, Digital Analytics, Customer Analytics, and Campaign Management.

When you partner with BrainWorks you will receive industry expertise coupled with quality, speed and results guaranteeing a great hire. We look forward to having the opportunity to work with you

To apply for this job please visit