Data Engineer

A Silicon Valley-based company with a proven track record of helping scientists and engineers from some of the world’s most influential companies to better understand various properties of their new products and technologies is hiring a Data Engineer. The engineer will design, implement and operate high-value data storage and analytics solutions on the platform. The company offers in-house measurement and imaging of advanced materials and strategically partners with customers and clients to solve development challenges, improve products and increase yields. This position requires a full-time, long-term commitment and a minimum of 4 hours overlap with PST.

Job Responsibilities:

• Be responsible for the data platform implementation
• Modify sophisticated functional and technical specs into detailed architecture and design
• Utilize the available data to deliver business insights and customer solutions
• Craft out unique pipelines to provide services that will impact functional and operational decisions
• Bring data requirements together through collaboration with scientists and engineers 
• Work actively on building the data pipelines and achieving high-quality data ingestion
• Design and develop systems that ingest massive amounts of structured and unstructured data 
• Maintain and manage data pipelines on AWS to enable AI & ML capabilities
• Integrate production-ready data workflows, including automations and CI/CD pipelines 
• Ensure adequate data quality and availability
• Carry out ad-hoc analysis and present results in a straightforward manner
• Research and select new Big Data tools, technologies, or frameworks required to provide requested capabilities
• Demonstrate efficiency in handling data, tracking data lineage, ensuring data quality, and improving discoverability of data

Job Requirements:

• Bachelor’s/Master’s degree in Engineering, Computer Science, Mathematics (or equivalent experience)
• At least 8+ years of relevant experience as a software or data engineer
• Prolific skills in building data or machine learning pipelines
• Track record of crafting and implementing operational data stores, as well as data lakes in production environments
• Profound skills in implementing AWS services, including AWS Glue, S3, Athena, DynamoDB, Lambda, and RDS
• Core competency with one or more relevant tools (Flink, Spark, Sqoop, Flume, Kafka, Amazon Kinesis)
• Solid expertise in developing software code in one or more programming languages (C#, Java, JavaScript, Python)
• Sound understanding of the modern distributed Machine Learning and Deep Learning frameworks such as TensorFlow, PyTorch, MXNet Caffe, and Keras
• Proficiency with DevOps, Continuous Integration, and Continuous Delivery
• Previous experience working in an Agile development environment