Remote Spark data engineer jobs

We, at Turing, are looking for talented remote Spark data engineers who will be responsible for cleaning, transforming, and analyzing vast amounts of raw data from various resources using Apache Spark to provide ready-to-use data to the developers and business analysts. Get a chance to work with the leading Silicon Valley companies while accelerating your career.

Apply as Spark data engineer

Check out the best jobs for September 2022here

Job description

Job responsibilities

  • Build and optimize Apache Spark ETL pipelines
  • Deliver scalable, cost-effective and flexible solutions to clients
  • Participate in iterative, end-to-end application development
  • Build and optimize Apache Spark ETL pipelines
  • Deliver scalable, cost-effective and flexible solutions to clients
  • Participate in iterative, end-to-end application development
  • Keep up with modern software development best practices and lifecycle management
  • Use ETL tools to load data from different sources into the Hadoop platform
  • Communicate regularly in an efficient manner with customers and stakeholders
  • Create Java-based Spark jobs for data transformations and aggregations
  • Conduct units tests for Spark transformations
  • Implement data processing pipelines with Spark

Minimum requirements

  • Bachelor’s/Master’s degree in Engineering, Computer Science (or equivalent experience)
  • At least 3+ years of experience data engineering (rare exceptions for highly skilled developers)
  • Expertise in established programming languages like Python, Java, Scala, etc.
  • Bachelor’s/Master’s degree in Engineering, Computer Science (or equivalent experience)
  • At least 3+ years of experience data engineering (rare exceptions for highly skilled developers)
  • Expertise in established programming languages like Python, Java, Scala, etc.
  • Mastery of Apache Spark and different Spark Frameworks/Cloud Services like Databricks, EMR, Azure HDI
  • Experience with technologies such as Storm, Apache Kafka, Hadoop, etc.
  • In-depth knowledge of Cloud (AWS, Azure) as well as CI/CD and data visualization
  • Practical experience with containerization technologies and container orchestration using Kubernetes, OpenShift, Docker, etc.
  • Knowledge of technologies like Spark and Hadoop HDFS, Hive, Hbase with deep expertise in Spark
  • Fluency in English language for effective communication
  • Ability to work full-time (40 hours/week) with a 4 hour overlap with US time zones

Preferred skills

  • Comfortable with ETL concepts, SQL (DDL, DML, procedural)
  • Hands-on experience on change capture and ingestion tools like StreamSets, Informatica
  • Strong experience in source code repositories like Git, SVN and Jenkins
  • Comfortable with ETL concepts, SQL (DDL, DML, procedural)
  • Hands-on experience on change capture and ingestion tools like StreamSets, Informatica
  • Strong experience in source code repositories like Git, SVN and Jenkins
  • Working knowledge of NRT and associated tech stack - Spark, MemSQL, etc.
  • Understands data architecture, data profiling, and data quality
  • Knowledge of data warehouse databases like Teradata, Oracle, etc.
  • Familiarity with Unix and Shell Scripting
  • Knowledge of diverse industry and tools and data warehousing technologies
  • Hands-on experience to build and administer VMs and containers
  • Working knowledge on HashiCorp Vault Consul is desirable
  • Excellent communication and organizational skills
  • Professional Certificates in AWS, RHCE, DevOps will be a plus

Interested in this job?

Apply to Turing today.

Apply now

How to become a Turing developer?

Work with the best software companies in just 4 easy steps
  1. Create your profile

    Fill in your basic details - Name, location, skills, salary, & experience.

  2. Take our tests and interviews

    Solve questions and appear for technical interview.

  3. Receive job offers

    Get matched with the best US and Silicon Valley companies.

  4. Start working on your dream job

    Once you join Turing, you’ll never have to apply for another job.

Leadership

In a nutshell, Turing aims to make the world flat for opportunity. Turing is the brainchild of serial A.I. entrepreneurs Jonathan and Vijay, whose previous successfully-acquired AI firm was powered by exceptional remote talent. Also part of Turing’s band of innovators are high-profile investors, such as Facebook's first CTO (Adam D'Angelo), executives from Google, Amazon, Twitter, and Foundation Capital.

Equal Opportunity Policy

Turing is an equal opportunity employer. Turing prohibits discrimination and harassment of any type and affords equal employment opportunities to employees and applicants without regard to race, color, religion, sex, sexual orientation, gender identity or expression, age, disability status, protected veteran status, or any other characteristic protected by law.

Work full-time at top U.S. companies

Create your profile, pass Turing Tests and get job offers as early as 2 weeks.

Apply now

Apply now