Hamburger_menu.svg

Data Engineer

Industry: Media & Internet
Remote
Company size: 251-10K
Full-time/Part-time

Apply as Data Engineer

Check out the best jobs for March 2024here

Find remote software jobs with hundreds of Turing clients

Job description

A U.S.-based company that has managed to develop a beloved social platform that enables users to effortlessly connect with each other, is looking for a Data Engineer. The engineer will be responsible for enhancing the data pipelines that produce training datasets for the fundamental ranking models. The platform enables users to build and join communities for like-minded people and bond over common interests. The company has managed to secure more than $1.1bn during their Series F round of funding. 

 

Job Responsibilities:

  • Create and implement data pipelines that quickly extract, convert, and load (ETL) information from a variety of sources into BigQuery
  • Using Apache Airflow, create and manage workflows
  • Create DAGs (Directed Acyclic Graphs) to organize and schedule data processing operations, guaranteeing timely and reliable operation of data pipelines
  • To speed up the process of adding new features to the datasets, create Python scripts
  • Drive the operations, architecture, and execution of relevant technical and product strategies for one of the biggest sites in the world
  • To ensure that data is structured appropriately for effective querying and analysis, create and maintain data models and schemas within BigQuery
  • Find and use performance-enhancing techniques, such as indexing, partitioning, and caching, for both BigQuery queries and Airflow processes
  • Implement data quality controls to assure the reliability and accuracy of the data
  • In order to proactively identify and address any pipeline or job issues, set up monitoring and alerting systems
  • Put access controls and security mechanisms in place to safeguard sensitive data in BigQuery
  • Establish permissions and roles for different teams and users
  • Add information to the data pipelines from many sources and platforms, such as APIs, databases, logs, and third-party services
  • Use Airflow and other automation tools to automate ETL processes to boost productivity and decrease manual involvement
  • Maintain data governance and adherence to pertinent data standards and legislation, which includes data anonymization, data retention guidelines, and data privacy protection
  • Collaborate closely with data analysts, scientists, and other key players to comprehend their data requirements and assist with data-related projects
  • Determine and fix any problems with the data pipeline, performance bottlenecks, and other potential technical difficulties
  • To promote knowledge sharing and the onboarding of new team members, keep documentation for data pipelines, ETL procedures, and workflow setups
  • Keep up with the most recent BigQuery and Airflow innovations and continually enhance your data engineering methods

Job Requirements:

  • Bachelor’s/Master’s degree in Engineering, Computer Science (or equivalent experience)
  • At least 5+ years of relevant experience as a data engineer
  • Extensive experience working with BigQuery, Airflow, and Python
  • Familiarity working with Spark is desirable
  • Strong understanding data sources, data transformation requirements, and data modeling
  • Extensive familiarity with BigQuery, including its architecture, recommended practices, and optimization methods
  • Prior experience crafting intricate SQL queries to quickly process and analyze huge datasets
  • Excellent spoken and written English communication skills

Interested in this job?

Apply to Turing today.

Apply now

Why join Turing?

Elite US Jobs

1Elite US Jobs

Turing’s developers earn better than market pay in most countries, working with top US companies.
Career Growth

2Career Growth

Grow rapidly by working on challenging technical and business problems on the latest technologies.
Developer success support

3Developer success support

While matched, enjoy 24/7 developer success support.

Developers Turing

Read Turing.com reviews from developers across the world and learn what it’s like working with top U.S. companies.
4.5OUT OF 5
based on developer reviews as of February 2024

How to become a Turing developer?

Work with the best software companies in just 4 easy steps
  1. Create your profile

    Fill in your basic details - Name, location, skills, salary, & experience.

  2. Take our tests and interviews

    Solve questions and appear for technical interview.

  3. Receive job offers

    Get matched with the best US and Silicon Valley companies.

  4. Start working on your dream job

    Once you join Turing, you’ll never have to apply for another job.

Resume

Learn how to create a perfect resume

Turing.com lists out the do’s and don’ts behind a great resume to help you find a top remote software developer job.

Frequently Asked Questions

We are a Palo Alto-based 'deep' jobs platform allowing talented software developers to work with top US firms from the comfort of their homes. We are led by Stanford alumni and successful A.I. entrepreneurs Jonathan Siddharth and Vijay Krishnan.

After uploading your resume, you will have to go through the three tests -- seniority assessment, tech stack test, and live coding challenge. Once you clear these tests, you are eligible to apply to a wide range of jobs available based on your skills.

It is the combination of our core business model and values that makes us different from others. We provide full-time, long-term projects to remote developers whereas most of our competitors offer more freelance jobs.

No, the service is absolutely free for software developers who sign up.

We, at Turing, hire remote developers for over 100 skills like React/Node, Python, Angular, Swift, React Native, Android, Java, Rails, Golang, PHP, Vue, among several others. We also hire engineers based on tech roles and seniority.

Communication is crucial for success while working with American clients. We prefer candidates with a B1 level of English i.e. those who have the necessary fluency to communicate without effort with our clients and native speakers.

View more FAQs

Latest posts from Turing

Leadership

In a nutshell, Turing aims to make the world flat for opportunity. Turing is the brainchild of serial A.I. entrepreneurs Jonathan and Vijay, whose previous successfully-acquired AI firm was powered by exceptional remote talent. Also part of Turing’s band of innovators are high-profile investors, such as Facebook's first CTO (Adam D'Angelo), executives from Google, Amazon, Twitter, and Foundation Capital.

Equal Opportunity Policy

Turing is an equal opportunity employer. Turing prohibits discrimination and harassment of any type and affords equal employment opportunities to employees and applicants without regard to race, color, religion, sex, sexual orientation, gender identity or expression, age, disability status, protected veteran status, or any other characteristic protected by law.

Explore remote developer jobs

Check out the best jobs for March 2024here

Work full-time at top U.S. companies

Create your profile, pass Turing Tests and get job offers as early as 2 weeks.