Lead data engineer jobs in top US companies

Job Overview

We are seeking a highly skilled Senior Full Stack Engineer with a strong focus on backend architecture and expertise in artificial intelligence (AI) to join our dynamic team. The ideal candidate will have 5-7 years of experience in designing, developing, and maintaining robust  full-stack applications, with deep expertise in Python, data structures, and backend database interactions, API design, authentication systems, and AI-driven technologies. You will play a critical role in architecting scalable, secure, and high-performance systems, integrating AI capabilities such as Retrieval-Augmented Generation (RAG), vector databases, large language model (LLM) APIs, and more to power our innovative solutions.

Key Responsibilities

● Design and implement scalable backend architectures for full-stack applications using Python and related frameworks (e.g., Django, Flask, FastAPI).
●  Develop and optimize complex data structures and algorithms to ensure efficient data processing and storage.
●  Architect and manage interactions with relational and non-relational databases (e.g., PostgreSQL, MongoDB) and vector databases (e.g., Pinecone, Weaviate) to support application and AI functionality.
●  Design, develop, and maintain secure, efficient, and well-documented RESTful APIs and GraphQL endpoints, integrating AI-driven features such as RAG and LLM APIs.
●  Implement robust authentication and authorization mechanisms (e.g., OAuth, JWT, SSO) to ensure system security.
●  Collaborate with frontend developers to integrate backend services and AI-powered features with user interfaces, ensuring seamless end-to-end functionality.
●  Develop and integrate AI solutions, including RAG pipelines, LLM API integrations (e.g., OpenAI, Hugging Face), and vector database queries for enhanced data retrieval and processing.
●  Perform data labeling, classification, and model training for AI-driven applications, ensuring high-quality datasets and model performance.
● Conduct red teaming exercises to evaluate and improve the security and robustness of AI systems and backend infrastructure.
●  Write clean, maintainable, and testable code, adhering to best practices and coding standards.
●  Design, implement, and maintain CI/CD pipelines to automate testing, deployment, and monitoring of backend and AI-driven applications, ensuring rapid and reliable delivery.
●  Optimize application and AI model performance, troubleshoot issues, and ensure high availability and reliability.
●  Mentor junior engineers, conduct code reviews, and contribute to architectural decisions, including AI strategy.
●  Stay updated on industry trends, emerging AI technologies, and backend development practices to recommend improvements and innovations.

Qualifications

● Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field (or equivalent experience).
●  5-7 years of professional experience in full-stack development, with a strong emphasis on backend systems.
●  Expertise in Python and its ecosystems (e.g., Django, Flask, FastAPI) for building scalable applications.
●  Strong understanding of data structures, algorithms, and software design principles.
●  Extensive experience with database management, including SQL (e.g., PostgreSQL, MySQL), NoSQL (e.g., MongoDB, Redis), and vector databases (e.g., FAISS, Quadrant, Pinecone, Weaviate).  
●  Solid understanding of embeddings and how these work with vector databases
●  Proven ability to design and implement secure APIs (REST, GraphQL) and authentication systems (OAuth, JWT, etc.).
●  Experience with AI technologies, including RAG, LLM APIs (e.g., OpenAI, Hugging Face), vector databases, and model training/classification.
●  Familiarity with data labeling, preprocessing, and red teaming for AI model development and evaluation.
●  Knowledge of frontend technologies (e.g., JavaScript, React, Vue.js) to collaborate effectively with frontend teams.
●  Experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization (e.g., Docker, Kubernetes) is a plus.
●  Strong problem-solving skills and ability to work in a fast-paced, collaborative environment.
●  Excellent communication skills and a passion for mentoring and knowledge sharing.

Preffered Skills

● Experience with microservices architecture and distributed systems.
●  Knowledge of CI/CD pipelines and DevOps practices.
●  Familiarity with testing frameworks (e.g., pytest, unittest) and writing automated tests for both backend and AI components.
●  Understanding of AI security best practices, including red teaming and compliance standards (e.g., GDPR, OWASP).
●  Good understanding of AI techniques (e.g. (CoT, reasoning, MCP)
●  Contributions to open-source AI or backend projects or a strong portfolio showcasing relevant work.
●  Experience with frameworks like LangChain, LlamaIndex, or similar for building AI driven applications.

Interview Process

• 1-2 technical rounds with the client

Offer Details

• Full-time contractor (no benefits)
• Remote only, full-time dedication (40 hours/week)
• Required 4-6 hours overlap with Pacific Timezone
• Competitive compensation package.
• Opportunities for professional growth and career development.
• Dynamic and inclusive work environment focused on innovation and teamwork

Job Overview We are seeking a highly skilled Senior Full Stack Engineer with a strong focus on backend architecture and expertise in artificial intelligence (AI) to join our dynamic team. The ideal candidate will have 5-7 years of experience in designing, developing, and maintaining robust  full-stack applications, with deep expertise in Python, data structures, and backend database interactions, API design, authentication systems, and AI-driven technologies. You will play a critical role in architecting scalable, secure, and high-performance systems, integrating AI capabilities such as Retrieval-Augmented Generation (RAG), vector databases, large language model (LLM) APIs, and more to power our innovative solutions.  

Key Responsibilities

● Design and implement scalable backend architectures for full-stack applications using Python and related frameworks (e.g., Django, Flask, FastAPI).
●  Develop and optimize complex data structures and algorithms to ensure efficient data processing and storage.
●  Architect and manage interactions with relational and non-relational databases (e.g., PostgreSQL, MongoDB) and vector databases (e.g., Pinecone, Weaviate) to support application and AI functionality.
●  Design, develop, and maintain secure, efficient, and well-documented RESTful APIs and GraphQL endpoints, integrating AI-driven features such as RAG and LLM APIs.
●  Implement robust authentication and authorization mechanisms (e.g., OAuth, JWT, SSO) to ensure system security.
●  Collaborate with frontend developers to integrate backend services and AI-powered features with user interfaces, ensuring seamless end-to-end functionality.
●  Develop and integrate AI solutions, including RAG pipelines, LLM API integrations (e.g., OpenAI, Hugging Face), and vector database queries for enhanced data retrieval and processing.
●  Perform data labeling, classification, and model training for AI-driven applications, ensuring high-quality datasets and model performance.
● Conduct red teaming exercises to evaluate and improve the security and robustness of AI systems and backend infrastructure.
●  Write clean, maintainable, and testable code, adhering to best practices and coding standards.
●  Design, implement, and maintain CI/CD pipelines to automate testing, deployment, and monitoring of backend and AI-driven applications, ensuring rapid and reliable delivery.
●  Optimize application and AI model performance, troubleshoot issues, and ensure high availability and reliability.
●  Mentor junior engineers, conduct code reviews, and contribute to architectural decisions, including AI strategy.
●  Stay updated on industry trends, emerging AI technologies, and backend development practices to recommend improvements and innovations.

Qualifications

● Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field (or equivalent experience).
●  5-7 years of professional experience in full-stack development, with a strong emphasis on backend systems.
●  Familiarity in Python and its ecosystems (e.g., Django, Flask, FastAPI) for building scalable applications.
●  Strong understanding of data structures, algorithms, and software design principles.
●  Extensive experience with database management, including SQL (e.g., PostgreSQL, MySQL), NoSQL (e.g., MongoDB, Redis), and vector databases (e.g., FAISS, Quadrant, Pinecone, Weaviate).
●  Solid understanding of embeddings and how these work with vector databases
●  Proven ability to design and implement secure APIs (REST, GraphQL) and authentication systems (OAuth, JWT, etc.).
●  Experience with AI technologies, including RAG, LLM APIs (e.g., OpenAI, Hugging Face), vector databases, and model training/classification.
●  Familiarity with data labeling, preprocessing, and red teaming for AI model development and evaluation.
●  Expertise in  frontend technologies (e.g., JavaScript, React, Vue.js) to collaborate effectively with backend teams.
●  Experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization (e.g., Docker, Kubernetes) is a plus.
●  Strong problem-solving skills and ability to work in a fast-paced, collaborative environment.
●  Excellent communication skills and a passion for mentoring and knowledge sharing.

Preffered Skills

● Experience with microservices architecture and distributed systems.
●  Knowledge of CI/CD pipelines and DevOps practices.
●  Familiarity with testing frameworks (e.g., pytest, unittest) and writing automated tests for both backend and AI components.
●  Understanding of AI security best practices, including red teaming and compliance standards (e.g., GDPR, OWASP).
●  Good understanding of AI techniques (e.g. (CoT, reasoning, MCP)
●  Contributions to open-source AI or backend projects or a strong portfolio showcasing relevant work.
●  Experience with frameworks like LangChain, LlamaIndex, or similar for building AI driven applications.  

Interview Process

• 1-2 technical rounds with the client

Offer Details

• Full-time contractor (no benefits)
• Remote only, full-time dedication (40 hours/week)
• Required 4-6 hours overlap with Pacific Timezone
• Competitive compensation package.
• Opportunities for professional growth and career development.
• Dynamic and inclusive work environment focused on innovation and teamwork