Senior Manager - AI & Data Engineering

We are looking for a Senior AI & Data Engineering Manager to lead the design, development, and scaling of enterprise-scale, production-grade data platforms and AI-ready data infrastructure that drive business value across Retail and Wholesale operations. This role bridges business strategy, data engineering, AI platform, and cloud technologies to deliver trusted, governed, and scalable data for Business Intelligence, Advanced Analytics, Machine Learning, and Generative AI applications.

Key Responsibilities

• AI & Advanced Analytics Enablement
• Lead the design, development, and deployment of enterprise AI, Machine Learning, and Generative AI solutions to support business transformation (e.g., pricing, promotion, automation, recommendation, demand forecasting, customer insights, and intelligent decision support).
• Build and manage scalable data pipelines supporting AI/ML model training, inference, feature engineering, Feature Store, RAG knowledge bases, and LLM applications.
• Develop reusable datasets, AI-ready data products, feature engineering pipelines, and semantic data models for AI Engineers, Data Scientists, and Business Analytics teams.
• Partner closely with AI Engineers to design, deploy, productionize, and scale AI/ML models, LLM applications, Agentic AI, AI Chatbots, Recommendation Systems, and Mobile AI applications.
• Design and maintain data ingestion pipelines for structured, semi-structured, and unstructured data from enterprise systems, APIs, databases, files, event streams, IoT devices, and third-party platforms.
• Develop scalable data pipelines supporting document ingestion, embedding generation, metadata management, vector indexing, and retrieval workflows for RAG applications.
• Identify opportunities to embed AI into business workflows and operational decision-making to improve efficiency, customer experience, and business value.

• Data Platform & Engineering Leadership
• Own end-to-end enterprise data architecture from source systems to Data Lake, Lakehouse, Data Warehouse, Feature Store, Semantic Layer, and AI-serving layers.
• Design, develop, and optimize scalable ETL/ELT pipelines supporting batch, micro-batch, streaming, and near real-time data processing.
• Design and maintain workflow orchestration for enterprise data pipelines using Databricks Workflows, Apache Airflow, or equivalent orchestration frameworks.
• Develop enterprise-scale Big Data solutions using Apache Spark and distributed computing frameworks.
• Design scalable logical and physical data models while optimizing database architecture for performance, scalability, reliability, and cost efficiency.
• Ensure enterprise data quality, governance, lineage, metadata management, observability, security, and compliance across data platforms.
• Implement automated data validation, monitoring, logging, alerting, and observability to ensure production-grade data reliability.
• Optimize SQL queries, Spark workloads, partitioning strategies, storage formats, and compute resources for maximum performance and cost efficiency.
• Analyze complex technical issues, identify root causes, troubleshoot production problems, and recommend infrastructure and platform improvements.
• Select, evaluate, and integrate modern data engineering tools, cloud technologies, and AI platform frameworks to support evolving business needs.
• Continuously evaluate emerging technologies in Big Data, Lakehouse Architecture, Data Platform Engineering, Cloud Computing, and AI Platform Engineering.

• AI Platform & Infrastructure
• Build and maintain enterprise AI data infrastructure supporting LLM, RAG, Agentic AI, AI Chatbots, Recommendation Engines, Intelligent Search, and Intelligent Automation platforms.
• Design and implement scalable AI data pipelines supporting batch, streaming, vector search, embedding pipelines, and Retrieval-Augmented Generation (RAG) architectures.
• Design scalable APIs, data services, and integration layers connecting AI applications with enterprise systems and digital platforms.
• Collaborate with AI Engineers to prepare high-quality datasets for LLM fine-tuning, prompt engineering, model evaluation, inference, and continuous model improvement.
• Support deployment and operationalization of AI products using DataOps, MLOps, CI/CD, containerization, and modern software engineering practices.
• Drive continuous improvements in platform scalability, availability, security, observability, resilience, and operational efficiency.

• Team Leadership & Capability Building
• Lead, mentor, and develop a high-performing team of Data Engineers, AI Engineers, and Analytics professionals.
• Foster engineering excellence through best practices in architecture design, coding standards, testing, code reviews, deployment, technical documentation, and software engineering.
• Drive architecture reviews, technical design reviews, engineering governance, and platform standardization across the engineering organization.
• Define engineering standards, technical roadmaps, platform architecture, and technology strategy aligned with business objectives.
• Collaborate closely with Product Owners, Business Stakeholders, AI Engineers, Data Scientists, Solution Architects, Infrastructure, and DevOps teams.
• Provide hands-on technical leadership with a strong engineering mindset and willingness to troubleshoot complex production systems.
• Promote continuous learning, knowledge sharing, innovation, and adoption of emerging technologies across the engineering team.

Requirements

• Bachelor's degree or higher in Computer Science, Computer Engineering, Information Technology, Artificial Intelligence, Data Engineering, Management Information Systems , or a related field.
• 6+ years of experience in Data Platform Engineering, Data Engineering, Big Data, or AI Platform development, with experience leading engineering teams.
• Strong experience designing and implementing enterprise-scale Data Lake, Lakehouse, Data Warehouse, or Modern Data Platform architectures.
• Expert proficiency in SQL, Databricks SQL, PostgreSQL , database design, and query performance optimization .
• Strong programming skills in Python.
• Hands-on experience with Apache Spark, Databricks, Delta Lake, Spark SQL, Unity Catalog , and distributed data processing.
• Strong experience designing and developing scalable ETL/ELT pipelines and workflow orchestration .
• Experience with streaming technologies such as Spark Structured Streaming, Kafka , or equivalent.
• Experience building enterprise-scale data platforms supporting AI/ML, Advanced Analytics, and Generative AI applications.