Data Engineering Technical Lead | Pipelines and Warehouse

A data engineering lead role at Paytm in Noida, building and running the pipelines behind India's largest payments business. The work is core data engineering: design and maintain scalable ingestion, processing and storage pipelines, consolidate data from many sources into a central warehouse or lake, and build ETL with proper validation and cleansing so the numbers can be trusted. You work with data scientists and analysts to turn their requirements into technical specifications, and you optimise for performance, scalability and reliability. Two details worth knowing upfront: the role carries a rotational on call responsibility for live service issues, and data architecture here has to satisfy security and compliance requirements. Stated bar is six or more years.

Minimum qualifications as published: a Bachelor's degree in computer science, computer engineering or a related field, or equivalent, and 6+ years of progressive experience showing strong architecture, programming and engineering skills. Technical requirements: a firm grasp of data structures and algorithms with fluency in Java, Python or Scala; strong SQL including the ability to write complex queries; and solid hands on experience with an orchestration tool such as Airflow. The posting separately notes java or scala and SQL as the working languages. Responsibilities: design, develop and maintain scalable data pipelines covering ingestion, processing and storage; build integration that consolidates data from multiple sources into a centralised warehouse or data lake; work with data scientists and analysts to understand requirements and translate them into technical specifications; protect data quality and integrity with robust validation and cleansing; optimise pipelines for performance, scalability and reliability; and develop and maintain ETL processes. Two responsibilities that are easy to miss in the middle of the posting and that materially change the job. First, you maintain operations of the live service as issues arise on a rotational on call basis, so this is not a build only role. Second, you are responsible for verifying that the data architecture meets security and compliance requirements, which in a regulated payments business is a real workload rather than a checkbox. The posting also asks that you learn fast and adapt at pace, which is consistent with the volume. Location is Noida, on site. Good fit for a data engineer with six or more years who wants ownership of a large pipeline estate and is comfortable carrying an on call rotation for it.