Staff Software Engineer | Ingestion, Lakeflow Connect

The highest experience bar in today's edition, and the work justifies it. Databricks is building Lakeflow Connect to extract data out of transactional systems while putting minimal load on those production databases, using techniques such as change data capture and log parsing. That is database internals work: you are reading write ahead logs and replication streams rather than issuing queries. The posting asks explicitly for engineers with core database internals experience who want to stay hands on, and describes delivering a highly scalable, available and fault tolerant engine processing hundreds of terabytes daily across thousands of customers. Low level systems debugging is named as part of the job. Stated bar is fifteen or more years of industry experience building and supporting large scale distributed systems.

Published requirement is 15+ years of industry experience building and supporting large scale distributed systems, which is the highest stated bar in today's edition. The team is Lakeflow Connect and the posting is direct that it wants engineers with experience in core database internals. What the team is solving: extracting data from OLTP systems while imposing minimal load on those production systems. The named techniques are incremental data capture and log parsing, which in practice means change data capture built on database write ahead logs and replication protocols rather than periodic query based extraction. Anyone who has run a query based extraction against a busy production database understands immediately why this matters, and it is a genuinely specialised problem that touches the internals of each source database differently. Stated impact and expectations: solve real business needs at large scale through software engineering; deliver a highly scalable, available and fault tolerant engine processing hundreds of terabytes of data daily across thousands of customers; and perform low level systems debugging and performance work. The posting also says it wants engineers who continue to be hands on and want to make a large impact on a problem important to the company, which reads as a deliberate signal that this is a building role rather than an architecture oversight role at this seniority. Location is Bengaluru. Honest fit guidance: fifteen years is a real filter and we would not encourage a stretch application at half that. But if your background is database engines, replication, storage engines or CDC systems, this is one of the few roles in India where that specific expertise is the central requirement rather than a nice to have, and those roles do not come up often. Good fit for a systems engineer with deep database internals experience who wants to stay technical.