About the Opportunity
Our client is a forward-thinking financial institution committed to redefining banking through a tech-first, data-driven approach. They are building a digital bank from the ground up, applying lessons from some of the world's most innovative technology companies to deliver exceptional products and customer experiences.
They are seeking an all-round Senior Data Engineer to join their Data Services team. In this role, you will design, maintain, and enhance the critical data infrastructure that powers analytics and operations across the organization. This includes managing the data lake, operational databases, high-volume batch and real-time processing systems, and metadata repositories—all working in concert to deliver accurate, timely, and actionable insights.
You will collaborate closely with data scientists to structure schemas and design data models, work with product teams to integrate new data sources, and partner with fellow engineers to bring cutting-edge data technologies to life.
The Ideal Candidate
Our client values logical thinkers who balance respect for best practices with independent critical thinking. They seek professionals who are adaptable, capable of owning projects from end-to-end, communicate effectively in English, and thrive in collaborative, high-performing team environments. While deep experience in specific technologies is valued, they prioritize candidates who are eager to learn, grow, and fill any gaps on the job.
Key Responsibilities
Develop and Maintain Data Infrastructure: Build, optimize, and manage the data lake and its associated processing frameworks, ensuring reliable and scalable data ingestion from diverse sources.
Design and Orchestrate ETL Pipelines: Architect and implement robust data workflows using orchestration tools, moving data from source systems through various processing stages into the lake.
Enable Analytics and Data Science: Structure data schemas and design data models to support business intelligence, reporting, and machine learning initiatives.
Integrate New Data Sources:Work with product and engineering teams to onboard new data streams, ensuring seamless integration and data quality.
Champion Technology Evolution:Evaluate and introduce emerging tools and frameworks to continuously improve the data platform's performance and capabilities.
Required Experience & Skills
Core Data Engineering Expertise:Solid understanding of data lake architectures and experience working with columnar big data databases (e.g., Athena, Redshift, Vertica, Hive/Hadoop). Familiarity with Iceberg is a significant advantage.
ETL & Workflow Orchestration: Proven ability to design and implement ETL pipelines and manage workflows using tools such as Apache Airflow, Luigi, or AWS Batch.
Programming Proficiency:Strong Python skills, with hands-on experience using relevant libraries (e.g., boto3, pandas, pytest). PySpark experience is highly desirable.
Cloud & AWS Services: Practical experience with cloud environments—particularly AWS (Glue, EMR, EC2, S3, Lambda, IAM, CloudWatch) or equivalent platforms.
Containerization & Orchestration: Familiarity with Docker and Kubernetes (or AWS ECS/EKS) for deployment and scaling.
CI/CD & Version Control: Experience with CI/CD tools (e.g., CircleCI, Jenkins, AWS CodePipeline) and solid git practices (branching strategies, collaboration workflows).
Agile Methodologies:Comfortable working within Agile/Lean frameworks such as Scrum or Kanban.
Bonus Skills (Highly Valued):
Distributed messaging and streaming systems (Kafka, Pulsar, RabbitMQ)
Streaming processing frameworks (Spark Streaming, Apache Beam, Apache Flink)
Metadata catalogue and lineage tools (Amundsen, Apache Atlas, Alation)
JVM languages (Java, Scala, Kotlin) and related frameworks
RDBMS/NoSQL databases (PostgreSQL, MySQL, DynamoDB, Redis)
BI tools (Tableau, Looker, PowerBI, QuickSight)
Logging and monitoring stacks (ELK, Datadog, Prometheus, Grafana)
Data privacy and security concepts (encryption, tokenization, Apache Ranger)
Experience Level
This is a senior position (L3–L4 level). Candidates with approximately 3+ years of relevant data engineering experience are encouraged to apply. What matters most is the quality of experience—our client values professionals who have worked in environments that foster continuous learning and skill development. Length of tenure alone is not a differentiator; they seek individuals who can demonstrate meaningful growth and technical progression throughout their careers.
Originally posted on Himalayas