Data Engineer II

Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.Title and SummaryData Engineer IIOverviewWho is Mastercard?Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships, and passion, our innovations help individuals, financial institutions, governments, and businesses realize their greatest potential.The Mastercard Services organization is a key differentiator, delivering cutting-edge solutions used by some of the world’s largest organizations to make critical business decisions. Focused on innovation and scale, Services provides data-driven capabilities across consulting, analytics, experimentation, and risk management.Team OverviewAs part of Mastercard’s Data Platform & Orchestration team, you will contribute to building next-generation data platforms that are critical to our global data ecosystem.Our team develops and operates scalable platform capabilities, including:• Cloud-native infrastructure and application provisioning• Standardized CI/CD pipelines and engineering tooling• Reusable frameworks and data platform componentsThese platforms enable teams to build, deploy, and operate data-driven solutions efficiently and securely at scale.Role OverviewData Platform & Orchestration is seeking a Data Engineer II to design and build next-generation, cloud-native data platforms supporting Mastercard’s global data ecosystem.In this role, you will lead the development of scalable batch and real-time data pipelines, enabling efficient data processing across Data Lakes and Data Warehouses. You will work at the intersection of data engineering, cloud platforms, and distributed systems, contributing to high-impact initiatives and driving engineering excellence.This role is ideal for someone who thrives in a fast-paced, collaborative environment, enjoys solving complex data challenges, and is passionate about building resilient, high-performance systems at scale.Role OverviewAs a Data Engineer II, you will design and develop scalable batch and near real-time data pipelines that power Mastercard’s analytics and operational systems.You will work across data engineering, cloud platforms, and distributed systems, building robust data solutions on top of Data Lakes and Data Warehouses. This role is highly hands-on and requires strong engineering fundamentals, with opportunities to influence design decisions and mentor junior engineers.You will contribute to building cloud-native data platforms, enabling reliable, high-performance data processing at scale.________________________________________Key Responsibilities• Design and build scalable data pipelines and microservices using Java (Spring Boot), Spark, and cloud-native technologies• Develop high-throughput, low-latency data processing systems for real-time and batch workloads• Design and develop batch and near real-time data pipelines using Spark, Kafka, and Java-based frameworks• Build and maintain ETL/ELT pipelines for structured and unstructured data• Develop data processing solutions across Data Lakes and Data Warehouse environments• Contribute to stream processing use cases (Kafka, and optionally Flink or Spark Structured Streaming)• Ensure data quality, validation, and reliability across pipelines• Optimize data processing workloads for performance and scalability• Develop and deploy data pipelines on AWS, Azure, or GCP• Leverage cloud-native services such as S3/ADLS/GCS, EMR/Databricks, BigQuery/Redshift/Snowflake• Contribute to Infrastructure as Code (Terraform, CloudFormation, or equivalent)• Build solutions with high availability, fault tolerance, and scalability• Follow best practices for secure data processing and cloud resource utilization• Follow best practices in coding, testing, and CI/CD pipelines• Contribute to automation, monitoring, and observability of data pipelines• Develop reusable components to improve engineering efficiency and consistency• Participate in code reviews and design discussions• Collaborate with architects, product teams, and cross-functional stakeholders• Support production deployments and troubleshoot issues in distributed systems• Contribute to a culture of continuous improvement and technical excellence• Mentor junior engineers and share knowledge within the teamRequired Skills & Qualifications• Strong proficiency in Java (JDK 8+), OOP/OOAD principles, Python is a plus. Experience with Spring Boot, REST APIs, Spring Security, Hibernate• Hands-on experience with distributed systems, multithreading, and messaging systems (Kafka preferred)• Hands-on experience with Apache Spark (Core, SQL, or Structured Streaming). Experience with Kafka or similar messaging/streaming platforms• Understanding of ETL/ELT pipelines, batch and streaming architectures• Familiarity with data formats (Parquet, Avro, ORC)• Basic understanding of data modeling (star/snowflake schemas). Understanding of distributed systems and multithreading concepts• Hands-on experience with at least one cloud platform: AWS, Azure, or GCP• Experience using cloud storage and compute services (e.g., S3, ADLS, Databricks, EMR)• Familiarity with containerization (Docker) and basic Kubernetes concepts• Exposure to Infrastructure as Code tools is a plus• Strong SQL skills and experience working with relational and analytical databases• Exposure to Data Lakes and Data Warehousing platforms• Familiar