Description:

PayPay is seeking a Data Engineer for the DaaS team to support the rapid expansion of its product teams and the need for a robust Data Engineering Platform.
The DaaS team is responsible for designing, implementing, and operating the platform using cutting-edge technologies such as Spark, Hudi, Delta Lake, Scala, and AWS suite of data tools.
The main responsibilities include designing, developing, and maintaining scalable data ingestion pipelines using AWS Glue, Step Functions, Lambda, and Terraform.
The role involves optimizing and managing large-scale data pipelines to ensure high performance, reliability, and efficiency.
The Data Engineer will implement data processing workflows using Hudi, Delta Lake, Spark, and Scala.
The position requires maintaining and enhancing Lakeformation and Glue Data Catalog for effective data management and discovery.
Collaboration with cross-functional teams is essential to ensure seamless data flow and integration across the organization.
The Data Engineer will implement best practices for observability, data governance, security, and compliance.

Requirements:

Candidates must have 5+ years of experience as a Data Engineer or in a similar role.
Hands-on experience with Apache Hudi, Delta Lake, Spark, and Scala is required.
Experience in designing, building, and operating a DataLake or Data Warehouse is necessary.
Knowledge of Data Orchestration tools such as Airflow, Dagster, or Prefect is expected.
Strong expertise in AWS services, including Glue, Step Functions, Lambda, and EMR is essential.
Familiarity with change data capture tools like Canal, Debezium, and Maxwell is preferred.
Experience with data warehousing tools like AWS Athena, BigQuery, or Databricks is required.
Proficiency in at least one primary programming language (e.g., Scala, Python, Java) and SQL (any variant) is necessary.
Experience with data cataloging and metadata management using AWS Glue Data Catalog, Lakeformation, or Unity Catalog is required.
Proficiency in Terraform for infrastructure as code (IaC) is essential.
Strong problem-solving skills and the ability to troubleshoot complex data issues are necessary.
Excellent communication and collaboration skills are required.
Candidates must be able to work in a fast-paced, dynamic environment and manage multiple tasks simultaneously.

The position offers a full-time employment status with the option to work remotely from anywhere in Japan.
Employees enjoy super flex time with no core hours, typically working from 10:00 am to 6:45 pm.
Paid leave includes annual leave (up to 14 days in the first year) and personal leave (5 days each year).
The salary is paid annually in 12 installments, based on skills, experience, and abilities, with annual reviews and a special incentive based on company performance.
Additional benefits include social insurance (health insurance, employee pension, employment insurance, and compensation insurance), a 401K plan, translation/interpretation support, and VISA sponsorship with relocation support.