Senior Platform Data engineer
Technology:
SQL
Python
AWS
NoSQL
Spark
Stack:
Full-stack
Type of Employment:
Full-time
Location:
Lisbon
About Project
Riskified empowers merchants and shoppers to realize the full potential of eCommerce by making it safe, accessible, and frictionless. Riskified global team helps the world’s most-innovative eCommerce merchants eliminate risk and uncertainty from their business. Merchants integrate Riskified’s machine learning platform to create trusted customer relationships, driving higher sales while reducing costs. Riskified has reviewed hundreds of millions of transactions and approved billions of dollars of revenue for global brands and fast-growing businesses across industries, including Wayfair, Wish, Peloton, Gucci, and many more. As of July 29th, 2021, Riskified has begun trading on NYSE under the ticker RSKD. Check Riskified Technology Blog for a deeper dive into R&D work.
About the Role
Big data analysis is at the core of our technology. Riskified’s broad spectrum of departments - From data science to performance analytics, to customer support and finance - All depend on fast and easy access to high-volume data. Our growth heavily depends on a system that can quickly and efficiently process data at scale. Our revolutionary approach for Data Science, a key component in our success, relies heavily on our Big Data team to digest, process and serve data produced by billions of events per day. Our key data pipelines include: Streaming events in near real-time from multiple high-scale production services into our data lake. Processing data for our complex model training flow. Data pipelines supporting complex fraud detection patterns, potentially preventing fraud on a scale of millions of dollars per day. Our Technological Stack includes: Scala/Spark, Databricks, Data Hub, Elasticsearch, Airflow, Snowflake, AWS, Kafka, Kubernetes, and more. As a Data Platform Engineer, you will solve complex problems that require a varied and multi-disciplinary skill set. You’ll be required to understand the bigger picture, design system architecture, build highly complex data flows, and manage multiple, multi-faceted projects at once.
What You'll Be Doing
- Be a significant part of our data platform engineering - help our fraud detection engine make sub-second decisions, support our model training pipeline that ingests terabytes of data and provides near real-time BI and analytics to our customers
- Architect highly scalable data solutions for diversified and complex data flows using Apache Spark
- Build and develop high-performance, near real-time ETL processes incorporating Apache Kafka
- Improve our scheduler workflow engine operation
Qualifications - Experience as a data engineer
- Experience working with NoSQL & SQL databases
- Development experience with Spark or similar frameworks
- Experience with data lakes on HDFS or cloud storage (S3 or similar) - Advantage
- Experience with Python/Scala/Java - Advantage
- Experience working with K8S/docker etc - Advantage
- Experience in AWS cloud - Advantage
- Health insurance with dental coverage
- Gym membership
- Monthly work allowance
- Meal allowance
- Cool office in Marquês de Pombal
Working conditions - Hybrid working model (2-3 days a week in the office)