Data engineer with 8 years of experience (5+ in data engineering) building data pipelines with Python, PySpark, SQL, dbt and Snowflake / Databricks. Experienced in regulated enterprise environments, migrating on-prem and legacy systems to governed cloud platforms.
Migration of an on-prem platform to the cloud on Snowflake, running on AWS and orchestrated with Apache Airflow. Building and maintaining pipelines (incl. PySpark) to feed governed Data Products for reporting and AI use-cases. Dimensional modelling (dbt-core) to build consumer-aligned Data Products.
Built an end-to-end Databricks (PySpark and SparkSQL) platform on Azure for reporting, integrating legacy on-prem flows into a single cloud data shop.
Built Kimball dimensional models in the presentation layer (sourced from a Data Vault 2.0 schema) and migrated the dashboards and data sources from Qlik to Tableau.
Engineering and data science overlapped heavily in this role: built the data pipelines ourselves (experimental, so not yet scheduled), did the data cleaning and feature engineering, and developed predictive models (LightGBM) for AI use-cases such as smart shopping lists.
Modelled the portfolio (R, Excel) for internal use and regulatory reporting (e.g. Solvency II).