I'm an independent data engineer specializing in Spark and Databricks, currently scaling data pipelines for clients. Previously, I built pipelines at KnowBe4 and SafeGraph.
I also founded HireUp, a fullstack applicant management system serving over 200 businesses.
Masters & PhD, Mechanical Engineering, 2011–2017
Of course, I spent a bunch of time studying math and modeling complex systems with differential equations. But, the fun part was the simulations! "What happens when you heat metals to insane temperatures"? Let's simulate it using a compute cluster running 32 beefy machines that message each other using MPI, crunch numbers with C++, and store large amounts of data in hdf5.
Senior Software Engineer · 2018–2020
I joined right as their machine learning models and data processing were bursting at the seams due to rapid growth and a data explosion. I suggested we try Apache Spark. It was actually an introduction to the library for both me and the company! We rebuilt the pipeline to process over 20M users daily without tempting memory limits. The company IPO'd after I left. Too bad they didn't offer equity at the time.
Data Engineer · 2021–2024
This company is a data processing powerhouse! Over 100M rows of geospatial data cleaned, refined, deduped, matched, and indexed multiple times a day in a single pipeline. The kind of scale that demands mastery of Scala & Spark. I'm proud to say that I championed strict data validation systems and rolled out a premium product for parking lot data during my tenure there.
Founder · 2024–Present
HireUp started as a small three-month experiment into Shopify app development and has since grown into a standalone, full-service applicant management system. It handles everything from setting up a job board to interviewing candidates. Currently servicing over 200 small businesses, including a few brands you've probably heard of.
Open to data engineering work. If you need help with Spark, Databricks, or messy pipelines, reach out.