Pipelines, migrations, real-time sync and automated web data collection.
We move and shape data: ingestion pipelines, ETL jobs, system-to-system sync and migrations, and automated collection of public web data.
Teams consolidating data from several systems or sources.
HealthcareWeb application
Took over an undocumented medical-claims web application and its data pipeline, fixing the claim-editing flow, implementing proper CSRF protection for automated submissions, and adding duplicate detection and encounter numbering. The pipeline was refactored onto an ORM so it can run against a remote production database.
FinanceSaaS
Built a subscription deal-flow platform that scrapes business-for-sale marketplaces every hour, matches new listings against each buyer's acquisition criteria and sends email alerts. The product includes user and admin dashboards, subscription billing and a mobile-first onboarding flow, and runs in production on the client's domain.
OtherWeb application
For a Canadian industry association we built a statistics dashboard that scrapes sector figures from public sources on a schedule, with fallback data and failure alerts so it never goes blank, and migrated its certification map tool to a rebranded, bilingual, mobile-friendly site.
E-commerceAPI/backend
Designed and built a customer segmentation engine for a multi-store e-commerce operator, scoring customers on recency, frequency and spend and grouping them into 11 marketing segments. Custom probabilistic data structures kept it fast on hundreds of thousands of records, with a concurrent server and API layer on top.
What is an ETL pipeline, and when does your business need one? A guide to extract, transform and load, the warning signs, and what reliable ones look like.
Engineering5 min read
