Posts on Hi, I'm Ben 🐚

What the semver!?

Wed, 10 Jun 2026 00:00:00 +0000

I’ve never used Elm properly, but since I found out about it a couple years ago, have been jelous of it’s automated versioning.

In theory at least (and from what I hear in practice too!), when you publish a package for Elm, it compares the type signature between your current and previous version, and determines from that, whether you are introducing a breaking change. The logic is pretty solid, if you have a fully typed language, the type signatures of your functions should indicate what calls will and won’t work, and therefore what is and isn’t a breaking change. I think it works along the lines of:

The worst and best laptop I ever had

Fri, 13 Mar 2026 00:00:00 +0000

I was recently reading about Apple’s new MacBook Neo. If you somehow haven’t encountered it yet, it’s a very cheap, but pretty competent MacBook. It made me realise that I owe a lot, to a really terrible PC I used to have.

It was about 5/6 years ago, I’d recently finished my masters degree and (probably related) had zero money. I needed a laptop, but couldn’t really afford anything with power.

Why melt?

Tue, 23 Dec 2025 00:00:00 +0000

There are only two hard things in Computer Science: cache invalidation and naming things.

It might not be the most obvious name for a function, but any time I use pandas melt method, it always brings a smile to my face. Polar’s unpivot is probably a more obvious and consistent name (no surprise there, polars has great naming!) but let’s be honest, it’s not as fun!

Anyway, I got really curious and decided to go deep on why pandas melt is called melt. Github took me as far back as the original function commit, but that doesn’t really make any reference to naming. Looking around a little bit more though, turned up that R’s reshape2 and before that reshape package have a melt function that predates the pandas one! Aha!

Data Testing in 1, 2, Something

Fri, 17 Oct 2025 00:00:00 +0000

I don’t get a lot of free time for development, but whenever I do, at the moment I’m plugging it into making data testing, and specifically, data contracts, as easy as possible!

Programming is messy enough anyway, but programming with data is extra messy, because your programme gets to keep changing every time it runs even if you change nothing. Luck you!

There’s a whole load of products that exist for data testing, but I built a new one anyway, it’s called Wimsey! Most existing tools for data contracts are prioprietary and non-open source, but even the ones that are often tied to open source products, that winds up making a huge impact on how they exist, they tend to want you to build an entire server and database to get them up and runner. Not enough people test data to start with, without us making it hard for them! That’s where Wimsey comes in.

Covering all bases

Fri, 08 Aug 2025 00:00:00 +0000

I while back, I made a few contributions to the Narwhal’s project, mostly helping get Dask support up and running. It was an awesome experience overall, not in small part because the lead maintainer Marco puts high five gifs of cats on pretty much every pull-request. One thing I hadn’t really seen before, was actual 100% coverage across the full codebase.

I’ve heard a lot of people advocate against 100% coverage, mostly on the principle that it doesn’t measure how good your tests are, or how much they cover the important edge cases that your code might hit. That’s all true of course, but after putting 100% coverage in place on some work and personal based projects, I’m completely sold on it- here’s why.

Hello Codeberg!

Tue, 24 Jun 2025 00:00:00 +0000

If you’re reading this, you’ve found me over on Codeberg! I’ve been wanting to start shifting projects over to Codeberg for a while now, and have been putting it off for various reasons. As of, well now, I’ve moved over - it’s always fun to play with new tools, and I’m pretty excited about Codeberg.

I should probably acknowledge that it’s at least in some part a move away from github. I don’t have a huge problem with the site, and it’ll continue being a place that a bunch of exciting open source stuff happen for years and years. But it’s hard to ignore its increasingly hostile, and occasianally exploitative position against open-source software and its community.¹

Planning to Fail

Fri, 16 May 2025 00:00:00 +0000

I read the hilarious “Please Please Please” Phorge entry today and it’s as hilarious as it is trauma inducing.

As somebody who spends a bunch of time working on libraries, I sometimes see feedback like this gem from the aforementioned entry:

Subject: help
X-Priority: 1
it dun work

There’s also the opposite side, where in my head an error stack is super expressive but I see this:

Subject: Strange Error

Not sure why but when I call this script:

Data Contracts as Therapy

Mon, 27 Jan 2025 00:00:00 +0000

I think I’ve heard people say data is the new gold at least twenty times. Having worked in data for a while, I’m pretty sure what they mean by that is that extracting and processing is a laborious and often violent process.

It’s possible that my take is inspired by working as a data engineer, and therefore being the victim of other people’s data continuously. I once, during my first few weeks at work, came in to find that multiple pipelines had failed because a huge data provider had suddenly replaced a bunch of values with “field-7”. Worst off, it wasn’t obvious that that’s what had happened, because the error was happening further down the line when it was being processed.

Fun & Torture with Github Actions

Wed, 13 Nov 2024 00:00:00 +0000

I’ve been working hard on a new project called Wimsey lately. I’ll save writing about the project itself for another day, but it got me thinking about github actions (or really any CICD workflow tool). And crucially, how my process for setting up a new CICD flow always winds up being:

Write a nice clean yaml config
Feel like everything is very neat and tidy
Push to a branch and try running it
It doesn’t work
Ammend, thinking I’ve fixed it, commit and push
It still doesn’t work
Repeat steps 5 & 6 at least 10 more times
Swear a bit
Repeat steps 5 & 6 at least 14 more times
Cry in front of my friends and colleagues
Repeat steps 5 & 6 at least 7 more times
It now works! Praise the lords of github actions
Vow never to touch it again

Setting up the CICD for Wimsey was no different, only this time somewhere between steps 6 and 11 I found a new step where I thought about the issues with this workflow, which I’m now converting into this blog.

Fun with Hy and Pandas

Wed, 02 Oct 2024 00:00:00 +0000

I don’t keep it much of a secret- I love functional programming. Or maybe I’m just burned form spending hours of my life chasing back inheritance to see where an object variable was defined. Either was, when I saw that hy lang v1.0 was release the other day, I was pretty kean to try it out!

One of the downsides of new or more experimental languages, is the lack of ecosystem while things take time to catch on. That makes languages like Hy or Clojure a way easier sell, since they bring everything you love about Lisps without needing you to give up on the libraries of Python or Java respectively.

Libraries over tools

Tue, 20 Aug 2024 00:00:00 +0000

Especially in the dark realm of data engineering, there’s a huge range of neat low-code/no-code UI tools. I don’t want to complain about those today, but I do want to talk about why libraries (as opposed to low-code UI) are really awesome.

Low code is good code!

One thing that I think get’s missed out, is that low-code can still be code. Plotly and dash is a really nice example of this. Here’s the code to make a bar chart:

An Imaginary Language

Wed, 08 May 2024 00:00:00 +0000

I don’t know where abouts we sit on the wave of yaml-domain-specific-languages. I really hope it’s the peak, and that things will simmer down. I like yaml a lot as a configuration language, but every time I was to work in a domain specific language pretending to be configuration, it sends chills down my spine.

A big part of it is that github actions and other similar yaml DSLs don’t have much tooling, the only way to run and test anything is committing. So I know that in 20 minutes my commit history will look something like this:

Learning Curves

Fri, 19 Apr 2024 00:00:00 +0000

People (sadly) don’t go around asking me my favourite text editor a lot. But if they did, I’d answer in a second, (“Helix!”)[https://helix-editor.com/].

But weirdly, I have worked a bunch in the past with people who are new to coding, and have recomended them editors, I’ve never recommended Helix to someone starting out.

That probably isn’t that wierd if you’re familar with Helix. If you’re not, it’s a lot like Vim or Neovim. You edit from the terminal (which I love) and you have different modes for writing, reading and selecting (which makes keyboard shortcuts very optimised for each of those). A lot of people have used modal editors, a lot of people love them, and a lot of people can agree that they come with a pretty steep learning curve when you first get started.

Empty Objects & Pandas' Pipe Method

Thu, 22 Feb 2024 00:00:00 +0000

The other day, I listened to the episode of Richard Feldmans’s Softward Unscripted with Greg Wilson. It’s a brilliant episode (in fact, the every episode is brilliant so you should probably check it out instead of reading this) and Greg has a lot of great insights around how we teach and discuss software engineering.

One thing that really got me thinking was when Greg pointed out that we are pretty bad at talking about design patterns and really bad at talking about design patterns with examples.

The Joy of Monads

Wed, 10 Jan 2024 00:00:00 +0000

I love taking some time out in December, but my code obsessed brain often results in me using it to learn about some new coding concepts. This year, I took to reading though Miran Lipovača’s awesome Learn You a Haskell for Great Good! which had previously sat on my bookshelf unread for a long time.

Probably one of the main reasons I didn’t approach it was that Haskell has a reputation for being a difficult language. I still think that reputation is pretty well earned, but more than anything, mainly because its such a big language. There’s a lot in Haskell, and it has a pretty complex type system. As a digression, I think this is what makes Rust famously difficult to pick up too. Rust doesn’t have very many “gotchas” compared to other languages, but that doesn’t stop the fact that there’s a lot of completely new concepts you’ll need to learn in order to pick it up.

Restricted Pandas

Wed, 06 Dec 2023 00:00:00 +0000

Bemoaning the design of pandas isn’t a very hot take - the success of polars which seems to have kinda rocketed out of nowhere, is probably down, not by a small amount, to just how many people are currently using pandas, but want it to be faster (always better right?) and also more elegant.

I’m not gonna say to use pandas over polars or vice-versa, but I do think that if you’re going to use pandas, joining the cult of method chaining with a reduced feature set makes life a lot easier.

Running Dask in Databricks

Thu, 02 Nov 2023 00:00:00 +0000

I should probably admit that there’s a bit of a contradiction between two thoughts that I have:

I really love spark
I really hate spark

Spark is one of the most powerful dataframe libraries on the planet. It can process multiple petabytes of data. But it’s also overkill and unwieldy for most jobs.

For smaller datasets, tools like Polars or Duckdb or pretty awesome - but for teams that sometimes deal with big datasets (but mostly don’t) they pose a problem. Namely, do you:

Coding Virtues

Fri, 01 Sep 2023 00:00:00 +0000

I have a pretty unusual and specific guilty pleasure. Well actually I have an entire music taste based on that premise. More specifically though, I have one that relates to programming: clean coding.

By that, I don’t mean that all my code is spotless or beautiful or anything, but some part of my brain really enjoys reading about clean code. I have a neat bookshelf of books on programming, with a healthy dose of books like Uncle Bob’s Clean Code, Fowler’s Refactoring and plenty more 90s joys that promised me they where going to solve all the problems I make for myself when writing programs. Actually, if I’m honest, I think it’s less about my own code, and more about fodder for when I don’t like something. Saying “I don’t like this” is rarely accepted, and never helpful, but somewhere along the way “this is a code smell” has become cannon.

Reinventing the Wheel

Fri, 25 Aug 2023 00:00:00 +0000

Good news if you work in software - impress your friends and colleagues with these magnificent sentences!

“Have we thought about scalability?”
“Let’s stop and think about what problem we’re actually trying to solve”
“We don’t need to reinvent the wheel”

You can say these at any point, in any meeting, without considering context and people will nod in agreement, and think you’re very clever. It’s not without good reason: scalability is normally important (though probably not quite as much as people think); it’s never a bad idea to refocus your thoughts and make sure haven’t lost sight of the task at hand; and if reinventing the wheel was a good idea, then step one of every project would involve writing the linux kernel.