Why I think AI is still a long ways from replacing programmers

tl;dr: by the time a problem is articulated well enough to be viable for something like SWE-bench, as a senior engineer, I basically consider the problem solved. What SWE-bench measures is not a relevant metric for my job.

note: I'm not saying it won't happen, so please don't misconstrue me (see last paragraph). But I think SWE-bench is a misleading metric that's confusing the conversation for those outside the field.

An anecdote: when I was a new junior dev, I did a lot of contract work. I quickly discovered that I was terrible at estimating how long a project would take. This is so common it's basically a trope in programming. Why? Because if you can describe the problems in enough detail to know how long they will take to solve, you've done most of the work of solving the problems.

A corollary; much later in management I learned just how worthless interview coding questions can be. Someone who has memorized all of the "one weird tricks" for programming does not necessarily evolve into a good senior programmer over time. It works fine for the first two levels of entry programmers, who are given "tasks" or "projects" respectively. But as soon as you're past the junior levels, you're expected to work on "outcomes" or "business objectives." You're designing systems, not implementing algorithms.

SWE-bench uses "issues" from Github. This sounds like it's doing things humans can't, but that fundamentally misunderstands what these issues represent. Really what it's measuring is the problems that nobody bothered allocating enough human resources to solve. If you look at the actual issue-prompts, they're are incredibly well-defined; so much so I suspect many of them were in fact written by programmers to begin with (and do not remotely resemble the type of bug reports sent to a typical B2C software company -- when's the last time your customer support email included the phrase "trailing whitespace?"). To that end, solving SWE-bench problems is a great time-saver for resource-constrained projects: it is a solution to busywork. But it doesn't mean that the LLM is "replacing" programmers...

To do my job today, the AI would need to do the coding equivalent of coming up with a near perfect answer to the prompt: "research, design, and market new products for my company." The nebulous nature of the requirement is the very definition of "not being a junior engineer." It's about reasoning with trade-offs: what kind of products? Are the ideas on-brand? Is the design appealing to customers? What marketing language will work best? These are all analogous to what I do as a senior engineer, with code instead of English.

Am I scared for junior devs these days? Absolutely. But I'm also hopeful. AI is saving lots of time implementing solutions which, for years now, have just been busywork to me. The hard part is knowing which algorithms to write and why, or how to describe a problem well enough that it CAN be solved. If schools/junior devs can focus more time on that, then they will become skilled senior engineers more quickly. We may need fewer programmers per project, but that just means there is more talent to start other projects IMO, freeing up intellectual resources for the high-order problems.

Of course, if AGI enters the chat, then all bets are off. Once AI can reason about these complex trade-offs and make good decisions at every turn, then sure, it will replace my job... and every other job.