We’re building software to solve business objectives! That sounds obvious, but it’s often easily forgotten.
Tech Investment (Amount)
It’s only important to keep up with the latest frameworks insomuch as it helps you achieve those business objectives. Startups often find themselves in the position of bouncing from one cutting-edge framework to another, spending more time on refactoring and infrastructure projects than actually building their products. They forget that the reason the new JavaScript library exists is so that developers can build software faster and more reliably; not just because switching to it is better. When a new technology can’t justify its existence by making the people using it more productive, we have a word for that: a toy.
Established businesses tend to have the opposite problem; investing so little in overall architecture that developing small features takes several times longer than it should. After a new toy has developed into the standard tool, productivity suffers if you don’t adopt it before it pushes your current tools into obsolescence. A lot of overhead and boilerplate develops as you spend more time and money to internally reinvent what the greater community has already built in the new technology.
Since hindsight is always 20/20, consider a technology that’s been around for a while: airplanes. If your parcel service ordered a fleet of Wright Brothers’ planes the day after their first flight, you’d probably have gone bankrupt; if you hadn’t figured out how to incorporate air travel by the 1930’s, you were on the road to irrelevance. (And let’s not forget to consider the possibility that the hot new front-end framework is actually the new BetaMax or Building a Website in Flash.)
The key is to stay somewhere in the middle. As a rule of thumb, I like to say that between a quarter and a third of development time should be spent doing stuff “for the internal team” and only indirectly benefits the end user... but the “right” amount really varies. One way to tell you’ve veered off into circular refactoring: update your resume — if you’re adding more acronyms than accomplishments, tack a little away from technological perfection. In the other direction: if you’re struggling to find people that “still” are familiar with the stack you’re using, it might be time to upgrade.
Examples of direct benefit to users:
- Fixed a bug
- Developed and tested a new feature
- Performance-tuned a feature that was noticeably slow
Examples of indirect benefit:
- Introduced a new unit test framework which will help find unspecified bugs in the future
- Improved our build system, allowing us to spend more time on development
- Upgraded to a new database that will scale better in the future
“Technical debt” is a very apt description. Like financial debt, the right amount is somewhere between zero and making minimum payments on everything. New technologies are long-term investments. If you’re taking on too much debt and investing too little, it’s unsustainable. If you’re rolling everything into low-yield treasuries, you’re crippling growth.
Tech Investment (How-to)
The amount of investment is probably less important than how it’s done. Massive refactoring-for-the-sake-of-refactoring projects rarely go well.
Let’s use an analogy from the physical world. Let’s say you wanted to switch our electricity infrastructure to use renewable energy sources.
Method 1: “Okay, so we use 1.21 gigawatts to power our country, let’s order 1.21 gigawatts of solar panels, hook them up, and then switch it over all at once!”
You laugh, but you’d be hard-pressed to find a software engineer who hasn’t seen a refactoring project handled this way. Let’s list what’s wrong with that approach: it’s expensive, it’s politically difficult, there will probably be massive problems on switch-day that you’ll be rushing to fix. It’ll take a long time, and there will be zero benefit until you’re completely done. By the time you are done, chances are that the solar panels you chose are long obsolete, and you’re already begging for the budget to get the newer and hotter model halfway through the project.
Method 2: “Coal plants are worse than natural gas, so let’s declare a moratorium on new coal plants. When a coal plant will reach its natural end-of-life, let’s be ready to replace it with solar, then. While we should generally try to steer fossil fuel funds toward solar and wind when possible, it’s only really a priority to cut over the worst offending plants, and the ones that are soon to need replacing for other reasons.”
If you’re trying to port your site to React, it’s counterproductive to focus on re-writing your Angular 1 code when you’ve still got MooTools hanging out somewhere else. It’s best to quarantine the MooTools, and then prioritize re-writes of those modules that had new features planned, anyway. If there’s no reason to rewrite a particular bit of MooTools code other than “MooTools is old,” maybe don’t bother. If history is your guide, React is not going to be the be-all and end-all of web development, so every step forward is a temporary one. Get used to never being “done.”
Plan for Flexibility
On the topic of never being “done,” this is true with regard to your engineering decisions, your business requirements, and your teams. The only constant in life is change — code, specs, and teams that can’t adapt are dead, too.
We’ve all — sort of — learned this. We all know that “waterfall” is old-school, a bad way of doing things, unless you’re in aerospace where fixing a mistake that makes it to the physical world costs millions of dollars (but even that’s being questioned these days). So, we all moved to Agile. Or, as most companies actually implement it, “waterfall but with a sloppy spec that changes after it’s coded.”
Code for Flexibility
There was a time when I would get a coding assignment, and immediately started typing out code. Most people get past that phase after a bit of experience; they first think about how they’re going to solve the problem, what API endpoints to create, the data model, sketch some stuff out… At some point I realized that I was making a mistake by asking myself “what’s the best way to implement this spec?” I began asking myself, “given that this spec is going to change, which parts are most likely to change?”
Your car gets 40 miles to the gallon, and your client is 400 miles away. How many gallons of gasoline do you need to have in order to travel there?
10 gallons?
Your client isn’t a gas station. You’re very embarrassed when it’s time to leave and your tank is empty.
That wasn’t in the specification!
I now try to design software around — not just the best way to solve the given problem — but the best way to solve any of a broad range of likely permutations of the given problem. For example, if I’m building a “profile” page with three tabs, I might start by writing a single database query that gets all the profile information. Let’s say it’s not fast but not slow — 100ms. In a similar amount of time and code I could have written three simpler queries to get it down to 60ms each, but I didn’t — because I know that before we ship, the designer is going to come back to me with a 4th tab. Or maybe one of the buttons on tab A moves to tab B. Or maybe something else changes as a result of something neither me nor the designer can foresee. I’ll save the optimization for when the product is ready to go live, or after it goes live, or maybe never—whenever the potential for changes seems safely low enough.
This prevents that awkward situation in which you’re frantically re-factoring code the week before the deadline because no one thought it should be that big a deal to “just move a button over a few pixels.” The solution might even be objectively worse when you consider only the given problem, but you’re rarely solving the given problem, are you? That’s a tough habit for us math-and-science people to break.
Design for Flexibility
This is just as true when you’re the one writing the spec. I organize specs around goals first — the overarching business objective that’s unlikely to change. Details come afterward and can often be filled in by the people closer to the ground, or filled in later. For example, “We’re going to need to create the concept of a fee-structure-type. Some clients might be charged a flat fee vs. a percentage vs. something else” will likely lead to very different code than “The software shall support both flat fees and percentage-based fees.” You might be an engineering manager and not running the sales team, but when you hear a salesperson say that our third client demanded a different fee structure in their contract than the first two clients, you might make the assumption that sales hasn’t yet compiled a full collection of fee structures that will be wanted.
This prevents the awkward situation in which you’re changing the spec on the engineer at the last minute, and the awkward situation in which you’re telling the sales team that a new fee structure means their client has to wait an extra two weeks to launch.
I also tend to apply a similar approach to things that aren’t requirements in the strict sense of the word, or that I don’t have a strong opinion on. I can tell the engineer that “this page needs to display a log of activity with X, Y, and Z events” without defining exactly what it’ll look like (assuming you’ve already got some sort of style guide). If the engineer can’t get it to look good, they can come back to me and I’ll try suggesting a few ideas, or go to a third person with better taste.
Manage for Flexibility
It’s not just your product that has to be flexible, it’s your team. This means that a few people should always be familiar with any given piece of code or functionality at any given time. If “your” code breaks, you don’t want to have to be the only person who’s able to fix it. While some managers consider the implied 24/7 on-call status to be a feature rather than a bug, they often get a rude awakening when someone eventually (or not so eventually, given that work/life balance) leaves the company and no one is able to pick up their projects.
For small teams, this means to avoid having a dedicated “front-end person” versus a dedicated “database person” — people can and should develop specialized expertise, but the database expert can’t be afraid to touch JavaScript, and the front-end guru can’t be ignorant of indexing. I like to make sure everyone gets a project using a technology a little out of their wheelhouse every now and then.
For a bigger team, the pitfalls are surprisingly similar — if you’ve ever seen a team with 100 SQL programmers, you’ve probably seen a team with 99 who wouldn’t go near that one critical stored procedure with a 10-foot pole.
Estimate for Flexibility
Once you’ve built flexibility into your work style, you can start to give more useful estimates on when projects will be completed. Businesses, of course, need to be able to plan in advance, which means as much certainty as possible about what products they’ll have available at what time (for sales), or what automation tools will be available when (for operations/staffing).
Just like with design and management, budgeting based around goals tends to beat budgeting based on specs. Most software projects go long not because it took longer to build something than you anticipated, but because of that thing you didn’t anticipate but then had to build, anyway. Remember, your end users, your sales team, your operators — they don’t usually care about the spec — they care about having a solution to their problem.
When you say you’re going to implement a spec, and that unexpected thing comes up, it throws off your whole estimate. “We’re going to have software that allows our admins to create new user accounts by January, but the admin tools likely won’t be done; we’ve planned a second phase based on feedback that we’ll get from the first,” will go over much better than, “we missed the deadline because stuff was added to the spec” or “we budgeted 3 months to do 6 weeks’ work because something always comes up” or “that feature from the spec didn’t make it because someone had a suggestion to add a different one.” Plus, in the mean time, you’ll be able to add new users that much sooner with the flexibility-accounted estimate.
This is more applicable to internal software than customer-facing software (sometimes it is better to hold off on consumer software until it’s perfect), but it’s worth considering there, too — especially when your customer is a business. They might want their accounting software to work as soon as possible so that they’re ready for that audit, even if the charting feature comes later.
On Code Golf
We do all know that “smallest possible number of lines (or characters) of code” isn’t a good objective to aim for, right? Remember, the reason why code reuse and every design pattern that encourages it evolved wasn’t to reduce the number of keys you have to press, but to reduce the amount of time it takes to produce software, and to create fewer opportunities for bugs when changing things (flexibility!).
The mentality that leads to code golf also seems, to me at least, to have a large overlap with the moods that lead to obsessive and counter-productive performance-tuning. Fight the instinct to scour your code for an unnecessary integer parse and remember: it’s always the database, or maybe the network. Penny-wise / pound-foolish is the phrase for someone who stands in the aisle calculating the dollar-to-yard ratio of dental floss before buying a spool, but then invests their retirement savings on a whim. If you spend five minutes changing foreach loops to for loops because they’re faster, that’s five minutes you’re not spending checking your indexes… and another hour you’re spending reading code that’s now too confusing too easily notice that you have an API call inside one of those loops that could be refactored into one request. It’s important to know the basics in that arena so that you don’t make a stupid mistake, but as long as you know the difference between parsing and casting, lists and arrays, etc, you probably didn’t make the stupid mistake in the first place.
Priorities, Approaches, and Seniority
As you advance and gain experience, you’ll ideally start approaching problems differently. It’s hard to consciously accelerate this process, but an example can’t hurt…
The task: “We need to pull X personal data for this list of 100 people.”
Junior
We could write a script that makes an API call in a for loop, we’d have to figure out how the authentication works… then paste the IDs at the top of the file.
Mid-Level
I should find out if we’re going to be asked to do this again; if it’s going to be often, we should make a quick page to paste the IDs and run the script without an engineer.
Senior
How big are the lists going to be? If they’re going to be big, maybe we should make it a background task with a queue so it doesn’t time out. Also, personal data? Maybe I should throw a permission check in there.
CTO
What are the legal limitations around pulling this data? We’ll need auditing to see who’s using it, and for what. We need some approval process to assign the permissions to use it. All of this considered, is it still a worthwhile investment to build?
Unfortunately, you can’t really skip steps here: the mid-level engineer is only able to focus on re-usability because they’re confident in their ability to build the thing in the first place, and so on…