Know Your Tools!
Use them to find the simplest way to solve your problem and be more productive.
Welcome!
Hi this is John with this week’s Developing Skills - Skills for Developers looking to develop their careers.
🙏 Thank you for being one of the 10,561 software developers who have subscribed, I’m honoured to have you as a reader. 🎉
If there is a topic you’d like to see covered, please let me know by replying to this email📧
Tip of The Week: Know Your Tools And Use Them To Find The Simplest Way, Then Iterate
A few years ago I joined a team who were trying load test a RESTful API. They’d spent weeks trying to figure out how to do it. Investigating tools and trying to determine how they could use the tools to reflect real world usage of the API.
Unfortunately they were going nowhere fast. Everything they looked at seemed to make the problem more complex.
I suggested another approach. It was this, let’s dumb down the problem to the absolute core. I did this by asking a few simple questions:
Q: What are we trying to do?
A: We were trying to understand what throughput the server currently provided.
Q: Why did we care?
A: We wanted to know how many servers we needed to support the current and expected demand, so we could determine how much it would cost to run the service.
Q: How do we know what real-world usage looks like?
A: They’d spent days arguing over this. For me the answer seemed obvious, the service handled over 1M requests per day, there was a record of real-world usage sitting there in the load balancer log files. A quick AWS CloudWatch Logs Insights query would give us an exact period’s real-world usage.
Q: How can we stress the server?
A: The simplest way would be to replay some real-world requests, at a higher rate that the server typically received them.
Q:What’s the simplest tool we can use to do so?
A: It would be trivial to re-send the requests in the log file using curl. We could do that with cat, xargs and curl. We could even make it concurrent using the options in xargs.
So that’s what we did, in less than 30 minutes start to finish we had a simple solution that could replay a days’ worth of requests in minutes, writing nothing more than a simple query to get the logs and a bash command line.
That’s the benefit of knowing your tools and how to use them. You can quickly build a simple solution to a problem. Then once you start that learning you can iterate if needs be.
Featured Jobs:
This is a new section of Developing Skills where I’ll feature a couple of open roles with the aim of introducing you as a great candidate to a great new role. Here are three I thought looked interesting:
VP of Engineering at Counterpart (Remote USA)
Staff Software Engineer at OneSignal (Remote USA)
Senior Software Engineer, Growth at Speechify (Remote Asia or Europe)
Two Ways I Can Help You Level Up As A Software Engineer:
I write another newsletter, Coding Challenges that helps you become a better software engineer through coding challenges that build real applications.
I have some courses available:
Become a Better Software Developer by Building Your Own Redis Server (Python Edition) which guides you through solving the Redis Coding Challenge in Python.
Build Your Own Shell (Go Edition) which guides you through solving the Shell Coding Challenge in Go.
Often time the solution is sitting right in the command line. I consider myself lucky that, as a kid, I had access to a PC and got introduced early to Linux. I still remember recompiling drivers for Slackware, OpenSUSE, and later Ubuntu. Not because I couldn't dual-boot into Windows but because it was challenging. I learned most of my Linux knowledge before I picked a profession. 😃
This is a very important point, John! Thank you for sharing.