Are Coding Interviews Even Remotely Testing?
There I am, taking a remote interview with a large software company. As a contractor, it’s something I’ve become relatively accustomed to, but this one is different.
After being a developer for over 20 years, I’m about to experience something new. Something I’m not looking forward to. In fact, I’m quite anxious about it. The 30 minute remote coding test, using CoderPad, on my web browser.
Gulp.
It Was Going So Well…
The interview was for a short term Java contract, using Spring Boot and AWS. I’ve had many interviews from both sides of the desk, so while I don’t relish the experience, I try not to get too stressed out about it. If it’s the right job, in the right place, our minds should find a place to meet.
This interview was more like one for a permanent role, with multiple parts, including a cultural fit meeting. The technical chat and questions seemed to go well, with smiles and nods. There was some drilling into specifics, with the result being mutually fruitful. While not exactly enjoying myself, it was relatively painless and I was getting time to talk about my experience.
Then came the remote coding test.
Given Covid-19 was still a concern, all of their interviews were being completed remotely. Instead of using IntelliJ, I was asked to familiarise myself with CoderPad, prior to the interview. Immediately, I was hit by what you couldn’t do with it. I could only have one file, I couldn’t use Mockito and JUnit is an ageing version.
I paused to ponder — how exactly am I going to demonstrate my capabilities here? How am I going to illustrate my mastery of Java or Spring, let alone AWS? How am I going to demonstrate how I can separate concerns, testing each one in isolation?
It turns out I’m not.
The session opens in my browser and I see the code, for the first time. The task is to take the existing code and then add some functionality to print out the data in the correct way. It doesn't seem like a particularly complex brief, so I try to settle myself into it.
Testing Time!
I quietly read through the written instructions, then hit run from CoderPad. About 14 warnings are spewed out, in a big wall of text. The interviewers assure me it will run fine and not to worry about them. I can't help but feel uneasy about this.
I start to think over the code. I can see that there is some looping element required. It looks like there is a recursive element needed too. Then I notice the odd cough on the line, then some breathing… it feels like they are literally hovering over my shoulder. Distracted, I read over the same line of code a few times and then start typing once more.
I hit run again, to see if the syntax is clean and another wall of text appears. This time it says there are errors. Errors where?!
I search the log, trying to find the cause of the problem. It is covered in warnings and I can’t see the key line in the poorly formatted output.
My mouth is dry. I lift my glass of water and gulp down hard.
Back at the editor, there is simple colour coding for syntax, but nothing to indicate the presence of an error. No squiggly underlines, no red text, no indicator next to the line number. Nothing.
I hear some more coughing and feel my anxiety growing. Navigating back to the console output, I carefully read it again, line by line. Through the fog of warnings, I finally find the affected line. The error message is terse, but there is at least a line number.
Scanning the code, it looks like it must be a variable type issue. Finally, I spot that I’ve called the wrong method on the object, so I change it and it looks like it should be returning a list as I expected. I run it again — now there are two errors!
A bead of sweat runs down my spine and I shift awkwardly in my seat.
Repeating the process, I still can’t see the errors easily through the crowd of warnings. One of the interviewers suggests that it may not always be a list returned. Looking at the code again, I spot that sometimes it is a list, but other times it is a string. Immediately, I’m embarrassed for not spotting it, but without an IDE it is hard to see.
Why is this function returning either a list or a string? The original code was using generics, but only partially — the warnings are all about raw data types. Great. If this was my code, I’d not be sharing it with anyone else, let alone using it as the basis for a test!
The Struggle Continues…
By this point, I’m starting to use instanceof checks, casts, temporary variables and all sorts of shoddy code to work around the poorly written input data structures. The code is a mess and every single missing bracket, angular bracket or brace isn’t flagged until I hit run… then I’m back to hunting the wall of text for the error message.
There are a few more helpful suggestions by the folks on my virtual shoulder, but each time they say something, it feels like a failure on my part. I try to stay calm. I keep working through the code and eventually it runs and produces part of what I expected. Small relief…
The output isn’t what I want though. A limb of the if statement isn’t ever run. Normally, I’d fire up a debugger, take a look at what the state is and derive why this is the case. Here, I’m flying blind. I’m back in time, 20 years ago, hacking away with EditPlus. I’m adding System.out.println traces to figure out which block is even running… old skooling it!
With the logic updated, I hit go, revealing a syntax error — a bracket, semi-colon, brace, parenthesis or something is missing, but which and where?! I search the screen, looking over the code again and again… I spot nothing.
One of the interviewers suggests I look at the red underline. Red underline? What red underline?! Everything looks normal on my screen, with no markers or anything. Is this basic editor not even showing me everything I should be seeing? Is my Internet connection playing up?
Mild panic is setting in. I blink, breath, then take another swig of water.
I check the log again, scrolling up and down trying to find the error message, but fail. Looking back at the code, a red line has materialised — the scrolling must have caused something to happen. I mention this across the video link, but I hear nothing back. I suspect they are sceptical, so I just soldier on. At least I know where the error is now.
The issue is fixed! It was a missing brace. The indentation wasn’t fully automatic in the UI and I must have added some code without adding an indent or the matching brace. I hit run again… the syntax error is vanquished, but the output remains the same.
My heart sinks. I'm out of time and out of luck.
What Was Learned?
I learned why most of us abandoned using a basic editor decades ago. Without all the lint checks, syntax warnings, code completion, type checks, code formatting, build error diagnostics and debugger, it is much harder to write good code. It is much harder to write any code. When the existing code is strongly typed and generating a pile of warnings, even more so.
Of course, we wrote code back then using similar tools, but it was something we learned to do. As with any tool, there is a technique and a process which works well. However, having better tools — power tools — still helps enormously.
What did they learn? I’m not sure.
On one hand, they may have been wondering what sort of joker I was. They could have expected me, with all my experience, to have wrapped it up quickly and easily.
On the other hand, they may have seen how hard it was to even use the tool to make simple code changes. They may have seen how I was having to battle against it, using old fashioned console output calls to navigate my way around the broken code. They may have seen that I couldn’t see the errors messages being generated through the fog of warnings. They may even have wondered why all these warnings were allowed to be there in the first place. Perhaps they were even mildly impressed that I didn’t get in a flap, slam the keyboard, swear, blame the tool or the test code.
The number of warnings played on my mind. Why did they leave them there? Was I expected to update the input, as well as define how it was output? Should they have mentioned that when they dismissed the warnings? Would I have even been able to fix the warnings and do the code in the allotted time, without having my usual IDE? Perhaps. Perhaps not. 30 minutes isn’t a long time.
What if I had breezed the test? Well, I would certainly have felt better. My ego wouldn’t have suffered such a blow. I would have successfully jumped through the hoops presented, giving me a warm, fuzzy, feeling inside. But what would it have proved beyond that?
Can years of experience be summarised in a 30 minute coding test? No, I don’t believe so. Did the test exercise deep knowledge of software architecture, design and development? No, it just touched on a limited aspect of implementation. Did it demonstrate how elegant components could be created, each separated neatly by concern, using a test driven approach to validate them? Hmm, no.
What Would Be Better?
If coding tests are an absolute requirement, they should be meaningful and appropriate. If it is for a senior role, define some basic requirements for an application, then expect a day or two to be spent delivering it. Ideally, it should contain similar problems to what developers will see in the role. This should be enough elbow room for a developer to flex their muscles and represent their abilities accurately.
However, unless the company is highly desirable, they may miss out on some great candidates. Even more so when other attractive companies are asking for less.
