Empowering The Backend

Introduction

I’ve been involved in all kinds of web-based projects throughout my career so far. I hope that pattern continues & I even hope that someday, I can even go out of my comfort zone & deal with technologies beyond the web. Be that embedded systems programming for IoT devices or just building some silly smart home applications, whatever, it would be fun. 🙂

But let’s not talk about my fantasies here. Let’s talk about building some awesome & powerful applications using the Node.js runtime environment, writing TypeScript & taking advantage of concepts such as railway programming.

The Runtime

I’m sure you’ll hear how so many developers talk about how great Node.js is & how it’s super fast, etc. But on the other side of that coin, you’ll also hear a group of developers state how it’s fun, but it’s not great for anything with heavy CPU utilisation. While both arguments might have elements of truth, typically speaking, in my own personal experience, the runtime environment that you’re working with usually isn’t the problem in regards to any performance bottlenecks that you’re experiencing.

Too many times I’ve seen a performance bottleneck arise from someone writing a horrific query, by relying on ORM’s that for a very specific use-case may not be doing a very good job. Or sometimes they just use the wrong architecture for a given problem space, for instance, if you wanted to have the ability to build up a report against some very large data set, maybe even using the likes of a Hadoop cluster because it’s that big. In that kind of instance, you won’t want to implement the feature to deliver the report in real time, rather it would make more sense to refine your process to send the user an email that the report has been generated once it is generated, notify the user that such a report is ready rather than making the user potentially wait around 5+ minutes while the backend is building this complex report.

I’m a very big fan of JavaScript as an ecosystem, I like how simple the language is, provided you’ve worked out how to deal with the many different gotcha’s. I like how easy it is to just get started, I like how there’s a huge community, etc. I just like the ecosystem, not to mention it is cross platform. I’ve heard a lot of .NET guys mention that & honestly, my experience with .NET core has been that yeah, a lot of stuff works cross platform, but there’s still just enough that is locked to the Windows platform to make it annoying.

Speaking of .NET & the Microsoft stack, it’s a great stack, but I have seen scenarios where the .NET stack has performed terribly. Now this could be due to hundreds of reasons, for instance, all of the different, detailed number of gotcha’s to working with .NET. I can’t mention the number of times where I’ve tried doing a slightly different LINQ expression, but the performance difference has been mind-blowing. Or even comparing strings where you don’t care about the case, these are just a tiny number of things that you need to be aware of with the .NET ecosystem. Something that might look the same, or similar to some other code, you need to know the low level difference.

Personally, I don’t like that, all the more reason why I like Node.js, it’s just simple. Sure, maybe there are some libraries where the above applies, but in my experience, this isn’t really the case. I’ve never come across this issue with Node.js code that I’ve written, where I rewrite it in an ever so slightly different format or style, such that if you were to glance at them side by side, you might miss the difference. In an ecosystem like .NET, annoyingly, it can make or break the performance.

Clean Code

I’m a firm believer in clean code, maybe not in a religious sense, but more so in a pragmatic sense. I like to think, will I be able to make sense of this code 6 months from now. If not, then heck, it’s back to the drawing board, I even try to think about designing a database in a similar sense, I try to think about how I can abstract things as much as possible. Granted, I don’t agree with redundant levels of abstraction, but that’s a thought process that I use in attempt to reach a nice level of single responsibility.

For example, one thing that I’ve been doing as of late on some of my personal projects is using an NPM workspace, where I have one project that holds all of the different CQRS implementations & interfaces. I then have a project above that where I have all of my business logic, which consumes the CQRS code base. In my opinion, this is quite beautiful, since it isolates some of the potential performance bottlenecks related to queries or commands, but it also cuts off a tight coupling to the business logic too. I like the idea of implementing use-cases these days, since you can really think about the different business level problems.

For instance; when singing up to some application that has MFA, you might have the following:

• Validate the request.
• Create a user record in the database.
• Send a confirmation code via email.
• Send a confirmation code via SMS.
• Once the user has confirmed both code, update the customer record.
• Send an onboarding complete email to the user.
• etc.

Granted, that’s a super generic process. But in my humble view, this type of process helps you really drill down & think about the different areas of complexity here. For example, if it were me & I had complete control on how I’d design this, given that money & time are big constraints, as is the case for most startups. I’d keep it simple, I’d probably use something like an outbox pattern in my one & only database, where the web application would write records, but I’d then have a background application reading from this database, running them on a completely different server. I’d do this purely to ensure that there’s no performance bleed, if for whatever reason, the process to send an email takes some time, it wouldn’t impact the main web application.

You might want to do this using an Azure Function, or a logic app, or something similar. I’ve not worked a great deal with the likes of GCP or AWS in a commercial setting, but I’d assume they have similar product offerings.

But by thinking about the overall performance here, you’re really thinking about the customer experience. Last thing I’d want to do as a user of some system is to wait for some loading thing to go away as it’s sending me confirmation codes in the background, etc. I’d want it to be as fast as possible, I’d want it to feel as swift as one could hope for.

Clean Architecture

I’m not one to jump on hype or to agree with the masses because of group think, but it’s safe to say that I have become a bit of a fan of mono-repos in recent years. I have tried working with different approaches & I’ve got to be honest here, mono-repos just make life easier in my humble opinion. And thanks to the likes of NPM workspaces, it makes it that much nicer to work with mono-repos.

Again, on some personal projects, my NPM workspace might look something like this:

• apps
  • tasks
  • web
• libs
  • application
  • common
  • core
  • data
  • integration
  • security
  • storage
  • infrastructure

Granted, I like to really drive home that single responsibility concept, where yeah, you could maybe merge the likes of data & storage into one & call it persistence or something along those lines. But I like to split them apart since it means that if I’m storing something to the database, I know that it’s going to utilise code in my data project. If I’m going to be storing something in blob storage or via FTP or whatever, then I know that it’s going to utilise code in my storage project. I simply like the separation since you can predict exactly what code is going to be used for a given use case before you’ve even finished implementing or writing the specs for your use case.

Previously I mentioned how I separate my business logic from the individual queries or commands, I do this by having my CQRS code live within the core project, I called it core for the simple fact that I’m struggle with one of the hardest things in software engineering; naming things. But then all of my business logic lives within the application project, again, naming things is hard & I’m not an overly smart chap.

Types

Now regardless of your views on TypeScript, I think it’s fair to state that TypeScript typically does make for much cleaner code. Granted, it’s not perfect, since you have to have an extra step in your CI process, unless you’re using a runtime that natively supports TypeScript like Deno, or maybe Bun.

But all in all, just by having the benefit of working with the likes of generics, in my humble opinion, it just provides a nicer development experience. Even when things may or may not be nullable, since we have types, you know this without having to scroll in attempt to find a if null statement.

DDD

I have a mixed view with DDD, it certainly has it’s place in the world, no doubt. But at the same time, I do find that it can become overengineered quite quickly. This is why I have a somewhat unbiased view on the subject matter. I love DDD, I really do, but when people talk about specific ideas such as having strong types for entity ID’s & so on, that’s where I think aren’t you just making life harder for GC & the class loader? Do you really need that?

Of course, this is just an opinion, and I’m all for the higher level & more abstract ideas of DDD, it’s just there are certain areas I’m somewhat on the fence about. Not to mention how a lot of developers out there have made a mess of some of the terminology.

But I do genuinely & firmly believe that it’s all about context, if you’re building some application that can justify needing all the bells & whistles of DDD, then hey, go for gold. I think I like to take a more pragmatic stance than what you can achieve by going hardcore with DDD.

Not to mention, you really don’t want to use DDD for MVP or prototyping style code, that would be pure insantiy.

Railway Programming

Now my first real hands on exposure to railway programming was back with the release of Java 8, where they introduced the optional class, this changed my life. When I saw this, it just completely blew my mind, it was like a light switch went off, whereby the implementation felt even more loosely coupled, such that you didn’t care about the specific outcome of a given action. Be that a database query, or handling an exception making a call to a remote service, that didn’t matter so much, you could just return this optional object. The consuming code would just take the optional object & behave in a specific way such that it just cared about the outcome of the optional object. Since then, I’ve basically been using a similar pattern with nearly everything that I work with, for instance, you could have something that looks like this:

Now granted, this isn’t strictly railway programming in the traditional sense, but it’s a foundation in my humble opinion. You can get an idea on how you could potentially extend something like this so that consuming code could just respond based on the state of the result object. I thought I’d include the likes of the service result for the fun of it, since I quite like the idea of having a result object for the more domain focused areas within the code. Anything that then deals with the likes of HTTP might then care about the HTTP status, thus the service result, to me it’s a nice & simple way to isolate the different concerns & so on.

If you were to take the foundations of that code & if you wanted something similar to the Java Optional implementation, then you may very well want to look at something closer to this:

Conclusion

I find that in the real world, if you focus on the realistic performance bottlenecks before trying to point your fingers at your runtime, you’re likely to have a pretty good performing application. The runtime for the most part is a matter of preference in my humble opinion, yes there are edge cases out there when a given language simply isn’t the right tool for the job & yes, you might very well need as much performance as you can get your hands on. But that in addition to choosing the right architecture, alongside keeping the level of code hygiene relatively high, etc. You can’t go too far wrong.

I titled this empowering the backend, since most of this is a lot more applicable to the backend, at least in the web domain. It’s not like you can use a completely different language to JavaScript, HTML & CSS for the frontend, but that does also reinforce the idea that you shouldn’t simply blame the runtime when you hit a performance issue. There are some very clever people out there that have build incredibly front end applications that can do some pretty amazing things, things that simply weren’t possible 10 or 20 years ago.

Think about trying to keep the different layers within a boundary of some form, this can also apply to both the front end & the backend. I find in the current state of things, the likes of fetch requests within the front end ecosystem is just bundled in with UI logic, validation logic & business logic. There’s simply no need, you could easily create the likes of a HTTP client for some given API(s), then wrap those up against some interface, then have them injected into some use-cases where they implement your business logic. From there, that can then be consumed by the UI logic, in my humble opinion, it really is that simple.

Yes, it’s very easy to get carried away & simply run with the first thing that works. But think about tomorrow, one week from today, 5 years from today. Would that code be nice to maintain? Could you even maintain it? I’ve personally seen too many instances whereby I couldn’t make sense of the work that was done by someone else. They might have literally written that code over 5 years ago, but going through that pain has only drilled it into my own process that you should think ahead.

By using just some of the subject matters I’ve talked about in this article, I genuinely believe that it can at the very least point you in the right direction. Granted, there’s no one size fits all in our world & I don’t believe that you should stick to one idea, opinion or practice, context is always king.