Software Architecture Anti-patterns Thinking Out of the Box
One of the things that has lingered in my mind from the days of being an Engineering student at University is the definition of an Engineer given to us by our Dean at our orientation : “An Engineer is someone that tries to do things with the least cost, least time and least effort”. In order to achieve this we constantly question “Why?” something is done the way it is done. To think out of the box we need to expand this “Why?” mentality to even the assumptions or so called axioms in the problem domain. In this article I explain how we used this approach to come up with some fundamental architectural changes that greatly improved the performance of our an Application.
During the pandemic our team delivered an enterprise application that is accessed from all over the world. In our case the latency between the location of the server and the end customer was as high as 400 milliseconds at times. There are many ways to speed things up when you have high latency including local caches, CDN’s. However these solutions introduce multiple components to your architecture and increase the cost. The solution that we came up with only changed the way our application behaved and even reduced the overall cost of running the application.
Suppose we want to create a simple application that displays a list of Hotels that a user looks up using a search. Using the REST principles, you would “of course” create a endpoint that provides the list of “Hotel” Resources given a simple HTTP GET Request. We create our Rest Endpoint and create an user interface that calls that endpoint to retrieve the list of Hotels. Given the 400ms network latency between the client and the server the fastest we could deliver the Hotel List would be 400ms. So, if we were to deliver 50 Hotel Results or 500, given sufficient bandwidth you would think we should be able to show something to the user in 400ms right?
Sadly this is not the case, although TCP does a good job in Segmentation and delivering the chunks of the response in parallel , as the size of the content goes up, the roundtrips (in high latency networks ) to look for missing packets etc. tend to increase the overall latency. You can easily visualize this by looking at how websites load images using a developer tool. you will notice that even though the bandwidth utilization was not at its max for the same connection larger images load slower than the time to load the same size in smaller size images. If all the components in your path up to the server supported and agreed on using larger MTUs you might be able to keep the decrease in speed to a manageable level.
You might be thinking , well … I could overcome that if I used HTTP streaming instead of GET wouldn't I! This is a good approach if you wanted to display a list of Items to an User and didn’t necessarily want to wait till all the items arrived at the client side. If you are using spring you could map this streaming response to your PATCH endpoint (because we don't want to break HTTP and no one seems to care much about PATCH) for the service and use the StreamingResponseBody as below:
In our case we have some additional complexity here. Our application needs to be able to tell the user how many total hotels there are and also give the user some filters such as how many of them are Five star hotels etc. In order to get that information you would definitely have to wait till you get all the Hotels from the Server. We also have to call out in real-time to many different providers to get the list of Hotels and therefore only knew the total number of Hotels available once we had collected all the information from everyone else.
Or … Do we?
This is where we started to question what we had been doing for so long and had accepted as the norm. Out of the filters and counts that we give to the user only the prices and availability are truly dynamic. I mean how often does a Hotels Star rating or the different rooms and meals they offer change? So why keep dynamically requesting all this all the time. But we had to have the “Budget” filter, there was no way getting out of giving that as a feature. So we looked at using two different approaches, rather than using the better architectural pattern of generalizing into a single request. We found that by utilizing the same REST endpoint but using varying level of information we could keep the response size down such that the filters based on the less dynamic content could be easily given to the user. we could also give the counts of hotels available. The variable factor was the pricing for which we used a Machine Learning algorithm to determine the approximate pricing buckets thus giving a good enough estimate for the “Budget” filter. Once we were able to show the filters, hotel counts, and enough hotels sorted according to his need to fill his screen, fetching enough Hotels to support scrolling through to the rest became just additional background threads.
By not accepting what we had been doing for a long time as the limit and using an anti — pattern we managed to provide a fast responsive interface that worked over high latency networks without introducing additional costs to the customer.
