A nightmare before Christmas

Don't worry, this is not just another blog post about Black Friday. I'm also saturated of hearing so much about Black Friday, Cyber Monday and what’s more. The worst thing is that this is a fad that is becoming a tradition, and at least in Spain it doesn't make much sense. But you don't read this blog to listen to my complaints about foreign words and the adoption of traditions that neither come nor go for us, you do it because I bring you the technological and business vision regarding the world of testing. And whether I like it or not, Black Friday is a key date in the calendar.

Are these companies going to wait for the pre-Christmas shopping nightmare to fail their customers again?

A large proportion of online retailers close 50% of their annual sales during the week of Black Friday, which is why business, marketing and IT managers have this date almost firmly marked on their calendars. But what about quality assurance managers? Is such a significant date in the calendar just as important to them as it is to others?

The performance of systems that was taken for granted years ago no longer exists, especially because the technological architecture is radically different

We already know that, thanks to software testing, we can protect the business by ensuring the correct functioning of the websites where the sale is generated. In this way, every Black Friday, companies that have established QA processes do not have to experience tremendous losses and headaches when their websites stop working for hours, with all that this entails for conversions and product sales, as well as damaging the prestige of a brand with something as basic as the correct functioning of a website. One more year I ask you... are our recommendations going to go unnoticed once again? Are these companies that prefer not to listen to us going to wait for the nightmare before Christmas to fail their customers again?

In one way or another the performance of systems sooner or later affects all organisations

You may think that this article is not for you because your business has nothing to do with Internet sales, but don't stop reading just yet! Because sooner or later the performance of systems will affect every organisation, including yours. Obviously, the causality in the online sales example is pretty clear; if my site is down or slow, my customers can't buy, or the user experience is so bad that they go to the competition. In short, I lose money. In other industries, the link between poor system performance and the impact on the company's bottom line may not be so clear.

What I am trying to convey to you is that the performance of systems that was taken for granted years ago no longer exists, especially because the technological architecture is radically different.

Let's assume that your company is one of the 240,000 SAP customers in the world. You have nothing to do with online shopping, yet as your organisation grows, your SAP system gets slower and slower; the productivity of your employees is impacted (financial losses), or the SAP BPC consolidation process is so complex that it brings your systems to a standstill for hours (financial losses...).

These are just a few examples; I could give you many more in environments that are assumed to be "fast" simply because they are SaaS, such as Salesforce, and where more and more customers are asking for our help to optimise it.

The classic solution of investing more money in the DPC (or rather, more euros in AWS or Azure) is not the solution either, because of costs and because it often simply doesn't work. The complexity of the software and the increase in the volume of users interacting through different devices makes performance testing and system monitoring mandatory, and should be done from the beginning.

Customer: “I think we're going to hit it on Black Friday, my head is on the edge. Where do we start?"

This is a phrase we hear every year around this time, from a company that, like many others, only looks for a budget when they see their times is up. As I said before, if you want the Nightmare Before Christmas to be just a great movie, and not the harbinger of the next downfall of your system, follow these tips:

1. Don't leave it to the end - Shift Left is not just about starting functional testing and automation earlier in the lifecycle, it is also about thinking about performance from day 0. Traditionally, performance testing was done near the end of development, as its focus was on business processes from start to finish. However, with the adoption of Continuous Development (CD), Continuous Integration (CI) or Continuous Deployment (CD) strategies, it is possible to integrate performance testing even every time we build. Of course, you need a performance testing strategy (and we are experts on this point), otherwise there is no point in simply launching tests on a constant basis. Think about evaluating changes in the trend of the graphs you get, rather than looking at overall system performance. And don't mess around with thousands of users, it's often enough to generate a minimal load and evaluate changes between releases / builds / commits...
2. Component testing - Traditionally the industry has focused on positioning tools that "record" what the real user does, then play it back by simulating hundreds or thousands of connections and generate the load. That paradigm of working is no longer necessary. It is now possible to work directly against the components, either at the webservices level or at the database level. My recommendation is that you test at the unit level, not looking to generate load, but looking to generate enough activity to identify trends, especially when comparing data between releases.
3. Simulate real-world scenarios - Don't forget that the purpose of performance testing is to anticipate what will happen in production, to detect potential problems before real users do. Your test plan should contain the detail of what you expect in production. Before the big day (be it Black Friday, the monthly accounting close, or the launch of a campaign...) create test scenarios that simulate these conditions and don't forget that there is no value in generating load to simply "pull" the system, we all know how to do that, but it is not performance testing.
4. Think about your dependencies - With heterogeneous environments, it is increasingly common to rely on third parties (be they other departments or other companies such as payment gateways). Make sure you know their testing plans to verify that they meet your estimates; and if it is not possible to have this information, think about service virtualisation. This is an excellent option, not only when you cannot test the performance of an external platform, but also when the development of some of your dependencies is going at a different pace (for example, when they are not yet finished and you already want to test).
5. Monitor in production - Don't forget that the application lifecycle has the word cycle in it for a reason, it is iterative. The information you get in production should be used to plan better performance tests. It is common for many of the tools used for synthetic production monitoring to also support the scripts you have used during performance testing.

If you follow these 5 tips, you're already a long way towards ensuring your platform performs as well as your users expect it to. As I said at the beginning of this blog post, if your systems suffered during Black Friday, don't wait until you have a pre-Christmas sales nightmare, and call in the experts.