Caching might be one of the hard things in computer science, but wow, it's fun when it works! Let's dive into some cool new caching strategies!

Sometimes, old stuff is the coolest. In a world saturated with new technology every day, I find solace in learning something new about a technology I have been using for decades. The fact that even the building blocks of the technologies I use every day still have something new to teach me, just feels good for some reason.

This year, my team and I spent three months implementing a new version of a fairly popular website in Norway, and during that process I spent a bunch of times trying to make everything as fast, responsive and available as possible. Since we were using Remix to implement the new web platform, I started looking into more of the core web fundamentals. What I learned turned out to be incredibly useful for me – and hopefully it will be for you too.

This article will teach you two (fairly) new caching strategies that helped us save money, reduce load time by a ton, and greatly simplify our code.

I know I should know how caching works on the web, but ehm…

Before we dive into my new cool findings, I want to give a quick introduction on what caching is, and how caching works on the web.

Caching is a way to temporarily store data, so we can retrieve it quicker than we would from the original source. We do it all the time, often without noticing. Our CPUs cache data it needs within a millisecond or two, memory chips store cache data we need almost as quickly, and even our hard drives cache stuff we’ve accessed recently so that we don’t have to look it up from the storage unit itself.

When doing web development, there’s a few ways we can cache data as well. We can cache data in our JavaScript applications (hello useState), we can ask the browser to cache stuff for us, and we can use a globally distributed cache layer, called a Content Delivery Network (or CDN, for short) to cache documents and assets closer to the user than our origin servers. There are lots of other ways, of course, but these seem to be the most common ones.

When caching data on the web, we typically use something called caching headers to specify what we want to cache, and how we want to cache it. There are lots of fun cache headers to play with, but the most important one – at least for the strategies mentioned in this article – is the Cache-Control header. Headers are part of the request that’s passed between the server and the client, providing meta information (information about information) to the receiver.

The Cache-Control header has a bunch of functionality, but here are a few examples:



There are lots more to learn about this header, and I highly recommend diving into the MDN documentation for more details.

However, there are a few strategies that turned out to be great resources for us to reduce response time, server load and cost.

A few words about CDNs

The site we implemented had a CDN (content delivery network, remember?) between the site’s servers and the end user. A CDN is a network of globally distributed machines, working as intermediate storage devices for any resources.

You typically see CDNs storing assets that never change, like images, videos and hashed JavaScript files. This lets the user download these assets from a nearby server (typically within the same state or part of the country) without going to the origin server. This is very practical, since it both drastically reduces response time for the user, as well as the load on the origin servers.

Our CDN (AWS CloudFront, if you were wondering) receives every request to our domain, checks whether it has that particular resource (file path) stored, and returns it to the user, without ever bothering the origin server. However, we also use it to serve files that do change, like CMS-backed HTML pages and JSON responses, which I what I want to talk about in this article.

stale-while-revalidate

As I mentioned earlier, the Cache-Control header accepts a bunch of different directives. One of most powerful ones, is one called stale-while-revalidate. This strategy tells our cache (CDN) to do the following:

If the CDN contains the resource that’s still “fresh” (hasn’t expired), return that resource. Done.

If the CDN contains the resource, but it has expired, return the expired resource to the user. Then, ask the origin server for a fresh version of that resource in the background, so that the next request for the same resource will be “fresh”

The tradeoff is basically this: Some users might get stale content, but will still get it quickly.

It looks like this:

What happens here is the following:

If a request comes in, and there’s less than 1 hour since the data was placed in the cache, return the resource. Don’t ask the origin server, just return the data as is.

If a request comes in, and there’s less than 1 hour since the cache expired (for instance, 1 hour and 30 minutes since the data was placed in the cache), return the stale data to the user. Next, ask the origin server for a fresh response. Save that data in the cache, and mark it as fresh again.

If a new request comes in right after (for instance, 1 hour and 31 minutes after the original data was stored in the cache), the user will receive fresh data without waiting for the origin server at all. The origin server won’t be asked for a new copy for another hour.

This is a great strategy to use when you have a bunch of consistent traffic, or if any changes to the content you’re caching can wait for a few seconds. If you have consistent traffic, the cache will always be updated, since any stale request will trigger what’s called a revalidation (”go check the origin server in the background”). Cool stuff!

Our site was based on a CMS, and any changes in the returned HTML site (or JSON data) would typically be changes to a landing page, articles etc. Yes, it’s cool if the updated page was available right away, but it doesn’t really break anyone’s experience if they are a few seconds behind the breaking news. This seemed like a wonderful strategy for us.

The result is that, for resources that change, we have a > 95 % cache hit rate. The only pages that are retrieved from the server is the ones rarely updated. We’ve set the cache threshold to about 1 hour, which means you might get 1 hour old content if you’re the first visitor to visit that page a while – but if that’s the case, chances are the page might not be the most frequently updated one you have anyways.

The stale-while-revalidate header is such a useful technique in combination with CDNs, I don’t think I’ll ever serve a site without it again.

stale-if-error

We all make mistakes from time to time. In my experience, making sure the production environment we serve is resilient from errors is the best investment you’ll ever make.

Meet the stale-if-error directive. It works in a pretty similar fashion to the stale-while-revalidate directive, but with a very specific difference:

If the origin server returns a 5xx server error (500 Internal Server Error, 503 Service Unavailable etc) for a specific resource, or if there’s a network error (a timeout) return the last cached successful response.

In other words, we can specify that if our website goes down for some time (it happens), we can return the last working version of that website for a specified amount of time. Sure, the user won’t get any new data, but at least it’ll get some data back. Your server will hopefully trigger an alarm, and you can get your server back online without the user ever noticing.

In our case, we set this cache timeout to be 12 hours:

In other words – if our website goes down, we have 12 hours to get it fixed before we ever show a 500 error page to the user. Neat!

The previous version of our website had some intricate code that dealt with downtime in downstream systems, like our previous CMS or other internal APIs. Now, however, we could delete all that code and just use The Platform like it was supposed to work.

These two directives have made wonders for both our users, servers and pocket books. We spent a bunch of time optimizing the different parameters to make them just right, and we’ll probably keep on doing so for the foreseeable future, as we learn more about our users’ usage patterns.

outline

Summary

Caching stuff is fun. Sure, it might be one of two hard things in computer science, but it’s sure a great tool to keep in your toolbox.

I hope this article inspired you to start experimenting with these caching directives on your own site, saving your team and users a bunch of headaches.

💰 is 👑 - a guide to the new 🆒 cache headers

Not too long ago I came across this pattern that helped me with a longstanding problem I have had when writing tests: generating valid test data without spending half of your day instantiating domain models. Now, I’ve had this idea before, but I’ve never actually ended up testing it. That changes today, because from now this is easily my go to method of creating valid test data.

Just looking for the actual pattern? Click here to open an interactive Kotlin playground!

Now, there are of course many ways of generating test data, and I believe I’ve tried most of them in the search of the perfect one. They all have their problems though, and while I won’t deny that this method comes with some caveats as well, so far it seems this is easily outweighed by it’s advantages. The biggest advantages, as I see it, are:

My tests are as free from setup code as possible. The focus is on testing actual behavior.

I don’t have to spend time coming up with valid values in my tests. This is solved elsewhere, and my head can focus on validating behavior.

Before we start, we need a couple of classes to test. I know it’s a cliche, but let’s model a person with contact details:

Not too complex, but still: instantiating a new Person for each and every test get’s tiresome

The problem with this becomes apparent when you have to write ten tests in a row, and each of them requires one or more Person instances. How do you come up with plausible, but different, values each time? Or do you copy you setup code around, so you always test with the same values? Both are valid, but bothersome and/or problematic approaches.

Let’s instead go completely crazy and use randomness! With the help from two Kotlin features we can end up with a nice, readable API. Let’s write a random `Person` generator!

String.generate is left out for the sake of simplicity. There are many ways to implement it, and if you need some inspiration you can check here.

Now, there are some things going on here, so let’s go through and explain it. Both functions follows the same pattern where you extend the Companion object with a function called `generate`, which accepts and instance of a Kotlin Random. You could let the function itself create an instance of Random, but injecting it gives you the control: instead of injecting a Random.Default, you can seed it with Random(mySeed) and get predictable values.

A generator returns a Sequence. Sequences in Kotlin are lazy, so even though you can spot a while (true) in there this is not an endless loop: if you ask for one instance, you generate one, and if you ask for ten, you generate ten.

Now, generating the actual values is where things get really interesting. There is not really a right or wrong here, it all depends on your domain. Can an age be between 0 and 100? Does your system only handle ages from 18 and above? Is a negative age OK? Do you care about the content of a name at all, or is «@17'!-_y» perfectly fine? The values you generate should be valid in your domain, and what’s valid is different from system to system.

With these generators in place, you can generate test data this way:

OK cool, but I sometimes want specific values and you only give me random ones >:|

Fear not, I’ve got something for you here as well. There are multiple ways of achieving this. Let’s say we want a random person, but we need the age to be exactly 40. If you have a data class, you could of course use the copy function:

But you don’t always have data classes. Also, it’s not very elegant or cool. Instead, you could add an optional parameter to your generator:

Both of these works just fine in one-off situations and if you only need to override a small number of properties. But it quickly gets tiresome if you need specific values for multiple properties. To fix this, we can use another Kotlin concept: function literals with receiver. Let’s rewrite our generators based on this pattern!

Each generator gets its own "spec" (or specification, if you want. Abbreviations ftw.) helper class which holds the random generated values, and the generators are extended with a... weird (?) looking parameter. If you want to understand this weird syntax, the Kotlin documentation is an excellent guide. If not, just think of it as a lambda parameter that get’s called on an instance of a specific type. With this setup, we can generate test data with either completely random values, or the specific ones we need.

And there you have it!

A closing note about tests and randomness

Before I started writing this blog post, I had a talk with a colleague of mine about the concept. His immediate response was somewhere along the lines of «wait what, you use random data in your tests??».

And his reaction was probably justified. Unit tests and randomness aren’t exactly the two most likely friends. You want predicable tests, and randomness isn’t exactly known to be predicable.

There are, however, a few reasons why I believe this won’t be an actual problem.

First of all, it’s important that you take care when you implement your generators. The values should probably not be _completely_ random. If you have strict requirements and validation in place to enforce certain values at the boundaries of your application, you need to account for this when you generate data for the _insides_ of your boundaries. _You_ control the amount of randomness, and can even model dependencies between properties in your generators if you need to. If you take care and write good quality generators, the randomness should be an assurance, and not a source of problems.

Second, there are ways of making those rare build failures actually useful. In our current code base, we have created a JUnit extension which can be added to a test file. This extension generates a new Kotlin `Random`, and stores the seed in a file. This way, if a test fails, you have the actual seed that made it fail and can consistently replicate the error locally. BAM. Your random build errors just turned useful.


Test data generators in Kotlin

Ofte setter vi oss selv i situasjoner der vi tenker for komplekst på problemstillinger; vi gjør dem mer komplekse enn det trenger å være. Et enkelt prinsipp kan benyttes for å dele et problemer inn i mindre, mer håndterbare delproblem, som kan bidra til mer analytisk tenking og dermed forenkle problemet.

Hvorfor havner vi ofte ut på en kompleks reise når det kommer til problemløsning?

Barbara Minto skrev i sin bok "The Pyramid Principle: Logic in Writing and Thinking," om et viktig aspekt ved problemløsning og kommunikasjon:



“The problem was the thinking, not the language,” (Minto) says. “People were starting to write without working out their thinking in advance. But how does one go about figuring out one’s thinking in advance?”



Hun understreker at mange begynner på løsningen før de har klargjort tankene sine. Dette fører til unødvendig kompleksitet og forvirrende tanker. Mintos poeng er at utfordringen ligger i å organisere og klargjøre våres tanker før vi kommuniserer dem.

Dette belyser et sentralt punkt i hvorfor vi ofte havner ut på en kompleks reise når det kommer til problemløsning. Minto antyder at løsningen ligger i forhåndstenkning, altså det å strukturere og bearbeide våres tanker før vi prøver å uttrykke eller løse dem.



MECE prinsippet

MECE er et grunnleggende prinsipp når det kommer til problemløsning. MECE står for "gjensidig utelukkende (Mutually Exclusive)", "samlet uttømmende (Collectively Exhaustive)". Dette prinsippet bidrar til klarere tenkning og forenkler komplekse problem ved å dele dem opp i mer forståelige, distinkte deler.



MECE-prinsippet følger to spesifikke regler for å kunne forenkle komplekse problem. Disse to kategoriseringsreglene er:

Gjensidig utelukkende (ME): Et delproblem kan kun være i én kategori om gangen.

Samlet uttømmende (CE): Alle delproblemene må inkluderes i én kategori; kategoristrukturen kan ikke utelate et delproblem.



Dette sikrer et klart og fullstendig perspektiv på situasjonen, hvor alle alternativene blir synlige. "gjensidig utelukkende" garanterer at elementenes uavhengighet forhindrer overlapping, mens "samlet uttømmende" sørger for at ingen dimensjoner eller mulige løsninger blir oversett. Denne helhetlige tilnærmingen gjør at problemet blir delt opp i mindre delproblemer, og gir deg en tydelig oversikt over hele problemet.



Hvordan funker prinsippet i praksis?

For å forstå prinsippet er det nyttig å se på et enkelt eksempel, dette kan vi gjøre ved se på en problemstilling rundt inntekter og kostnader av en bedrift sine ulike avdelinger.



Problemstilling: En leder ønsker å analysere bedriftens inntekter og kostnader

Gjensidig utelukkende (ME): Bedriften deles inn i tydelige avdelinger som Regnskap, Markedsføring, Salg, HR og IT. Hver ansatt blir kun satt til en avdeling for å unngå overlapping.

Samlet Uttømmende (CE): Alle avdelinger i bedriften inkluderes i analysen for å sikre at ingen områder blir oversett, og at alle inntekter og kostnader blir vurdert.



Dette eksemplet viser hvordan MECE-prinsippet kan brukes for å strukturere en kompleks analyse på en klar og oversiktlig måte.



Områder der du kan bruke prinsippet

Bruksområdene for MECE-prinsippet er mangfoldige, spesielt innen organisering og strukturering av data. Her er noen områder der MECE prinsippet er nyttig:



Nedbrytning av komplekse problemstillinger: Del komplekse problemstillinger ned i distinkte hypoteser for enklere analyse og løsning.

Deling av prosjekter i arbeidsstrømmer: Bryt ned prosjekter i separate, klart definerte arbeidsstrømmer for effektiv håndtering og gjennomføring.

Valg av KPI-er: Velg KPI-er slik at hver av de måler et distinkt aspekt, uten at de overlapper hverandre.

Forenkling av data: Reduser lange lister med data til et oversiktlig sett med nøkkelideer, slik at hver idé representerer en unik del av dataen.

Gruppering av informasjon: Organiser informasjon i klart definerte, ikke-overlappende grupper for bedre oversikt og forståelse.

Strukturering av presentasjoner: Organiser omfattende presentasjoner i distinkte seksjoner der hver seksjon dekker et unikt tema eller poeng.

Sammenfatning av artikler:Forenkle artikler med mange seksjoner til noen få sentrale hovedtemaer for å fremheve de mest kritiske punktene.

Opprettelse av sammendrag: Lag korte, presise sammendrag av komplekse temaer, hvor hvert punkt dekker et unikt aspekt av temaet.

Utforming av spørreskjemaer for innsikt: Lag spørreskjemaer med eksklusive og uttømmende svaralternativer, slik som å liste alle avdelinger i et avdelingsvalgspørsmål.

Utforming av menyer: Skap menyer der hver del er klart definert og unik, for å gjøre det enkelt å forstå og velge det som er mest relevant.



Dette er et prinsipp som fungerer godt i de fleste tilfeller hvor du står overfor et problem som er komplekst, enten det er i en privat eller profesjonell sammenheng. Ved å bryte ned problemet i mindre deler som er "gjensidig utelukkende", altså at de ikke overlapper hverandre og "samlet uttømmende", der alle relevante aspekter blir inkludert, kan du unngå at viktige detaljer blir oversett. På denne måten sikrer du en grundig og strukturert tilnærming til problemløsning, noe som bidrar til en mer effektiv og oversiktlig håndtering av utfordringen.





Du kan benytte prinsippet overalt, du må bare finne ut når det er riktig for deg ❤️

MECE prinsippet for å forenkle ethvert problem

Designdrypp er en artikkelserie hvor forskjellige designere fra Bekk forteller om jobben de gjør, hvilke utfordringer de møter på, og kanskje kommer med et lite tips!





Hvem er jeg?

Navn: Anna Helgestad

Alder: 25

Fra: Brumunddal

Studerte: Master i Informatikk: design, bruk, interaksjon på Uio

Jobbet som designer i 2,5 år ish (startet fulltid nå i 2023)

Jeg elsker å være designer, men hva førte til at jeg endte opp med akkurat interaksjonsdesign? Jeg har også alltid vært veldig nysgjerrig på folk og syntes det har vært gøy å være kreativ. Takket være min mor fant jeg et yrke som kombinerte dette perfekt. Hun jobbet med innovasjon og kunne tipse meg om design av teknologi - og brått satt jeg på UiO for å studere nettopp det. "Design, bruk og interaksjon" trakk meg inn, mens "informatikk" var helt fremmed for meg. Heldigvis hjelper det å stille de dumme spørsmålene og sitte  rundt smarte folk med litt tålmodighet. 


Jeg synes at noe av det beste med å være designer er å kunne skape noe sammen - med utviklere, selskapet og brukerne. Jeg synes det er kult at vi lærer gjennom å prøve og samtidig ønsker å bli bedre. 


Hva gjør jeg?

Jeg sitter hos Skiforeningen som en del av et Spire-team. Vi jobber med "Barnas Marka" - en samleside for innhold rettet mot barnefamilier. De siste par ukene har vi jobbet med noe som har vært spesielt gøy, nemlig julekalender🎄✨ 

Her har jeg fått teste kreativiteten, tegne og kose meg, men det som har vært aller morsomst, er samarbeidet og engasjementet det har skapt i teamet. Ettersom vi er et Spire-team hos en litt travel kunde og satte i gang arbeidet med julekalenderen på eget initiativ, har vi også hatt rollene som markedsførere og medieansvarlige, samtidig som vi har utviklet og skrevet innhold til hver luke. 

Generelt sett er jobben som designer i Barnas Marka veldig variert. Som mange andre designere sikkert kjenner seg igjen i, er det alltid mye å sette fingrene i. Vi har bygget Barnas Marka opp fra bunnen og det har virkelig vært en gavepakke å få muligheten til det! 

Prosjektet har gitt meg muligheten til å jobbe med alt fra designsystem, relasjonsbygging med kunde til konseptutvikling. 


Hvilket tips ville jeg gitt meg selv i starten av oppdraget/karrieren?

Det er helt greit å ikke kunne alt, og det er lov å ikke være interessert i alle sider av faget ditt eller å være ekspert på alt.

Med en master fra UiO har jeg relativt begrenset erfaring innen det grafiske. Da jeg startet, følte jeg at det var et helt fagfelt jeg måtte lære fra bunnen av. Heldigvis har jeg en flott personalleder som spurte: "Er dette noe du synes det hadde vært spennende å jobbe mer med, eller er det noe du sier fordi du tror det forventes?" Når jeg var ærlig med meg selv, innså jeg at jeg hadde mye mer lyst til å utvikle meg innen andre aspekter av faget.

Samtalen fikk meg til å forstå at målet nødvendigvis ikke er at man skal være en "femer" på alle områder. Man er ikke først en god designer når man kan sjekke av at man "kan" alle disse områdene.  


Hva eller hvem inspirerer deg?

Møter og samtaler med brukere inspirerer meg mest. Det er utrolig spennende å forsøke å forstå hvordan verden oppleves fra deres perspektiv og oversette det til design.

Jeg blir også veldig gira når utviklere ser muligheter og liker å samarbeide 🫶🏻

8-Dec

Today's posts

💰 is 👑 - a guide to the new 🆒 cache headers

Test data generators in Kotlin

MECE prinsippet for å forenkle ethvert problem

Designdrypp med: Anna Helgestad