Prepare to die - An introduction to Spot Market

What if I told you that renting these servers could give you a discount of up to 90% compared to a regular on-demand server. Still not convinced? Have a look at the graph below.

The graph shows the pricing around Black Friday 2019. As you can see, if you had hosted a server at Amazon eu-west-1c, you would have paid only 10% of the on-demand price! Sounds nice right!? But, there is a catch! Depending on the cloud provider, your provider reserves the right to at any time shut down and remove your server at almost no notice! Let’s dive deeper into the world of Spot Market, and why it's good to prepare to kill your application.

The Spot Market is in short, the unused/spare resources available at a datacenter/zone. Instead of these resources just being left in an idle state, these resources are auctioned to the highest bidder. So, how do we bid, and how does that process work? Well, TLDR; in order to get an instance, you specify the maximum amount you want to buy your resources for. If your max value is lower than the market price, you will have your server. However, should the market price surpass your bid, then that server is given to someone else. Pretty simple actually. Its kinda like buying and selling stocks.

All the four major cloud providers have some sort of spot market and they are called:

• Amazon - EC2 spot instance
• Google - Preemtible VMs
• Microsoft - Low priority VMs
• IBM - Transient virtual servers

Some of the features varies across all the providers, nevertheless, the concept remains mostly the same. For a more detailed comparison, have a look at this article. In this article, we’ll focus on Amazon EC2 spot instances.

To utilize the Spot Market (safely), there are some steps and takes I want to share. There are more to the Spot Market than I can write about in this article, but here are some key takeaways.

The first step into getting ready for the Spot market is to identify what parts of your app that holds states. Our goal is to make every interaction with our app as atomic as possible, meaning the app should not have processes that depend upon a previous task. This could be as simple as a write-cache, a background task or a file processing that is handled after a file upload is completed, you name it. I recommend having a look at the 12factorapp for some best practices and suggestions that may help along the way.

Managed services (often) have the advantage of scaling to your demand. Therefore, leaving the responsibility of hosting and scaling the service to the cloud provider could be a good idea. Some of the services to consider having as a managed service are typically:

• Raw storage (Block storage, object storage).
• Database services (SQL, NoSQL, Document Storage, K/V store, Column stores).
• Messaging and related services (Event streams, Message brokers).
• CDN Services.
• Notification services (Email, SMS, Websockets, chatbots(?)).

The list goes on and on… Some may even use Blockchain as a service. Nonetheless, the main point is to utilize these services and let somebody else manage the scaling where that makes sense for your application. Just keep in mind that in some cases it might not make sense to use a managed service if it does not fit the needs of your application.

Finding out how your application should scale is an important step in your journey to the cloud. In most cases, applications should be written in a way that makes them able to scale horizontally (adding more servers). However, if your application or logic makes that impossible, vertical scaling could also be an option. If your application receives little to no traffic during specific intervals (i.e nights), then running many instances might not be a requirement.

With this knowledge, you are on your way towards the spot markets. I would recommend investigating the various features that your cloud provider has. Remember, the Spot Market is simply a tool that can be utilized. In many cases, this tool reduces the costs of running your application. In my opinion, if you have an application that works for the Spot Market, then you have a very resilient application that actually will handle an enormous amount of workload. If you found this interesting, I recommend digging into some of the links below. Good luck with your next cloud project.