Welcome to this edition of CG Net Nonprofit Tech Talk.  Today, our guest is Ken Novak, CG NET’s Senior Technical Advisor.  We recently interviewed Ken about how to estimate your Amazon web services hosting charges, and we’d like to continue that discussion today, covering how you can minimize Amazon Web Services pricing.  Welcome Ken.

Ken Novak:     Hi Tim.

Tim:     In our previous interview, you said that most of the hosting charges are for the virtual machine itself, so is that where you look to save money?

Ken Novak:     Yep, that’s basically it.  Sometimes, on some applications, storage is important.  Let’s say if you’ve got more than a few hundred gigabytes of storage, or especially terabytes of storage, but that’s not on most applications that we see.  Most of the time we see things that are in those low hundreds of gigabytes where the storage doesn’t amount to that much from one month to the next. So it’s really the virtual machines that cost the most.

Tim:     Do you have any idea sort of what the rate for storage is?  I mean an order of magnitude?

Ken Novak:     Sure, the most common storage charges on Amazon are from about 7 cents to 10 cents per gigabyte per month, which means that for a hundred gigs, it’s $10.00 a month.

Tim:     Not huge.

Ken Novak:     Doing a lot of work to try to economize on that is not so great.  If you’ve got 10 terabytes and you’re now up to $1,000.00 a month, well, that’s a different ballgame, and there are lots of things you can do to try to cut those charges.  That might be worth a separate discussion sometime, but I think for most people, the storage is manageable.  It’s the virtual machines that cost the money.

Tim:     How do you save money on your virtual machine charges?

Ken Novak:     Well, it’s a funny thing.  They’re not all one price.  In some ways, occupying a chunk of RAM and a part of a machine to be a virtual machine on in Amazon is kind of like occupying a seat on an airplane.  The person next to you may be paying three times as much as you are. There are actually several different pricelists, and it depends a lot on the conditions that you buy it under.  The standard price that you see all the time is what Amazon calls on‑demand pricing, when you log onto their site and say I want a machine. And as fast it can, typically in 20 or 30 seconds, you’ve got that machine running. From then on you pay by the hour, and whenever you shut it off, you’re paying nothing. That’s a typical way it’s done.

That’s kind of like walking up to the airline counter and saying, “Book me on the next flight,” but for other folks who might be willing to say “Put me on a flight when it’s convenient for you, Amazon,” you might get a cheaper price. Or you could say, “Put me on, and guarantee me a seat on every flight for the next year. I’ll pay in advance.”  Well, you might get a discount then, too. That’s basically the way it works at Amazon.  There are these different flavors of pricing, if you’re willing to accept something less than just walk up on‑demand pricing.

Spot Pricing

Tim:     Can you give me example of how that might work? How you could get to the cheapest one?

Ken Novak: I guess we could start with the one that’s the absolute cheapest at Amazon.  It’s called spot pricing, and they name it that because it’s kind of by analogy with the spot marketplace.  They look at all their spare capacity, and they invite people to say (it’s almost like Priceline) I’m willing to go, but only if it costs me a certain amount or less.

They look at their spare capacity. They look at all the offers people have wanting to buy at different prices, and they pick the price at which the market will clear. They’re doing this calculation every 15 minutes or so.

IT depends on what’s going on. Let’s say it’s a busy time of year or some customer has come as an on‑demand customer and required a lot of capacity. In that case, there might not be much to sell, and as a result it’ll be a relatively high price at that time, whereas if it’s the middle of the night or on the weekend or something, the price can get very low indeed.

Tim:     Okay, so let’s see if I’ve got this clear.  They’ve got on‑demand pricing, which is a set fee that you can predict what you’re going to pay.  That is to say, you’ll use it for different amounts of time, but there’s a given rate.

Ken Novak:     Yeah.

Tim:     And then there’s spot pricing that depends on when you come in and when you place your order.

Ken Novak:     Right.

Tim:     Now, when you place your order, does the spot price stay the same throughout your usage period?

Ken Novak:     No.  It’s changing minute by minute, and if the spot price ever goes above the rate that you said you were willing to pay, they’ll shut your machine off.

This was really invented for the classic use-case. Let’s say that someone was making a movie and had a lot of video to render, so they would set the computer up to generate all these video images. They’d set it up so they could use maybe 200 machines, and within an hour the whole, video clip would be done. They would need a lot of capacity, 200 machine hours, to get this thing done, so they would go to the spot market and say “Fine, give me up to 200 machines, but only at a fifth of your usual price. We don’t care when it gets done. It can get done overnight; it can get done tomorrow.  It’s still better than us setting up our machines and letting them run for 200 hours.”

It’s gonna get done a lot faster as soon as there’s capacity on Amazon, and it’ll be really cheap, because we’ll just wait until it’s convenient for Amazon.  You set up a machine on an on‑demand basis and then stop it. And then it’s ready to run whenever, as long as it finishes some unit of work.  As long as it stores it in a storage system on Amazon that’s outside of your machine, then it doesn’t matter when it gets turned off.  It’ll always store as much as it got done.

Tim:     So you could say okay, I’m gonna pay this price, and so when it hits that price, it would start up your machine and if the spot market exceeded that price, it would shut off your machine and then when the next time the spot market got down to that right place, it would automatically turn it on again?

Ken Novak:     Yep.  That’s the idea.

Tim:     That’s cool.

Ken Novak:     Yeah. That was the classic use for spot, and it was a way for them to get some money out of any unused capacity they had.

I was talking to some friends of mine a few years ago after it first came out, and they turned me on to a whole different way to use spot pricing.  They’re a conventional company that provides an Internet service, and they are developers and testing people. They always need some capacity on Amazon to work with. They got the clever idea of asking for spot instances for their developers, but setting the price to be essentially the same as the on‑demand price, meaning, in other words, they were willing to pay a relatively high price so they wouldn’t get turned off, but whenever the spot price dropped, they would pay the lower price. As a result, they would typically go months without getting turned off. They reported something on the order of once or twice a year that the market would get so hot on Amazon that the spot price would actually be higher than the on‑demand price. That almost never happens.

Tim:     So when you order the spot price. You get the benefit if it goes down from what your price was.

Ken Novak:     Exactly.

Tim:     That’s very clever.

Ken Novak:     Yeah. The thing is, if you do it that way, and you’re willing to tolerate that once every few months your machine suddenly gets turned off and you have to go manually turn it back on again, then you can save literally 70 percent off the on‑demand price.

Tthe only other downside to spot pricing is that when you have to request the capacity at the spot price, it’ll be a few minutes before it’s made available to you, never more than 10 or 15 minutes. But as a result, if you’re really in a high-intensity work mode and you just want to get the thing on as fast as possible, well, you really want on‑demand.  If you’re willing to wait five or ten minutes for the spot processor to come along and start you up, then you can benefit from the lower prices.

Tim:     That sounds very cool, and you say the savings can be…

Ken Novak:     Typically the average spot price is less than a third of the on‑demand price on a 24/7 basis. I guess it’s just that out of 168 hours a week, between weekends and nighttimes, if you’re in a U.S. Data Center, there’s an awful lot of spare capacity available, a lot of hours that there are machines that aren’t being used very much. So most of your hours are at very, very low cost.

Reserved Pricing

Tim:     I think I understand spot pricing now, but earlier on you mentioned reserved pricing.  How do you save money with that?

Ken Novak:     Well, the idea here is that you pay some amount of money in advance, and for some length of time you get a cheaper price.  It kind of reminds me of the, of an amusement park I went to once where you could get in for nothing, but then you’d pay $3.00 a ride. But if you bought a VIP pass, you could pay $1.00 per ride.  That pass cost you about 20 bucks, so they expected you to spend a lot of time on the rides to earn back the pass.

That’s kind of what Amazon does. You aren’t quite paying the full price for a virtual machine all year as your upfront price.  You’re paying something less than that, but then you pay a very low price every hour you use it. So it’s a way of paying in advance to reduce the cost.

The most common pattern is that someone will pay what’s called “heavy utilization pricing,” where you pay a significant amount of money, equivalent to running the machine fulltime for a few months, and then you pay a low hourly rate. By the end of the year, you will have paid 40 percent less than if you had just bought an on‑demand virtual machine of the identical size and the identical type. So it’s a pretty significant discount.

If you know you’re gonna run this machine around the clock all year, then it’s for sure a good approach to take. You’ll break even at about the eighth month, and you’ll get essentially 40 percent off.

They have a few variations on that.  They allow you to pay a little bit less upfront, then a little more by the hour, and you can kind of work it out if you think you’re not going to run it 24/7 all year. They have a one‑year pricing; they even have a three-year pricing. If you happen to be in a project that’s got a fixed budget over a long period, you can pay a bit more upfront, and I think then you get as much as 60 percent off the on‑demand, but of course, you’re committing to three years. That’s a pretty long time.

Tim:     That’s pretty exciting.  Are these discounts well known?  It seems to me that they’re not all that well publicized.

Ken Novak:     They’re pretty much on the website, but it does take wrapping your head around and making a little model and all that to figure out what the real rate would be. I think that once people have committed to Amazon, they go shopping for how can they reduce their rates, but if you’re trying to plan it and budget for it in the first place, it’s worth it to spend some time with their calculators or with your own planning.

One issue that you have to watch out for with the reserve pricing, on Amazon anyway, is that it’s done by type of machine. You really are just buying a ticket for a certain kind of machine to get it at a lower price.  It’s not attached to a particular machine.  Let’s say I have five Windows machines and they’re up. They’ve all got 7 gigs of RAM.  They’re the same kind of machine.  Maybe they’re in a cluster group for Share Point, or something like that. If I buy three reserved instances, what it really means is that I will get the cheaper hourly rate on the first three machines of that type that are running, and if I later go from having five machines running to having three machines running, they’re the ones that will continue to get the special rate.

It’s an interesting system, and Amazon’s made it so that you’ve got quite a bit of flexibility.  They don’t have to be concerned about which machine is the reserved one, or anything like that. The only thing is that it is specific to the type of machine, so if you go and change one of those 7 gig machines to being a 30 gig machine, then that reservation no longer applies.

That’s why they even went an extra distance. They’ve created this thing called a secondary market for reservations, if you can believe this. Let’s say that I’m in that situation where I’ve got another six months to go on this lower priced ticket, but I don’t have any machines of that type for it to apply to. I can go offer that to other Amazon customers.

Tim:     So there’s some kind of list you can put yourself on and say, “This is available,” sort of like Craig’s List, or something?

Ken Novak:     Absolutely, and Amazon’s built it right into their interface, so when you go saying I want to buy a reservation, right in that screen there’s an option to ask whether there are there any reservations you can buy from other customers.

Tim:     That’s fascinating.

Ken Novak:     They list it right with their own reservations that they sell, so I guess they’re basically trying to make it sort of as transparent as possible. Making these things transferable is kind of an odd thing.

Anyway, we start sort of running down the rabbit hole of all these considerations, right?  How much will you really run it? What size a machine do you really need?  What’s your break-even point?  Could you even sell the reservation?  So, in a way it’s kind of more complicated than it needs to be, but the price reduction is very significant.  You know, when you start looking at 40 percent off, that’s worth trying to figure out.