That is literally what we do at my job.
Three copies: One for the client who paid for it, one for us (internal processing and testing only), and one as a backup goes to a storage location that is a converted cold war era bomb shelter.
Oh god my story. Okay so I was building out a video transcoding service for a company. We all know video transcoding is hella expensive. So I’m using kubernetes to help manage scale, and we’re on the cloud. I warn them hey, cloud is hella expensive, this is going to be… a lot. Well what do you recommend? Glad you asked, and I pitched that we have 3 heavy server nodes sitting either in a rack if we want it official, or even we were small enough we could just have them in the office. They would be VPN’d into the cluster, members of the cluster, and those get the priority. If a transcode job comes in use those nodes, only spin up cloud nodes if the scale is too high. I quoted about 20k for 3 beefy performant machines for the node.
Executives balked at the price. Way too much money, what a ridiculous idea anyway, we’re a cloud company.
Two months into the cloud only solution they were averaging 12 grand just on CPU compute! Why is it so high?! That’s ridiculous!
Absolute fuckers, the morons. I swear I’ve seen so many companies hemorrhage money because they refuse to listen to legit experts in the field. You fuckers, I was trying to save you money, but no your MBA and accounting degrees taught you how to run fucking cloud operations.
We spent several hundreds of thousands of dollars last year doing geophysical processing in azure. But it was an emergency: It was a hot fix to avoid losing out on hundred times that amount. Turned out the contract negotiator never bothered telling operations that they agreed to deliver the data with some processing already applied.
We considered building a processing cluster on site, but buying the necessary hardware
and shipping it halfway around the world in a timely manner would’ve been even more costly. Plus I would be the one who had to build the rig, and I was all tied up on a different project a few countries over at that time.
I hate that it’s so hard to get these people to agree to capex. My current company runs a few datacenters, and we have some teams that use them for their base load. It saves a shitload of money! Like, I don’t get why this is a concept that MBAs reject. You don’t have to go all in on capex for your infrastructure, just find a nice mix of capex/opex. If you’re afraid that you won’t use the shit you bought later on, then you should probably make sure that the market is there for whatever you’re selling before you dive in headfirst.
Because data center costs are OPex and on prem server costs are CAPex, and companies very much prefer things to be in the OPex (operating expense) column.
On-prem still has its uses
Platter harddrives are still useful
Tapes and tapedrives aren’t obsolete
On prem is, in almost all cases, cheaper than cloud. Even when you include the salaries of the folks managing it.
But MBAs will pay a LOT for outages to be someone else’s problem.
Also, do you really need high performance SSDs? Are you actually writing the drive volume a day?
If you’re not archiving old data on tapes and shipping them off to a converted bomb shelter, you’re not doing it right.
That is literally what we do at my job.
Three copies: One for the client who paid for it, one for us (internal processing and testing only), and one as a backup goes to a storage location that is a converted cold war era bomb shelter.
Oh god my story. Okay so I was building out a video transcoding service for a company. We all know video transcoding is hella expensive. So I’m using kubernetes to help manage scale, and we’re on the cloud. I warn them hey, cloud is hella expensive, this is going to be… a lot. Well what do you recommend? Glad you asked, and I pitched that we have 3 heavy server nodes sitting either in a rack if we want it official, or even we were small enough we could just have them in the office. They would be VPN’d into the cluster, members of the cluster, and those get the priority. If a transcode job comes in use those nodes, only spin up cloud nodes if the scale is too high. I quoted about 20k for 3 beefy performant machines for the node.
Executives balked at the price. Way too much money, what a ridiculous idea anyway, we’re a cloud company.
Two months into the cloud only solution they were averaging 12 grand just on CPU compute! Why is it so high?! That’s ridiculous!
Absolute fuckers, the morons. I swear I’ve seen so many companies hemorrhage money because they refuse to listen to legit experts in the field. You fuckers, I was trying to save you money, but no your MBA and accounting degrees taught you how to run fucking cloud operations.
We spent several hundreds of thousands of dollars last year doing geophysical processing in azure. But it was an emergency: It was a hot fix to avoid losing out on hundred times that amount. Turned out the contract negotiator never bothered telling operations that they agreed to deliver the data with some processing already applied.
We considered building a processing cluster on site, but buying the necessary hardware and shipping it halfway around the world in a timely manner would’ve been even more costly. Plus I would be the one who had to build the rig, and I was all tied up on a different project a few countries over at that time.
Should have told them it was an on prem cloud lol.
I hate that it’s so hard to get these people to agree to capex. My current company runs a few datacenters, and we have some teams that use them for their base load. It saves a shitload of money! Like, I don’t get why this is a concept that MBAs reject. You don’t have to go all in on capex for your infrastructure, just find a nice mix of capex/opex. If you’re afraid that you won’t use the shit you bought later on, then you should probably make sure that the market is there for whatever you’re selling before you dive in headfirst.
Because data center costs are OPex and on prem server costs are CAPex, and companies very much prefer things to be in the OPex (operating expense) column.
But the drive is pricey :)