For my org. I don’t have budget for a dedicated in-house opsec team, so if I on-prem it triggers additional salary burden for security . How would I overcome this?
You can't. That's the use case FOR AWS/GCP. Once the differential between having a in-house team and the AWS premium becomes positive is when you make the switch.
A lot of the discussion here is that the cost of the in-house team is less than people think.
For instance: at a former gig, we used a service in the EU that handled weekends, holidays and night time issues and escalated to our team as needed. It was pretty cheap, approximately $10K monthly fee for availability and hourly rate when there were any issues to be resolved. There were a few mornings I had an email with a post-mortem report and an invoice for a hundred euros or so. We came pretty close to 5 9's uptime but we didn't have to worry about SLA's or anything.
There is also the factor that the idea that you don't need administrators for AWS is bullshit. Cool idea, bro. Go to your favorite jobs portal. Search for "devops" ... 1000s of jobs. I click on the first link.
Well, well, they have a whole team doing "devops administration" on AWS and require extra people. So not having the money for an in-house team ... no AWS for you.
I've worked for 2 large-ish firms in the past 3 years. One huge telco, one "medium" telco (still 100s of people). BOTH had a team just for AWS IAM administration. Only for that one thing, because that was company-wide (and was regularly demonstrated to be a single point of failure). And they had AWS administrator teams, yes teams, for every department (even HR had one, though in the medium telco all management had a shared team, but the networking and development departments still had their own AWS teams, who, btw, also did IAM. The company-wide IAM team maintained an AWS IAM and some solution they'd bought that also worked for their windows domain and ticketing system (I hate you IBM remedy), and eqiupment ordering portal and ...)
AND there were "devops" positions on every development team, and on the network engineering team, and even a small one for the building "technics" team.
Oh and they both had an internal cluster on top of AWS, part on-premise, part rented DC space, which did at least half the compute work (but presumably a lot less of the weird edge-cases), that one ran the company services that are just insane on AWS like any kind of video.
they sell "you don't need a team"... which is true om your prototype and mvp phase. and you know when you grow you will have an ops team and maybe move out.
but in the very long middle time... you will be supporting clients and sla etc, and will end up paying both aws AND an ops team without even realizing.
Familiarize yourself with your company’s decision process on strategic decisions like this. Ensure you have a way to submit a proposal for a decision on making the change (or find someone who has that access to sponsor your proposal), build a business case that shows cost of opsec team, hardware and everything else is lower than AWS (or if cost is higher then some other business value is gained from making the change — currently digital sovereignty could be a strong argument if you are EU based).
If you cant build a positive business case then its not the correct move. Cash is king. Sadly.
If you don't have budget for someone to handle this for you, you can't afford AWS either, as you still need to handle the same things and they're generally more complex when you use AWS.
Funny, we are working to implement this same logic in our in-house financial categorization agent. When we have a repeat prompt it goes to a json that stores answers and only goes to AI for edge cases.
Awesome to hear you’ve done similar. JSON artifacts from runs seem to be a common approach for building this in house, similar to what we did with the muscle mem. Detecting cache misses is a bit hard without seeing what the model sees, part of what inspired this proxy direction.
The concern is not the course, but the ability to adjust a layout due to deviations from the plan due to normal construction errors.
For example a pipe might not be in the location shown on plan for many reasons ranging from simple human error to a delta between the plan location when the pipe was layed and the time the robot got its data…keep in mind that when the pipe went in there was only dirt, not anything to accept ink.
But at that point it's back to engineering to figure out what to do (leave the pipe where it is and adjust around it _or_ move the pipe - possibly cutting concrete and perhaps untensioning/retensioning post-tensioned cables at substantial delay/cost) or move the piece of equipment that the penetration is serving.
One nice thing about automation like this is that the "as built" plans are more likely to be accurate because the only way to get "the computer" and "the robot" to stop squawking is to change the plans they are operating off of.
If this can't handle dirt surfaces, future generations/models probably will if there's demand. Perhaps such models would use spray paint/stencils or driving pins into the ground for marking purposes (or something more practical - I'm a software guy and this sounds like a hardware problem!).
My experience is with small residential builds but I would hope on large projects the location of each "unmovable" pipe/conduit etc that will end up penetrating a slab is already carefully verified before the next step is taken (such as placing concrete). Hopefully this is done with a total station rather than guys with chalk lines and tape measures. But a solution like this could reduce manual checking mistakes (of course, it's less likely to result in an experienced subcontractor noticing that the plan must be wrong because there's no reason for a conduit for 1KV electrical cables to come up 2cm away from a toilet trap in a multi-stall public bathroom - GIGO).
each "unmovable" pipe/conduit etc that will end up penetrating a slab is already carefully verified before the next step is taken (such as placing concrete)
My experience is as an architect.
The CAD file isn’t the building no matter how much everyone might wish it were.
Even if the plans were the building odds of everybody using the same plan revision all the time is just about zero.
And most of the time, nobody is gonna pay for a super accurate as-built BIM. Because the point of the exercise is a certificate of occupancy.
Weirdly it turns out to be cheaper/faster than paying a human being to do the same thing in use cases where you have large concrete slabs with complex walls/casework layout
Ya but then you need to pay for a team to maintain network and continually secure and monitor the server and update/patch. The salaries of those professionals , really only make sense for a certain sized organization.
I still think small-midsized orgs may be better off in cloud for security / operations cost optimization.
I mean, by definition yes they are. RDS is locked down by default. Also if you're using ECS/Fargate (so not EC2) as the person writing the article does, it's also pretty much locked down outside of your app manifest definitions. Also your infra management/cost is minimal compared to running k8s and bare metal.
This implies cloud infrastructure experts are cheaper than bare metal Linux/networking/etc experts. Probably in most smaller organizations, you have the people writing the code manage the infra, so it's an "invisible cost", but ime, it's easy to outgrow this and need someone to keep cloud costs in check within a couple of years, assuming you are growing as fast as an average start-up.
I think it's completely different ballparks to compare the skill sets...
It is cheaper/easier for me to hire cloud infrastructure _capable_ people easier and cheaper than a server _expert_. And a capable serverless cloud person is MUCH cheaper and easier to find.
You don't need to have 15 years of a Linux experience to read a JSON/YAML blob about setting up a secure static website.. of you need to figure out how to set up an S3 bucket and upload files... And another bucket for logging... And you have to go out of your way now to not be multi-az and to expose it to public read... I find most people can do this with minimal supervision and experience as long as they understand the syntax and can read the docs.
The equivalent to set up a safe and secure server is a MUCH higher bar. What operating system will they pick? Will it be sized correctly? How are application logs offloaded? What are the firewall rules? What is the authentication / ssh setup? Why did we not do LDAP integration? What malware defense was installed? In the event of compromise, do we have backups? Did you setup an instance to gather offloaded system logs? What is the company policy going to be if this machine goes down at 3am? Do we have a backup? Did we configure fail over?
I'm not trying to bash bare metal. I came from that space. I lead a team in the middle of nowhere (by comparison to most folks here) that doesn't have a huge pool of people with the skills for bare metal.. but LOTS of people that can do competent severless with just one highly technical supervisor.
This lets us higher competent coders which are easier to find, and they can be reasonably expected to have or learn secure coding practices... When they need to interact with new serverless stuff, our technical person gets involved to do the templating necessary, and most minor changes are easy for coders to do (e.g. a line of JSON/YAML to toggle a feature)
This comment pretty much sums up this argument. Well said.
As with everything, choose the right tool for the job.
If it feels expensive or risky, make a u-turn, you probably went off the rails somewhere unless you’re working on bleeding edge stuff, and lbh most of us are not.
I very much understand this, and that is why we do what we do. Lots of companies feel exactly as you say. I.e. Sure it is cheaper and 'better', but we'll pay for it in salaries and additional incurred risk (what happens if we invest all this time and fail to successfully migrate?)
This is why we decided to bundle engineering time with the infrastructure. We'll maintain the cluster as you say, and with the time left over (the majority) we'll help you with all your other DevOps needs too (CI/CD pipelines, containerising software, deploying HA Valkey, etc). And even after all that, it still costs less than AWS.
Edit: We also take on risk with the migration – our billing cycle doesn't start until we complete the migration. This keeps our incentives aligned.
That used to be the case until recently. As much as neither I nor you want to admit it -- the truth is ChatGPT can handle 99% of what you would pay for "a team to maintain network and continually secure and monitor the server and update/patch." Infact, ChatGPT surpasses them as it is all encompassing. Any company now can simply pay for OpenAI's services and save the majority of the money they would have spent on the, "salaries of those professionals." BTW, ChatGPT Pro is only $200 a month ... who do you think they would rather pay?
I think the argument is that dev with some vibe coding can successfully setup servers that are good enough already for 10x less cost and 95% reliability
Plus that 5% left out is a one in twenty chance that some business critical service may fail when least convenient.
And when it does, the person that vibed it into existence will only have ChatGPT to fall back to, having no personal or organizational experience to rely on.
But they have a 95% chance of getting it right, if they don't panic too much.
I would pay you 100x that amount monthly to perform those services, as long as you assume the risk. If you're convinced this is viable, you should start a business :)
If you haven't had to fight network configuration, monitoring, and security in a cloud provider you must have a very simple product. We deploy our product both in colos and on a cloud provider, and in our experience, bare-metal network maintenance and network maintenance in a PaaS consumes about the same number of hours.
I'm not sure I like the incentives that creates, though. If this is the precedent, then if you're committing financial fraud already you might as well go as big as possible, since there appears to be a cap on punishment.
I mean from mine, and im sure many other people's view, that is already mostly true. Her biggest mistake to me seems like messing with a big dog, rather than screwing over little guys that don't have millions to throw at the courts or teams of full time lawyers. If JPMorgan defrauded the same amount of money from a bunch of regular joes I doubt anybody would be in jail or even that any potential fees would come anywhere to matching the amount of money defrauded.
ah yes, so clearly we must make sure that they are even less enforced. what could go wrong? hell, Enron lost investors-- bunch of billionaire fat cats, the lot of 'em-- $75 billion... they shoulda made Kenneth Lay Chairman of the Fed! or i guess maybe he his net worth had one too many zeroes to be sympathetic... hell, Kawhi Leonhard and his uncle blatantly and deliberately broke all of the NBA salary cap rules in the way that an undercover cop asks everyone they come across if they can buy weed from them, but the Clippers are owned by Steve Ballmer, so it's apparently okay. we hate the player because of the game in everything except for sports lmao.
I'm unconvinced. Banks wield massive political power through their capital, and so get away with highway robbery all the time. 2008 financial crisis comes to mind. It seems to me that the rules are one sided and so it seems this "foundational part" of high functioning society is just guarding the highwaymen with the cityguard, when the swords should be facing the other direction.
The banks at the center of the 2008 crisis are largely international. Is your question about the entire western international finance system and all societies comprising it?
Anti fraud rules sending people to jail are what we're discussing, and the proposition that they're necessary for a high functioning society. If we have these rules, and society is not high functioning, then, it seems you and I agree, that they're not necessary for a high functioning society.
> Strong anti fraud rules (even for massive banks) are a foundational part of a high functioning society.
Sometimes it seems we're missing most of the "foundational part[s] of a high functioning society" except the ones that serve elites (which then are so sacred *and must never be questioned).
Eventually people stop caring about the elite's protections, even if that breakdown is ultimately harmful. It's like the murder of that United Healthcare CEO. His company ground down the common man's benefits in a way that probably killed thousands (at least), but we're supposed to cry for him. Betray people long enough, and they're no longer interested in holding up their side of the bargain.
If they want a high functioning society, the elites need to work harder at holding up their end of the bargain.
