Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Sudden disturbing moves for IT in large companies, mandated by CEOs (reddit.com)
283 points by lnguyen on Sept 14, 2022 | hide | past | favorite | 81 comments


This is Reddit, so taking any claims with a severely hefty earth sized grain of salt..

> Odd part 1: ... moving large numbers of VMs (100,000-500,000) over to Linux based virtualization in very short time frames.

> Odd part 4: Every one of these requests involves moving the VMs off VMWare or Hyper-V onto OpenShift, specifically.

As a Solution Architect at Red Hat, no sane sales rep would ever recommend or propose moving VMware footprints of that size onto OpenShift via OpenShift Virtualization[0]. As amazing as that payout would be, that would literally be account suicide if it ever got signed off on. The whole purpose behind OpenShift Virtualization is to aid in organization modernization as a way to consolidate workloads onto a single platform while giving app dev time to migrate their work to containers and microservice based deployments.

We are working on making OpenShift Virtualization as capable as we can (considering we're killing the Red Hat Virtualization product [upstream project: oVirt]) but it's not really meant, especially right now, to be a VMware replacement. That's what solutions like Nutanix are for.

This entire thing, if at all true which is unlikely, would smell of typical negotiation games to attempt to gain better pricing/discounts when it comes time for their VMware renewals. We see this a lot in attempts for customers to try and get better pricing when it comes Red Hat products, and potential customers the other way around with their existing vendors. Business is business and everyone will try to get the best deal they can, but the games do get annoying after a while.

[0] https://www.redhat.com/en/technologies/cloud-computing/opens...


> We are working on making OpenShift Virtualization as capable as we can (considering we're killing the Red Hat Virtualization product [upstream project: oVirt]) but it's not really meant, especially right now, to be a VMware replacemen

Didn't know RHEV was being killed. It makes sense, the enterprise virtualisation market is shrinking by the day, but it's still a bummer there's less competition. Is oVirt being maintained?

> That's what solutions like Nutanix are for.

Nope. Nutanix is for when you want to replace your hardware and software, have specific workloads that fit a hyperconverged hardware deployment, want a good piece of software with a ton of extras that don't really work all that well, being bash and random FOSS smushed together with duct tape. Oh, and you have money to burn.


For LXC and VM's one might consider Proxmox as counterbalance to the VMware walled garden.


Second Proxmox. While our footprint is likely much smaller than many of the other folks on here, it's been a life saver for managing LXCs.


Another Proxmox vote here, even if I'm also small scale.

Proxmox has been solid and near problem free for our simple deployment

As far as Linux system containers go, is the landscape still raw LXC, LXD, and Proxmox?

LXD snap had caused some issues for me. I didn't have any issues with it when it was packaged as a distribution package. Plus I briefly considered it again when VM management came out, but Proxmox still won over it.


Disclaimer: I work at Hashicorp.

There's also Nomad which has an LXC plugin alongside all the others.


>Didn't know RHEV was being killed. It makes sense, the enterprise virtualisation market is shrinking by the day, but it's still a bummer there's less competition. Is oVirt being maintained?

With the release of RHV 4.4 a few years back, it's become the last RHV release and will be maintained until its EOL in 2024 (ELS 2026). Like any open-source project, oVirt will be maintained by whoever wants to work on it, be it Red Hat engineers, other virtualization organizations, or just random contributors. But RH engineers won't be payrolled to work on oVirt on a daily basis.

> Nope. Nutanix is for when you want to replace your hardware and software, have specific workloads that fit a hyperconverged hardware deployment, want a good piece of software with a ton of extras that don't really work all that well, being bash and random FOSS smushed together with duct tape. Oh, and you have money to burn.

I don necessarily disagree, I was just pointing out one commercial virtualization solution that theoretically could operate large scale deployments. I don't typically see alternate products like Proxmox come up often in discussions (I honestly only know one person personally using it), but virtualization isn't my forté either, so there's that.


oVirt is open source, although it unfortunately doesn't have a large development community. But it would still be a good starting point if someone wanted to create an open source VMware replacement.


Perhaps the poster is using the term 'virtualization' imprecisely, if s/he is deep in the VMWare world. Another interpretation would be that companies are looking to move to containerization on OpenShift, which would make a lot more sense.


If you read it carefully, it's very clear that the poster is talking about multiple customers, not just one.

A dozen or more companies (poster claims it's exactly 14) can move 100k-500k VMs in a short time frame, without killing anything.

Otherwise, your concern would have been spot on.


I didn't see the 14 on my first read through last night (I read this at like 1:30 in the morning), but that would make much more sense. I interpreted the "some" as a handful at most, which is why I said "footprints" and not "footprint". Even in that situation, I would probably still recommend not going the OpenShift Virtualization route. Obviously these groups would have different deployment sizes, but I would still personally be wary of moving several thousand VMs to OpenShift as part of a cost saving measure if there wasn't an intent to migrate to containers over time (based on the OpenShift Virtualization of today).

That's not to say it's not possible or a bad idea, just that each org would need to evaluate for themselves. We do have customers that have made this just and are running OpenShift clusters with >1k VMs on OpenShift Virtualization.


So OpenShift virt replaces oVirt? What was the reason for dropping oVirt?


Remember, oVirt an upstream virtualization project, the product we derive from it is Red Hat Virtualization (RHV). From the development and deployment side of things, Red Hat has chosen a direction that is primarily container based going into the future, betting a lot of this on OpenShift. This decision was made a few years ago before anything with Broadcom/VMware was occurring.

OpenShift Virtualization (based on KubeVirt) exists to provide a path of migration for "legacy" VM environments to containers, allowing admins to maintain one platform in a consistent manner (Kubernetes resources) while giving the teams that can to evolve their applications to containerized deployments.

I believe there are plans to get the management capabilities of RHV into OpenShift Virtualization, I'm not quite sure how far that's gotten. In terms of virtualization solutions from Red Hat, we have the following:

- KVM on RHEL. With the announcement of the RHV EOL, we removed the restrictions on how many guest VMs you can run simultaneously on RHEL. Note that this is different from the RHEL for Virtual Data Center (VDC) subscriptions we sell; it's just the removal of the contractual limitation, it doesn't entitle your guests.

- Red Hat OpenShift Virtualization: already discussed.

- Red Hat OpenStack: If you need to be running a broad private cloud platform that has a virtualization component.


KubeVirt is the upstream. Essentially so there is common management/operational model between virtualization and containers rather than having a Kubernetes-based one and a largely independent virtualization one.


Yeah, I think the hint is that something is going on here.

Like possibly a national security issue of some kind.

'We' are at 'war' with Russia, after all.


I've no information on what's going on, or even if anything's going on. Other than today's SEC announcement of fraud findings: <https://techcrunch.com/2022/09/12/the-sec-revealed-today-tha...>

That said, I'd keep an eye on any future 0-day announcements ... if there's anything here, it has that kind of smell about it.


That's what I mean. And because it's CEOs and not CTOs it's likely a Zero-Day related to something geopolitical.

The West is at war, my downvoting detractors are naive to not contemplate that the world's #2 Army is not running shenanigans as they have literally publicly stated they would, and we have direct evidence to support it. And literally Biden gov. urging US businesses to 'patch up' at the start of the war. They are looking for whatever leverage they can. And there are other capable enemies notably China and North Korea.

If in fact this rumour is true, i.e. 'CEOs doing this at the same time' then this could very well be a national security issue.


That's close to my read, though again, based on a report of undetermined validity and circumstantial information only.

So: heightened alertness, but not immediate panic.


My suspicion: these CEOs are trying to scope out the cost of adopting OpenShift, so that, when they're negotiating prices with VMWare, they can show how much money they would save with OpenShift in the long term. This may be essentially just an attempt to gain some negotiating leverage in the face of potential VMWare price hikes.


Not to mention that you need to have the ability to switch anyway in case there is some other problem with VMWare. Anytime you have a single supplier that you depend on you should have some mitigation plan ready to go should that supplier fail you. The mitigation plan may be a lot more expensive than the supplier you are using, the goal isn't to use them it is just to ensure you have an option.


We have an open source tool for moving off VMware to KVM: https://libguestfs.org/virt-v2v.1.html

It installs virtio drivers into Linux & Windows guests and updates configuration and registry, so the VM will boot straight away on the target.


Interesting, it does seem plausible that the cause is the VMWare price hike; as they specifically stated aiming at large enterprises because they’re slow to move, I wonder whether they may be overplaying their hand.

On the other hand, it’s hard to imagine these enterprises moving have a million of VMs before the end of the year, so they probably do have a point. But it just seems like short-sighted short-term thinking here, giving up enterprise market share in exchange for some additional $ now.


Any reduction in memory managed by VMWare would be savings though. Say they need 50% of the VMWare license this year and 5% next year they're probably still happy.

At least happy until whatever IBM has been promising behind closed doors runs out, and then IBM start turning the screws.


I am struggeling so hard... IT departments of two customer companies just moved VMs with a one week deadline to Azure and everything stopped working and I have the responsibility as external developer...

Weird SSL errors I have no clue how to fix. I think they are talking about my clients in this reddit post.


> Weird SSL errors

Wild stab in the dark: the only things that would kill SSL/TLS on a disk-image based lift & shift where the certificates and associated config moves with the VM are:

- An internal Certificate Revocation List (CRL) Distribution Point (CDP) was forgotten about and not moved along with everything else. These are often Enterprise PKI certificate authorities on servers like AD domain controllers. Admins will typically deploy new domain controllers in the cloud, and move everything else. They'll forget the old CAs and maybe even turn them off. This then causes SSL issues after... about a week.

- Another possibility is that they incorrectly moved an outbound access restriction. Again, overzealous network security admins tend to block Internet access on servers and forget about CRLs. It's an especially common issue on "secure" environments where someone decides to block HTTP outbound and only permit HTTPS because it's "secure". However, CRLs (and OCSP) absolutely require HTTP and will never work via HTTPS by design.


Azure stopped supporting TLS 1.1 recently as well.

If these are old systems, it could be this... if they're running .NET apps compiled against framework pre... 4.7? 4.8? TLS ain't gonna work...


They would only be affected by the Azure TLS changes if they already used Azure APIs while on-prem, which would be unusual for an on-prem-to-cloud lift & shift.


Fantasy fiction of a sysadmin that uses anonymity on the internet to craft themselves into someone more important than they are in real life.


This seems unfair, it isn't necessary to attack the author just because the tone is a little conspiratorial. I'm sure there is a little embellishment in the post, but there doesn't seem to be any reason to doubt the author has noticed a real effect that may have a reasonable explanation. The VMware price hike idea seems like a very reasonable explanation


I lived through EMC's VMware tightening their grip, killing off our vendor "reflex firewall" by restricting their use of network APIs (to boost their Nicira acquisition which became NSX), then making us pay the vRAM vtax as part of the VSPP program (public cloud), and launching project Zephyr (vCloud Air) where they directly competed with us.

RIP VMware. Been getting Novell vibes from them for the last 10 years.


Novell... Now that is a name I haven't heard in a long time


The hate for IT/sysadmins here is funny. The above would of course never happen here or if they where a dev!


*were ?


This, unfortunately.

Fortunately it's quickly nipped in the thread on reddit. The news isn't new, and companies already had shy of a quarter to work on it, but not that we hit Q3 and forecasting for Licensing is now kicking in for Q1-Q2 23' it's now sudden that the capex will be $$$ so there is a huge rush to migrate to mitigate licensing cost and sunk cost of a yearly licensing option.

Most IT/Sysadmins do not communicate with Finance/FPA, but the ones that do are really ahead of the game. I'm fortunate that in my past and present org(s) I always include them on talks, and once they are on your side they are an invaluable ally. Money talks, finance talks.

IT and Finance are back office administration, or the other side of the house in many orgs and should be talking about all terms of licensing and proper forcasting, but I do see it strained because IT focuses on what the tech can do, and finance only understands money and liability.

But IT can also mitigate liability if it's framed properly. This MSP sysadmin is clearly only in tech and never went out of it. Which isn't bad. But it is a very, very common silo and trope for external IT partners.


> Fantasy fiction of a sysadmin that uses anonymity on the internet to craft themselves into someone more important than they are in real life.

No. Everything on reddit is true.


On HN too

(On the fence if this story is true or not, it could very well be)


What’s strange here is OpenShift as a target. The last I looked, VMs were just getting initial support - certainly not enough functionality or proving to encourage one to rehost 100k VMs onto it as a platform. Also why the assumption that IBM would treat them better than Broadcom/CA?


A commenter on the thread mentioned the VMWare price hikes, so it could be as simple as IBM sensing an opportunity to poach customers from VMWare (probably more overlap in potential customers than AWS/GCP/Azure) and aggressively selling OpenShift to the C-suite with a promise of "if we can get a deal done this year, we'll give you heavily discounted pricing for X years".

It does seem a bit strange that all of these requests supposedly come specifically "from the CEO" though, as I would expect most companies would have CTOs making purchase decisions like this.


Not strange at all, CEO and Finance talk about budget. If cost is going to 10x, by doing nothing then it is going to be an quick decision. Most CTO's would've brought it up in the last quarter and be working on it now.

This reddit post is just a late canary in the minefield that shows a what one sysadmin is doing at one MSP.


Unless something has changed recently, IBM doesn't push OpenShift but rather Cloud Paks which are tailored solutions built on OpenShift. IBM sellers don't get paid on vanilla OpenShift.

I think if the story is to believed, the Red Hat sellers dedicated to big Fortune 500 accounts were given marching orders to reach out about recent negative publicity regarding vmware and the c-suite is asking for pricing/feature comparisons. This doesn't seem sudden or disturbing but rather what happens multiple times a year.


Though I do agree with your sentiment here, this saying comes to mind:

"Nobody ever gets fired for buying IBM".



For what it’s worth, I’ve heard that when the current war in Ukraine started the Moscow/Russian employees of a big US networking hardware company of which everyone on this forum has heard about had their access cut off instantly, as in the invasion started at around 2AM and when the employees tried to get to work that morning they found out they could no longer access the company’s resources.

Which tells me that that company’s HQ had already put a sort of kill switch in place for cases like this.


2 AM Moscow time is 3 PM Pacific time - if they had someone working west coast times it could be done in regular working hours. Dropping users from their VPN LDAP group (or directly from the VPN servers) doesn't take long.


I know of another SW company that did the same thing. Everybody at the company was then told not to contact or communicate with any of the employees in Russia. Then one of the executives told everybody that it was a vendor who was providing the remote access solution who pulled out of the country in a blame shift we saw right through.


For sure. Critical technology companies for sure have these kinds of plans and things ready.

And these big brand companies with international offices are full of spies it's good cover.


Possibly related to recent VMWare SEC fraud charges

https://news.ycombinator.com/item?id=32834988


I seriously doubt there is a single executive or IT decision-maker in the world that will drop VMware because they were accused of revenue-smoothing.


I'm about 100x more concerned that VMware got bought out by Broadcom than I am that VMware is led by fraudsters.


How is this disturbing? People moving away from Windows must make everyone happy in my book. Probably just leverage in contract negotiations with VMWare as other comments have pointed out though.


It's a different market segment for sure, but at my company we spent considerable engineering resources migrating the codebase away from a proprietary embedded toolchain to gcc. The toolchain vendor's licensing scheme was expensive and annoying, and had no chance of sustainably scaling with the company's growth. They weren't willing to negotiate, and so they priced themselves out of relevance permanently. Maybe they thought we were bluffing?


It’s the sysadmins sub - ie the crowd dealing with the fallout when migrations go bad. So sudden big moves of questionable technical merit are disturbing to them


Sure, but if you have ever administered Windows vs Linux servers, you’ll know which is much more pleasant to work with. Being on the hoo for the migration though …


Top comment:

> VMWare had a price hike in August and is going to a very aggressive subscription model so that may play a role here.

Sounds like CEOs trying to gain leverage in negotiating with VMWare.


Yes, and there are comments about Broadcom's history of price gouging after acquisitions.


This is a possibility when pushing companies from perpetual licenses onto subscription and then upping their subscriptions. Wall street is heavily pressing SW companies to increase their ARR (Annual Recurring Revenue) or some call it, you will own nothing and be happy. Switching to free software is looking more appealing at the enterprise level.


I would suggest scanning through a few pages of OPs history to determine for yourself whether this is legit or not.


I tried to follow the advice. It's clearly someone who is working with computers and for a long time. The comments are touching various topics (from tech to society to random stuff) and it's likely that the person is behind a keyboard, not on a mobile phone: comments are rather long on average. That's quite compatible with sysadmin psychological profile.

Hard to say for sure, but at least I am confident that the OP is not a dog!



how so?


This looks fake. I just imagine someone shorting VMware’s stock on the other side of the keyboard… but who knows.


I quit my role at VMware a couple years ago to return to consulting. Nearly client I have worked with since then has dramatic reduction of vSphere footprint, if not outright elimination, on their roadmap.

VMware was burning bridges long before the Broadcom announcement. Broadcom is fuel on the fire.


Came to say exactly this.

As much as I don't like VMWare as a company, I have to agree, this stinks.


They know something you don’t. That’s how most people make money. In all seriousness this appears to be cost cutting. They sat down with their CTOs and asked give me a list of cost cutting measures right down to the bone. The CTO says boss we pay for windows licenses, we could move to Linux and save a zillions of $$. CEO says “what are you waiting for do I have to tell you to get a new toilet roll if you run out of one ?”


Redhat acquisition of Makara (Aug 2021) probably enabled a sweeter price point for CxO to buy and virtual host themselves.


Has anybody considered that this could simply be a marketing tactic for OpenShift to get people talking about it.


The idea that Fortune 500 CEO's are cold-emailing companies to ask about assistance in re-platforming VM's is plainly ridiculous.

Leadership taking a strategic decision to shop around for options following the VMWare price increase isn't so crazy. In fact, I'd say it's expected.



worth pointing out, that while the timing of these two articles might make you think that, cost increases in light of the broadcom acquisition seems far more likely.


This definitely feels fake. Either just a fabulist or someone mad at VMware and wants to scare them. CEOs of Fortune 500 companies do not know about the Linux virtualization market.


Yeah this seems odd for a Fortune 500 company’s CEO asking specifically for Linux VMs.

There’s not enough hours in the day where these CEOs would be so far down in the weeds of the business that they’re making VM flavor decisions.


I can actually see this happening.

Obviously a fortune 500 CEO does not know jack about IT. But someone has raised the alarm on the price hike and presented a few possible solutions, that have reached all the way to the board.

Given the high risk, tight schedule, and the amount of work something like this must be driven by the CEO (or at least directly sponsored by him) or it will not happen.


This is not from a fortune 500 company, but I know the CEO in a BIG oil & gas company and that CEO do in fact know a lot about IT & computer science. He is in fact a big fan of Neo4j, and gets really frustrated when he talk to software architects/developers that have never heard about neo4j or graph databases.

He also knows about Linux and license free virtualization. But making a move requires people, and people with Linux knowledge are very rare.


Data science and tools are especially critical to oil and gas management. Including downstream: refineries are built for the qualities of available discovery and take as long to plan. Anticipating the availability, extraction costs and chemistry of a new deposit is a major advantage to anyone in the industry. I was given, cold, a six figure number of barrels of Kazakh crude to market, three years before the standard blend was assessed. That was fun...


I looked at your profile and was surprised to see we don't work for the same company!


That's why it's remarkable and worth well remarking about.


As VMware is getting considerably more expensive it's reasonable to assume some CEOs are aware of that. It would be enough if one or two of them for one reason for another googled "cheap virtualization" or talked to some "tech guy" and somehow "Linux is cheap/free and can do virtualization" is what stuck with them. Then they talk to their friends at the golf club and after a while the topic reaches critical mass in those circles and spreads across those companies, probably not without playing "Chinese whispers" along the way.


There are close to 0 Fortune 500 CEOs who would be DIRECTLY reaching out to Linux virtualization companies. There could easily be a CEO who convenes a meeting with the CIO and COO to "Find 10% savings on operational software costs" but the idea that they are directly reaching out to vendors is one of those "tell me you've never worked with a Fortune 500 C-suite without telling me that you've never worked with a Fortune 500 C-suite" memes.


Nowhere in my post did I suggest CEOs reaching out to "Linux virtualization companies" themselves. Even the linked reddit post doesn't. It's talking about CEOs ordering someone/some team from their company to contact a tech consulting company to figure out whether switching to Linux based virtualization would be feasible.

CEOs hearing about some stuff and immediately jumping on it and ordering someone to look into it is quite common.


That depends on the company. Some companies spend a lot more on tech than others. CEOs tend to be aware of where the high costs are, so if your company spends a lot on Tech, I could see the CEO knowing at least something about it.


I think you're underestimating how psychopathic CEOs are.

They might tell someone below them "we need Linux VMs, stat!" the same way they might say "We need to move everything onto the blockchain!", all depending on what they pick up at a conference or the golf course.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: