Today it's usually implicitly designed for iphone, designed for 1080p, or ipad, and you have to guess, strong correlation with whatever device the designer uses in his personal life.
> Today's sites use responsive design and adapt to pretty much any screen size.
Today sites certainly can and some (many) so. But some (also many) definitely don't…
A lot are locked to a maximum width, which is OK enough as l……o……n……g lines of text are unpleasant to read, but only because browsers hack the meaning of dimension settings to make text zoom work consistently.
A lot also have an effective minimum width (even if they use responsive styling to move/ minimise/hide side decoration before a certain point) that is not always convenient. Try browsing with a thin window so you can have something in the other side of your screen. Some assume no one on desktop will ever have a browser window less wide than 1280 pixels (or equivalent on a zoomed higher res screen) - not the case on my 1080p portrait screen and I sometimes want things thinner than 1280 on my 2560x1440 screen. You could say I'm just odd and they can't cater for everyone, but 1080 or a bit less wide is hardly miles away from many devices physical layout so if a design can't display nice in that can it really call itself "responsive" (I suspect any such design would fail on many mobile devices too - 1080px effective width is rather common there, as are smaller widths).
You can assert whatever you want, but Polars is a great answer. The performance improvements are secondary to me compared to the dramatic improvement in interface.
Today all serious DS work will ultimately become data engineering work anyway. The time when DS can just fiddle around in notebooks all day has passed.
Pandas is widely adopted and deeply integrated into the Python ecosystem. Meanwhile, Polars remains a small niche, and it's one of those hype technologies that will likely be dead in 3 years once most of its users realise that it offers them no actual practical advantages over Pandas.
If you are dealing with huge data sets, you are probably using Spark or something like Dask already where jobs can run in the cloud. If you need speed and efficiency on your local machine, you use NumPy outright. And if you really, really need speed, you rewrite it in C/C++.
Polars is trying to solve an issue that just doesn't exist for the vast majority of users.
Arguably Spark solves a problem that does not exist anymore: single node performance with tools like DuckDB and Polars is so good that there’s no need for more complex orchestration anymore, and these tools are sufficiently user-friendly that there is little point to switching to Pandas for smaller datasets.
> Pandas is widely adopted and deeply integrated into the Python ecosystem.
This is pretty laughable. Yes there are very DS specific tools that make good use of Pandas, but `to_pandas` in Polars trivially solves this. The fact that Pandas always feels like injecting some weird DSL into existing Python code bases is one of the major reasons why I really don't like it.
> If you are dealing with huge data sets, you are probably using Spark or something like Dask already where jobs can run in the cloud. If you need speed and efficiency on your local machine, you use NumPy outright. And if you really, really need speed, you rewrite it in C/C++.
Have you used Polars at all? Or for that matter written significant Pandas outside of a notebook? The number one benefit of Polars, imho, is that Polars works using Expressions that allow you to trivially compose and reuse fundamental logic when working with data in a way the works well with other Python code. This solves the biggest problem with Pandas is that it does not abstract well.
Not to mention that Pandas is really poor dataframe experience outside of it's original use case which was financial time series. The entire multi-index experience is awful and I know that either you are calling 'reset_index' multiple times in your Pandas logic or you have bugs.
"Data Science" has never been related to academic research, it has always emerged in a business context. I wouldn't say that researchers at Deep Mind are "data scientists", they are academic researchers who focus on shipping papers. If you're in a pure research environment, nobody cares if you write everything in Matlab.
But the last startup I was at tried to take a similar approach to research was unable to ship a functioning product and will likely disappear in a year from now. FAIR has been largely disbanded in favor of the way more shipping-centric MSL, and the people I know at Deep Mind are increasingly finding themselves under pressure to actually produce things.
Since you've been hanging out in an ivory tower then you might be unaware that during the peek DS frenzy (2016-2019) there were companies where data scientists were allowed to live entirely in notebooks and it was someone else's problem to ship their notebooks. Today if you have that expectation you won't last long at most companies, if you can even find a job in the first place.
On top of that, I know quite a few people at the major LLM teams and, based on my conversations, all of them are doing pretty serious data engineering work to get things shipped even if they were hired for there modeling expertise. It's honestly hard to even run serious experiments at the scale of modern day LLMs without being pretty proficient at data engineering related tasks.
I have not work with Polars, but I would imagine any incompatibility with existing libraries (e.g. plotting libraries like plotnine, bokeh) would quickly put me off.
It is a curse I know. I would also choose a better interface. Performance is meh to me, I use SQL if i want to do something at scale that involves row/column data.
This is a non-issue with Polars dataframes to_pandas() method. You get all the performance of Polars for cleaning large datasets, and to_pandas() gives you backwards compatibility with other libraries. However, plotnine is completely compatible with Polars dataframe objects.
it's the management structure that's broken. Plenty of decent engineers around microsoft who could fix it, plenty of customer and enterprises willing to pay, but they are not allowed to work on it because of prioritization bullshit, allegedly they could get more money elsewhere
That's literally the issue, management by KPI frameworks
I think it has more to do with bundling reducing the need to compete to zero. Change that and the economics of competition would take over and the changes would get prioritized but nobody at Teams needs to sell a single license, so the priorities become the bs like internal status and visibility and not product success.
How many companies have Teams for basically free with their 365 license but still pay for Slack? The marginal value of Teams is nearly zero.
There is also a matter of selective effort by staff senior enough to make their own choices. Many SDE3 (or whatever MS equivalent is) wouldn’t want to be associated with a dumpster fire product like Teams.
You check the transaction history. So your argument is that they are worth equal as long as the buyer doesn’t have all the information? It’s like claiming that a new CPU is worth the exact same as an old broken one because you can trick someone to buy it for full price if they don’t check it before paying.
This brings to my mind: Chainalysis or the exchanges Co should offer such service like "bitcoin cleanyness lookup" or similar :-)
(maybe some do already and Im not aware of...)
But the thing is: These service work wallet-based.
It has different issues, but wireless headsets nor hibernation are among them
reply