Feels like postgres is always the answer. I mean like there's gotta be some edge case somewhere where postgres just can't begin to compete with other more specialized database but I'd think that going from postgres to something else is much easier than the other way around.
It's not like PostgreSQL hasn't been in development for close to 30 years, covering basically every use case imaginable just through millions of deployments.
In addition, SQL in itself is a proven technology. The reality is that most problems you might think about solving with specialized databases (Big Data TM etc) could probably easily be solved with your run-of-the-mill RDBMS anyway, if more than five minutes are spent on designing the schema. It's extremely versatile, despite just being one layer above key-value storage.
It's very good. Postgres by itself can handle a very high volume of inserts (I did over 100,000 rows/s on very modest hardware). But timescale makes it easier to deal with that data. It's not strictly necessary but it's very time series friendly (good compression, good indexing and partitioning etc). Nothing a pg expert can't accomplish with a vanilla postgres but very, very handy.
I haven’t tried timescale, but I have found postgres with time-based partitions works very well for timeseries data. Unless you’ve got really heavy indexes, the insert speed is phenomenal, like you said, and you’ve got the freedom to split your partitions up into whatever size buckets makes the most sense for your ingestion and query patterns.
A really nice pattern has been to use change data capture and kafka to ship data off to clickhouse for long-term storage and analytics, which allows us to simply drop old partitions in postgres after some time.
I think timescale will compress them heavily on your schedule so if that's acceptable to your use case you might be able to do away with clickhouse. Hard to say of course, without knowing details around your insertion and query patterns, retention requirements and aggregations you need. But timescale can do a lot of that with pretty straightforward syntax.
I have used TimescaleDB in my last work place. We needed a easy way to store and visualize 500hz sensor data for few 10s of devices. We used it and Grafana to build a internal R&D tool and it worked way better than I imagined. Before I left I think the DB was using ~200GB on a compressed btrfs volume in DigitalOcean droplet and still performed fine for interactive Grafana usage.
What situations do you encounter where you don't care about the structure of the data? The only ones I've ever encountered have been logging, where it's only purpose is to be manually text searchable, and something like OpenStreetMap where everything is just a key value store and the structure is loosely community defined.
As soon as you have a loosely defined object you can't access any specific keys which makes it useless for 99% of times you want to store and retrieve data.
PG requires a lot of expertise to keep running when you get to a billion rows or massive ingest. It can do it, but it doesn't just do it out of box running the defaults.
Unfortunately, most places go with hardware first, so the problem grows larger and larger. When they do finally hire someone who knows how to address it, it’s a monstrous undertaking that dev teams are reluctant to do, because it’s a lengthy pause on feature development (so really, PMs are reluctant), and also future velocity is typically somewhat slower – turns out proper RDBMS data modeling is rigid, and doesn’t suffer fools.
Till about yesterday, MySQL was little more than a CSV file on wheels, and even current popularity is mostly driven by two decades of LAMP.
You can be sure that PostgreSQL will be applicable for any work load that MySQL can handle, but not necessarily the other way round, so if you actually have the freedom to make a choice, go with PostgreSQL.
In particular, because PostgreSQL has a lot more features, people imply that the other one, for the lack of those features and its associated complexity, must automatically be faster. Which isn't true, neither generally, nor in special cases, since the latter one can go either way - your particular query might just run 4x on PostgreSQL. There is also no universal approach to even benchmark performance, since every database and query will have completely different characteristics.
> Till about yesterday, MySQL was little more than a CSV file on wheels
That's utter nonsense. What are you even referring to here?
InnoDB is a rock-solid OLTP storage engine, and it's been MySQL's default since 2010. A very large percentage of the global economy relies on InnoDB, and has for quite some time.
<Not intending to start a flamewar...> MySQL shines for simple OLTP. My PostgreSQL friends all have "paged at 3am to vacuum the database" war stories. I like simple and dumb for transactional data and MySQL delivers.
Not that this solves it but we could easily cover a whole state in trees. Have you seen Indiana and Illinois. Those states are entirely made out of corn. At this point it isn't an engineering problem it's just an economic one.
I highly recommend Stargate. They actually go to Atlantis. I know most media outlets have to say Atlantis is "just a story", but if you want the real truth. Go to the Canadians.
I actually was in a situation where I had such a debate during Covid with someone I was in school with. I tried to take time to argue.
He would show some numbers and make completely wrong conclusions. There was no way I could get him to try to listen to me. Whereas back at school, he was never arguing with me about maths given that he had bad grades and I didn't.
So somehow I really feel like there is this "authority" thing missing. Now that he is an adult, he feels like his opinion about everything is valid.
Your comment makes me wonder if a large part of what's driving this is the proliferation of so many disparate (sub)cultures, and specifically the brain drain away from the subcultures that had been the traditional societal defaults.
When you were at school, you were both part of the same social hierarchy. On the topic of math proficiency (general smartness, really), it was easy for him to perform a mental check and see that you were above him in that aspect (even if he was above you in other aspects), and so he should listen to you.
But then you probably went your own way, essentially getting out of his way with him continuing to advance in that traditional subculture. So now he feels he's earned being considered closer to the top of the hierarchy, even though it's no longer a pan-society hierarchy!
And then he also cannot understand where to slot you into his mental model. So you just get written off as an outside attacking force to be opposed.
This also explains why they continue calling themselves "conservatives" despite gleefully working to destroy our institutions and standing in the world - what they want "conserved" is their perceived prominence of the subcultures they bought into.
That's an interesting point of view: in a way he "doesn't know me anymore", so why would he trust me? And he has to trust me because if he doesn't understand why I am better in this specific topic (maths), then the only thing left is his belief...
I once heard flat-earthers in a train. One was saying "It's like school, e.g. physics: they purposely make it such that we don't understand it to confuse us. Because we're not worse than anybody else, so we should understand, right? If nobody understands, that's because they bullshit us!". I really wanted to say "counter-example: I could understand physics at school :-)".
Thing is that people are entitled to have different opinions and we can argue about those all day. But people are not entitled to different facts, and if someone pretends 2+2=5, then there's very little arguing about that. It's not that I'm too tired to argue, but if someone is so far out of the common ground, there's no basis on which it makes sense to argue because they'll just declare all of your premises void and "win".
And perhaps a broader point is: "Facts are simply opinions we generally agree upon."
At its best, "Science" doesn't claim the force between two objects is proportional to the product of their masses. It merely observes that this has been the case in all known, well-documented instances, and attempts to find exceptions have failed, so that this "law" is valid and reliable, and expected to remain so forever.
We use the term "fact" very casually, and eagerly. "The Dow is up 20% year-over-year, unemployment is under 4%, and the dollar is strong. The economy is strong. That's a fact!" Not only is that not a fact, it's not even a fact that unemployment is under 4%. That's (presumably...) just the U3 number as published by the Department of Labor.
My point is: the best "facts" still require a certain amount faith in the data sources around us. That's fine. I don't wish to visit every country on Earth to ensure that it exists, and observe the weighing of the Kilogram in Paris. (I mean, unless I can take time to sightsee a bit.)
There has been a genuine breakdown in that faith in authority in US culture. Maybe Ukraine attacked Russia first. Maybe poor people are inherently lazier than rich people. Maybe the Earth is flat.
I believe this breakdown has been intentionally orchestrated and groomed and fed by nefarious people like Rupert Murdoch, but that's just an opinion. Based on things I like to call "facts".