It is true that every major DB ventor, SQL or not, is smashing the AI/vector keyword on their front pages. In Elastic for example, their vector capabilities have gone from laughable to respectable in a year. Its a lot simpler to just use one DB instead of many.
But a question for true DB experts here:
1. Is there any real advantage to building a dedicated vector DB from scratch?
2. Is vector DB something that can be just 'tacked on' to a normal DB with no major performance penalties?
We know from history, that data warehouses are genuinely different from databases, and cloud data warehouses are overwhelmingly superior to on-prem ones. So that emerged as a distinct, enduring category with Snowflake/Databricks/Bigquery.
Data warehouses are columnar stores. They are very different from row-oriented databases - like Postgres, MySQL. Operations on columns - e.g., aggregations (mean of a column) are very efficient.
Most vector databases use one of a few different vector indexing libraries - FAISS, hnswlib, and scann (google only) are popular. The newer vector dbs, like weaviate, have introduced their own indexes, but i haven't seen any performance difference -
Elastic does a great job with it. My one-person company builds software that mirrors emails from IMAP accounts to ElasticSearch, and adding a vector search on top of that data to "chat" with emails was fairly simple. I was expecting there to be an untold number of hurdles, but the only requirement was to have at least v8.8.0 of ElasticSearch (this was when they increased the supported vector sizes so that OpenAI embeddings would fit into it), and that's it. https://docs.emailengine.app/chat-with-emails-using-emaileng...
Single node database systems that are not horizontally scalable and that are not built on a distributed system foundation (e.g. Postgres) will certainly have scaling bottlenecks if you just add more and more complexity to the workload... however many modern database systems are built on a distributed system foundation with horizontal scaling and the ability to independently scale different constituent parts of the backend.. these engines should have no problem
The trade off that you are interested in isn't about storing vectors, but rather about whether an index should be a part of the DBMS or external to it.
Some advantages of having a separate index is that it can work with different backends, it can be independently scaled, and it can index data for more than 1 database server.
Some disadvantages are increased latency, increased complexity, and distributed system problems.
After spending the last six months working with a vector database I qualified postgres with its vector extensions this morning and I am trying to toss out everything else.
The operational pains if you need to self host this stuff are real, split brain, backup/restore not really considered (compared to a normal databases features), things like replication and sharding _exist_ but often are a buggy mess.
OLAP is definitely distinct from OLTP, and most of these vector queries have some aspect of both - they are similar to OLAP in that they need a decent amount of preprocessing to be useful (inferrence) and they are similar to OLTP in that they are often used for serving point queries or tiny lookups.
GPT written code gets committed to github (After human review, so its working code only). Github data gets shared with OpenAI (Microsoft). So GPT continues to improve.
The recent deterioration is due to reduction in parameter count for GPT-4 to save money.
The customers want to donate money to local taxi operators and drivers?
The taxi drivers who barely understand what open source is?
This can work, but definitely not through crowdfunding. This better works through say business associations, say a national taxi association (composed of all small/regional taxi operators). That association can allocate a small portion of the funds to build this open taxi framework.
The truth is that I'd be happier donating to the project of a non-exploitative marketplace platform than tip a driver out of starvation while virtually all of the nominal rate goes to making a unicorn golden.
I think that is extremely obvious, given their interface looks 90% like github.
They are trying to branch out into training AI models and running inference themselves. But their costs for inferencing look extremely uncompetitive compared to GPU providers like coreweave.
Which, per the EPA [0], is roughly the tailpipe CO2 you’d emit by burning a little under 3 gallons of gas in your car, right? Which is to say, driving 65ish miles in the US [1]?
It amazes me that computational feats of this magnitude can be so energy efficient in the scheme of things.
It’s not that surprising actually, computation moves nothing and all energy has to turn into heat. If computation would use as much energy as moving physical objects, the heat would burn everything down
H100s are intended to be retailed piecemeal. The big enterprise model is DGX GH200, at $10 mil each.
Its just that current supply is extremely short, so the H100s end up only available to big buyers. But that will be resolved in time. Nvidia wants every university lab to have a H100 so no competitor sneaks in there.
This seems like a very promising product, a ChatGPT interface, enterprise version, is a large gap in the market.
However, this part in your advertising, sounds very dubious:"Credal can be deployed fully on-premise for Large Enterprises, including the large language models themselves"
What do you mean the LLMs themselves? Open source I can understand, how are you going to move GPT-4 to on prem? OpenAI is not giving you the weights.
Thanks for your encouragement and that is a totally fair criticism - when we say that, we mean two things:
1. We support using Credal with your own, open source LLM, which can of course be fully on prem in every sense
2. We also support using Credal with your own Azure OpenAI instance. As you say, OpenAI aren't giving us the weights, but many of our customers have procured Azure OpenAI from Microsoft and then we point GPT 4 usage at their Azure instance, meaning that the data never goes to Open AI at all.
One of the things that's going to be really interesting to see moving forward is whether the open source models are going to be able to compete with the blistering pace and funding that the closed source ones - Bard, Claude, and GPT-X are going to be able to attract (and maybe Mistral?). For the sake of the industry, I really hope that the OS models catch up but given the amount of funding (and now in OpenAI's case, revenue) the closed source models are generating, its hard to see how that happens
The cutting edge for graphics is all raytracing, and Nvidia still dominates. DLSS3, pathtracing, etc, these are for 'graphics', but heavily dependent on AI post processing, so Nvidia still rules.
So in the gaming market, Nvidia still commands a huge premium. No AMD card can play Cyberpunk on overdrive 4k.
> The cutting edge for graphics is all raytracing, and Nvidia still dominates. DLSS3, pathtracing, etc, these are for 'graphics', but heavily dependent on AI post processing, so Nvidia still rules.
IMHO, unless you have a $1000+ GPU, RayTracing is still not worth the performance hit. I prefer playing at 100+ FPS with RayTracing off, than turning it on and have my frame rate cut in half.
The stock market reacted quite positively to this acquisition (Basically confirmed a few days ago). Buying a soon-bankrupt startup is very cheap, and getting a strong tech stack and already formed team with strategic synergies with the main business is going to be valuable.
This is contrasted with panic acquisitions like say Adobe & Figma.
And so, if this is the first time you have heard about Snowflake acquiring Neeva you now know the level of asymmetry of information that exists in the 'public' market.
Use this as a lesson to exercise caution when personally investing in individual stocks in the public markets - even if you are buying tech stocks and feel as though you follow tech trends closely because you work in the industry.
Wall Street already knew and priced it in before you did.
Traders may have heard this from The Information: "Snowflake in Talks to Buy Search Startup Neeva in AI Push" from 5/17, then "Snowflake Nears Acquisition of Search Startup Neeva" from 5/19.
Which is totally fine, unless you’re day trading, which you probably shouldn’t be if your goal is to maximize returns.
Day to day and month to month ups and downs don’t matter if your investment time horizon is long.
The dips are actually great because they create opportunities to buy more of the stock cheaper, assuming you’re not planning to sell it all in a few days.
Okay, but if we assume last week's small rise is attributable to the confirmation of the Neeva acquisition, that seems more like a "market reacted tepidly" to me.
But a question for true DB experts here:
1. Is there any real advantage to building a dedicated vector DB from scratch?
2. Is vector DB something that can be just 'tacked on' to a normal DB with no major performance penalties?
We know from history, that data warehouses are genuinely different from databases, and cloud data warehouses are overwhelmingly superior to on-prem ones. So that emerged as a distinct, enduring category with Snowflake/Databricks/Bigquery.