IME this is a longstanding pain point with Go. There's an attempt to propose an encoding/json/v2 package [1] being kicked around at the moment [2], spawned from a discussion [3].
This at least seems to improve the situation of marshalling to/from an interface directly slightly by providing the ability to pass custom Unmarshalers for a specific type (via json.WithUnmarshalers and json.UnmarshalFunc) to the Unmarshal functions, but it appears to still have the inefficient double-decode problem. Or I just haven't found a decent way around it yet.
Looks like they're intentionally punting on a first class solution until (if) the language gets some sort of sum type, but I still think the second-class solution could do a bit more to make this extremely common use-case more convenient. Pretty much every serious production Go app I've worked on in the last 10 years or so has had some horrible coping strategy for the "map a field-discriminated object to/from implementations of an interface" gap, often involving some sort of double-unmarshal.
Quote from the proposal [1]:
> First-class support for union types: It is common for the type of a particular JSON value to be dynamically changed based on context. This is difficult to support in Go as the equivalent of a dynamic value is a Go interface. When unmarshaling, there is no way in Go reflection to enumerate the set of possible Go types that can be stored in a Go interface in order to choose the right type to automatically unmarshal a dynamic JSON value. Support for such use cases is deferred until better Go language support exists.
Understatement of the year. But it’s really not limited to encoding but generally lack of sum types is excruciating after having tasted them (in Rust, in my case). They click instantly as an abstraction and they solve countless real-world logic bugs. Not to mention their ergonomics in seemingly unrelated things like eliminating null and error handling with result types. Just sprinkle some pattern matching on top and you’re in paradise.
That's not what I found in my own experiments, I still had to unmarshal once inside the callback to get the `type` field out, then again once I knew what the type was. Do you have an example handy?
I don't know about recently, but people were asking about them from the first announcement in 2009; and got the answer that they were "under consideration"
To be fair, it's a significant advantage of go that they have been strict about keeping it's feature set small.
Edited to add:
There is this 2023 proposal from Ian Lance Taylor (on the go team) https://github.com/golang/go/issues/57644
But it makes all sum types include nil, which seems suboptimal
> I would've probably picked the modernc variation
Heads up about the modernc library, it has been stuck on an old version of sqlite for several months [1]. It seems like maintainer time is the limiting factor [2]. There has been a call to arms on that issue page, the maintainer is looking for help, but it looks like not much has arrived. It seems like it might trace back to blockers in the C-to-Go compiler.
It's a major undertaking and a very impressive piece of work, but I'm not surprised it's a struggle when big roadblocks get hit. I hope they find a way to progress, but I'm very relieved to be seeing some CGo-free alternatives like ncruces/go-sqlite3 emerging. I'm going to give it a try for sure and see if I can live with the compromises.
Squinn-go looks very compelling too, but I don't like that it requires the squinn binary to already be installed on a user's machine, I think that gives with one hand and takes with the other: sure, I get to avoid CGo, but I also lose the turnkey, single-command install, static build benefits Go brings out of the box.
Seconding the point about nitty gritty, I'd read it for sure too!
About my driver, the major compromise to be aware of is that WAL mode support is limited. No tiptoeing here: I call that caveat out in the front page.
My driver reimplements the entire SQLite OS Interface (VFS) in pure Go. This makes it very portable, but it's also a risk: it's definitely not as extensively tested as SQLite's. And the WASM sandboxing model makes shared memory a hard problem to solve, which prevents WAL mode from working.
I have some ideas for work arounds, but: they're hard to implement, and hard to "prove" correct. Idea 1 is a VFS for WAL where SQLite believes it's using memory journal mode, but the wrapper uses the standard WAL format to persist disk. It's a lot of work, but standard SQLite tools can recover a crashed DB. Idea 2 is for the WASM host to simulate shared memory through copying. This should work (there are explicit locks and memory barriers to use as copy points), but it's hard to "prove" correct (without bothering SQLite developers to death with details).
I had some hope that the work being done in libsql to virtualize the WAL would help, but their API for this is just terrible. They simply extracted and exposed internal SQLite stuff, but have no documentation, no usage examples, or even any public VFS implementation using these APIs. And I honestly don't trust their ability to maintain a quality fork. An API that's not dogfooded nor documented is not an API.
Thanks for the info, I think I'll be using this SQLite library next time this comes up, then. I do think the modernc library is really cool, but there are some aspects of it that I already disliked, so I'm more than willing to see if the web assembly approach is better (it looks surprisingly tractable.)
One thing I tried to make sure, to avoid the pitfall modernc is having, is to make sure building "the WASM BLOB" is easily reproducible with widely available tools:
I do apply some light patches to SQLite, but so far they've always cleanly applied, and I can produce a new release within hours of being notified of SQLite releases.
Currently not, there are significant problems that I'm trying to solve before making it available. No, I'm not trying to make it perfect, but I'd like it to be actually usable before opening the code.
Since PEP 591 [1], if using mypy with Python 3.8+ or the typing_extensions module, you can also take advantage of typing.Final, which lets you statically verify something isn't changed. The catch is that it isn't enforced at runtime.
There are fragments of discussion about the download timeout throughout the issue tracker, which end up leading back to this still-open but seemingly forgotten issue about adding InactivityTimeout: https://github.com/golang/go/issues/22982
I'd love to see this one addressed but it's not looking too hopeful at this stage.
There's also the "That Won't Work" person's sibling: the "I Don't See a Use Case" person. In my experience, "I Don't See a Use Case" people require more sophisticated tactics if you actually want to get something done while they're around, not least of all because they move the game to a harder field: "that won't work" at least states a fact, whereas "I don't see" moves us firmly into "belief" territory, and "a Use Case" invites us into the trap of dreaming up our own strawmen for them to tear down one by one.
I wonder if they started out as "That Won't Work" people, but evolved better defenses?
When I hear this phrase, I think of GNOME. Can't count how many times the developers of GNOME have knocked back perfectly sensible pull requests or feature proposals on this basis.
...often only to implement the exact same feature later when it appeared to be their own idea instead of coming from the outside.
It's just an excuse to be insular and shut anything down, which is their default response to anything coming from the outside.
I promise I’m actually curious and not trying to be difficult: is this the same as asking “what problem does this solve” or something different? I’d love a concrete example but I know that might be hard without using a real topic.
I think it could be the same but it doesn't have to be. "What problem does this solve" is a bit more generous as at least, on the face of it, it may offer an opportunity to provide an answer that will be received in good faith.
But I think this is less about specific phrases one might need to be careful not to say, and more about the kinds of default attitudes that can turn into ongoing impediments to the free flow of ideas. It's definitely possible ask "what problem does this solve" without becoming the "What Problem Does This Solve Person", and the same goes for the other phrases too.
The phrasing is less important than the attitude - if you're actually open to a use case to justify the request, I wouldn't care how you asked. If you want me to put a round in the chamber so you can shoot my idea down, I think I'll keep my user stories to myself.
I really didn't enjoy the game with the biters turned on, but absolutely loved it when I switched to peaceful mode. There are enough problems to sort out without them, like "why do I suddenly not have enough iron plates", or "why do I suddenly not have enough copper plates", or my personal favourite, "why do I suddenly not have enough iron plates again"!
"Why did I install the Angles and Bob's mods", "How do I build a waste processing loop so my entire base doesn't lock up", "Why did my wife and kids leave me"
This at least seems to improve the situation of marshalling to/from an interface directly slightly by providing the ability to pass custom Unmarshalers for a specific type (via json.WithUnmarshalers and json.UnmarshalFunc) to the Unmarshal functions, but it appears to still have the inefficient double-decode problem. Or I just haven't found a decent way around it yet.
Looks like they're intentionally punting on a first class solution until (if) the language gets some sort of sum type, but I still think the second-class solution could do a bit more to make this extremely common use-case more convenient. Pretty much every serious production Go app I've worked on in the last 10 years or so has had some horrible coping strategy for the "map a field-discriminated object to/from implementations of an interface" gap, often involving some sort of double-unmarshal.
Quote from the proposal [1]:
> First-class support for union types: It is common for the type of a particular JSON value to be dynamically changed based on context. This is difficult to support in Go as the equivalent of a dynamic value is a Go interface. When unmarshaling, there is no way in Go reflection to enumerate the set of possible Go types that can be stored in a Go interface in order to choose the right type to automatically unmarshal a dynamic JSON value. Support for such use cases is deferred until better Go language support exists.