We have this nifty util in our codebase:

```ts

/*

* A function that asserts that a value is never.

* Useful for exhaustiveness checks in switch statements.

*/

export function assertNever(x: never): never {

  // eslint-disable-next-line @typescript-eslint/restrict-template-expressions

  throw new Error(`Unexpected object: ${x}`)

}

```

Do any of these tools use the rich information from the AST to pull in context? Coupled with semantic search for entry points into the AST, it feels like you could do a lot…

Don’t they all do this? Surely they’re not just doing naive text, n-gram, regex, embeddings, etc, right?

they are just a language model generating text which happens to be code, ASTs are not used afaik

> Their fundamental issues (black box + hallucinations)

Aren’t humans also black boxes that suffer from hallucinations?

E.g. for hallucinations: engineers make dumb mistakes in their code all the time, normal people will make false assertions about geopolitical, scientific and other facts all the time. c.f. The Dunning Kruger effect.

And black box because you can only interrogate the system at its interface (usually voice or through written words / pictures)

A 15 minute side project I cooked up on a late Monday night.

Fun examples:

```

  howto git:(main) howto "print hello world" | sh
  hello world

  howto git:(main) howto "rebase changes from a branch B onto main that was branched off from a branch A that has now been squashed into main"
  git checkout B && git rebase --onto main A B

  howto git:(main) howto "update macos"
  softwareupdate --all --install --restart

  howto git:(main) howto "list all direct non-empty child directories in current directory, each directory on a new line with no additional details" | sh
  target
  .git
  src

```

GPT-4 (note I do have specific instructions in my settings which push the model towards step-by-step reasoning)

To calculate the number of legs for ten elephants, considering that two of them are legless, we first need to know the standard number of legs on an elephant, which is four. Then, we subtract the missing legs for the legless elephants.

- Normal elephants have 4 legs. - Legless elephants have 0 legs.

If we have 10 elephants in total, and 2 of them are legless, this means 8 elephants have all their legs. The calculation would be:

- \( 8 \text{ elephants} \times 4 \text{ legs/elephant} = 32 \text{ legs} \) for the elephants with all their legs. - \( 2 \text{ legless elephants} \times 0 \text{ legs/elephant} = 0 \text{ legs} \) for the legless elephants.

Adding these together gives the total number of legs. Let's calculate this.

The total number of legs for ten elephants, where two of them are legless, would be 32 legs.

Unless of course the maneuver takes a significant part of a year