One thing that strikes me as a person who grew up with mouse pointer interfaces (starting with Workbench on Amiga) is how strange it is that the terminal allows for a time-lag between a mistake in typing and the user's becoming aware of it by pressing Return. Intense tab spamming removes this alienating phenomenon though, and I imagine that with a perfectly smart terminal (and commands), virtual tab-spamming is automatic so that a user is immediately informed and even prevented from typing something that could not possibly render a valid command.
I suppose this is how voice command will evolve too. Cortona will stop you short when you're not making sense, and even learn your idiosyncrasies in expressing yourself much like a spouse does.
By the way, I for sure thought this link was about a the complete J.S. Bach collection.
That's an interesting invariant that I've been thinking about: hitting TAB should never generate something that would be an invalid command (or more pedantically, generate a command prefix that would have a syntax error once the user fills in the rest of the command).
You would THINK that shell tab completion already works that way. But it doesn't in bash, and not even in fish or zsh.
Not only does the completion logic duplicates the tool's own parsing logic, it doesn't attempt to do it faithfully.
(1) First, there's version skew in all the repos for bash, fish, and zsh.
(2) Leaving that aside, the parsing models they use aren't rich enough to express some command lines.
"find" is a good example. They will let you do something like
find -type f -dept<TAB> -- completes -depth, but it's not valid here
This is because find is split up into options and the expression language. There are probably some simpler and more compelling examples I can come up with.
Another one is completing flags in this situation:
grep -- --c<TAB>
After --, there are only args, not flags, so it's misleading to complete.
----
Being able to spam TAB is what I'm going for with Oil. That's not true of bash. One annoyance the (y/n) prompt that bash gives you when there are thousands of completions (e.g. hitting TAB on an empty line).
Glad to hear someone is thinking about this. Right, hitting tab should never generate something invalid. That's the shell being sound. Conversely, the system should also be able to determine whether what's being typed could possibly become a valid command or not. That's the shell being complete.
Yes, I think it basically reduces to the problem of using the tool's ACTUAL parser. There are enough variations in flag syntax that it's not possible to be 100% correct without running arbitrary code. And if you're going to do that, you might as well run the code that's already written (with some caveats).
That could be solved by my proposal for a "Shellac protocol", akin to Microsoft's Language Server Protocol (e.g. for VSCode):
1. The protocol was mainly motivated by the problem of bootstrapping a corpus of shell completions. Every shell has to "boil the ocean" itself, i.e. provide completions for many popular commands. git's completion is thousands of lines in each of bash, zsh, and fish. Upstream only maintains bash.
2. What I noticed in implementing completion for Oil is that bash fundamentally CONFLATES two problems:
a) The problem of completing the shell language itself, e.g. ${H -> ${HOSTNAME, or ~a -> ~andy
b) The problem of completing command line flag syntax.
This is a huge mistake from the POV of implementation complexity. And obviously if you want shell-agnostic auto-completions, (b) needs to be separate from (a), which is shell-specific.
Luckily the authors of bash-completion tried to separate these two problems a bit, and I was able to salvage most code.
Since I wrote that post, and implemented a compatible completion API in OSH, it's become a lot clearer to me how the protocol should work.
3) The invariant you mention would be a third benefit of having a more principled completion API that can run arbitrary code (in arbitrary languages).
Now, I'm not saying that EVERY command needs to use this. There are some engineering problems with this approach too. But I think you should have the POSSIBILITY of doing something correct if you want. Heuristics may be OK in the simple cases.
But overall what I discovered is that the system is a huge mess. The bash-completion project tries to parse bash in bash, and unsurprisingly does a poor job. My upcoming blog posts will explain this.
Bash/ZSH can have arbitary completion functions/scripts that receive the full input.
A lot of the completions are auto-generated from man files, which don't contain information like "argument x is only available if argument y is present", etc. (at least in a structured way)
It's "just" a matter of implementing completions manually to enable what you are asking for.
General Developer Experience (DX) speaking, IMO, File and Folder are kind of better at CRUD-ing json. One file per attribute, and foldable attributes are folders (object and array types). It scales quite well rather than loading a big file into memory.
Tools around file system are battle tested. Basic operations; read=ls, create=touch/mkdir, update=mv, delete=rm. Linking data = symbolic link. File explorer/manager helps managing attributes much nicer than in-file operations. It can even use extension as types, structural folders as schema.
jq is nice but you've already got: find, grep, awk, sed programs.
The only downside I can think of is size might be larger depending on what's minimum block size (e.g. 4096 bytes) hmm
I was going to suggest that somebody try this in Oil shell, which can now run thousands of lines of bash completion scripts [1], but I can already see it won't work because of the VERSION DETECTION anti-pattern.
It feels like I need to write a blog post to explain this, since this knowledge seems to be waning. I recall reading about it 10+ years ago, but not recently.
Richard Stallman understood it very well and IIRC strongly advocated for feature detection in autoconf. (Despite all the bad things about autoconf, it does actually work on tons of platforms, and it's solving a hard problem. It wouldn't "scale" if it were using version detection.)
Here's an outline of a blog post:
1. Version/browser detection is why the User-Agent string from Internet Explorer, Chrome, Safari, Edge, etc. all start with "Mozilla". Fun fact: if everyone used feature detection, we could have saved exabytes of data transfer over the last 20 years by trimming 10-50 bytes from every single HTTP request ever made.
2. jQuery switched from version detection to feature detection early in its life. I recall some threads where Resig was criticized for this, and then he saw the light.
5. My hand-written configure script also uses feature detection, e.g. to detect the presence of the readline library. It basically tries to compile a small program with the C compiler. There are various tricks in this domain.
6. eval is useful for feature detection in dynamic programming languages. In fact, this might be the ONLY thing eval is good for! (exaggerating a bit)
