shok / motivation

There is a command shell buried deep within nearly every major computing platform. It provides a backbone interface to the operating system, the filesystem, and for running programs. The shell is appreciated for its powerful and scriptable view of the system and for providing important system primitives that are not so easily accessable elsewhere (e.g. process return codes, signals, pipes and redirection, remote shell to other systems). A text-mode interface also has many natural advantages including its expressivity, UI simplicity, and low system and network requirements.

Unfortunately, most shells have severe problems. To name a few: confusing and error-prone syntax, latent security vulnerabilities, and incomplete and unhelpful help systems. The behaviour of a shell or script is easily inconsistent across versions and platforms, and the right way to do something is usually not the easiest or most obvious.

There have been many attempts to obviate or replace the shell; for example, scripting languages and libraries/toolkits that provide shell-like primitives. These are nice but I don't think they can truly displace the command shell — where the natural action is to run another program, and general-purpose programming is a secondary and complementary task.

There is a need for the shell. And it needs to be upgraded to a modern, elegant, powerful interface between the user and the computer at their command.

Another command shell?

There really aren't that many. I consider the major categories as follows:

All are interesting and relevant, but shok is primarily exploring a different set of ideas. Its most defining feature is the delicate way it integrates a programming language into the shell environment.

Another programming language?

With shok's syntax for switching between the command-line and its programming language, one could imagine embedding any existing language back there behind the {}'s. However, shok asserts that the domain-specific requirements of an OS shell motivate a new language designed for that domain. Specific considerations include:

  • Smooth mode-switching between the command-line and the scripting language
  • Domain-specific language features like path literals and other shell-related primitives
  • Deterministic memory profile appropriate for long-running sessions on memory-constrained systems

Additionally, a paranoid obsession with security is appropriate for a command-shell. The shell is one of the most notoriously dangerous aspects of system administration, for several reasons including:

  • Filesystem interaction: The shell is used for directly modifying and deleting files. We assume the shell is capable of easily and irrevocably destroying much of the user's data, restrained only if at all by a very coarse permissioning system provided by the OS.
  • Invoking malicious programs, or regular programs with unintended arguments, can wreak havoc on a system.
  • Basic shell primitives and system tools are notoriously tedious to use and error-prone.
  • Operating systems are very complex, and the shell is typically a thin interface on top of the OS's convoluted semantics.
  • The command-line is often used to access different computers, so the user has to be aware of how the shell might behave differently depending on where they are logged in.

For all these reasons, the language must be predictable and safe. shok considers it a responsibility to perform as much static analysis as possible on a block of code before executing any part of it. To this end, it uses a strong static type system and a controlled, reliable memory ownership mechanism, among other efforts to bring security to shell programming.

Last updated: 2013-12-25