Explanation for newbies:
-
Shell is the programming language that you use when you open a terminal on linux or mac os. Well, actually “shell” is a family of languages with many different implementations (bash, dash, ash, zsh, ksh, fish, …)
-
Writing programs in shell (called “shell scripts”) is a harrowing experience because the language is optimized for interactive use at a terminal, not writing extensive applications
-
The two lines in the meme change the shell’s behavior to be slightly less headache-inducing for the programmer:
set -euo pipefailis the short form of the following three commands:set -e: exit on the first command that fails, rather than plowing through ignoring all errorsset -u: treat references to undefined variables as errorsset -o pipefail: If a command piped into another command fails, treat that as an error
export LC_ALL=Ctells other programs to not do weird things depending on locale. For example, it forcesseqto output numbers with a period as the decimal separator, even on systems where coma is the default decimal separator (russian, dutch, etc.).
-
The title text references “posix”, which is a document that standardizes, among other things, what features a shell must have. Posix does not require a shell to implement
pipefail, so if you want your script to run on as many different platforms as possible, then you cannot use that feature.
Lol, I love that someone made this. What if your input has newlines tho, gotta use that NUL terminator!
God, I wish more tools had nice NUL-separated output. Looking at you,
jq. I dunno why this issue has been open for so long, but it hurts me. Like, they’ve gone back and forth on this so many times…~$ man !!
Does this joke have a documentation page?
unironically, yes
Nushell has pipefail by default (plus an actual error system that integrates with status codes) and has actual number values, don’t have these problems
nushell is pretty good. I use it for my main shell
although, i still prefer writing utilities in python over nu scripts
Nushell is great as a more powerful scripting language, but a proper language like python is useful too
I’m fine with my shell scripts not running on a PDP11.
I’m glad powershell is cross-platform nowadays. It’s a bit saner.
Better would be to leave the 1970s and never interact with a terminal again…
I’m a former (long long ago) Linux admin and a current heavy (but not really deep) powershell user.
The .net-ification of *nix just seems bonkers to me.
Does it really work that well?
The .net-ification of *nix just seems bonkers to me.
It IS bonkers. As a case study, compare the process of setting up a self-hosted runner in gitlab vs github.
Gitlab does everything The Linux Way. You spin up a slim docker container based on Alpine, pass it the API key, and you’re done. Nice and simple.
Github, being owned by Microsoft, does everything The Microsoft way. The documentation mentions nothing of containers, you’re just meant to run the runner natively, essentially giving Microsoft and anyone else in the repo a reverse shell into your machine. Lovely. Microsoft then somehow managed to make their runner software (reminder: whose entire job consists of pulling a git repo, parsing a yaml file, and running some shell commands) depend on fucking dotnet and a bunch of other bullshit. You’re meant to install it using a shitty
setup.shscript that does that stupid thing with detecting what distro you’re on and calling the native package manager to install dependencies. Which of course doesn’t work for shit for anyone who’s not on debain or fedora because those are the only distro’s they’ve bothered with. So you’re either stuck setting up dotnet on your system, or trying to manually banish this unholy abomination into a docker container, which is gonna end up twice the size of gitlab’s pre-made Alpine container thanks to needing glibc to run dotnet (and also full gnu coreutils for some fucking reason)Bloat is just microsoft’s way of doing things. They see unused resources, and they want to use them. Keep microsoft out of linux.
I usually only use the terminal when I don’t find an alternative, but powershell usually feels a bit saner than bash.
I’ve also tried nushell, also nice, but in a few situations powershell despite usually being verbose was more “elegant”.
What type of .net-ification occurs on *nix? I am current linux “admin” and there is close to 0 times where I’ve seen powershell not on windows. Maybe in some microsoft specific hell-scape it is more common, but it’s hard to imagine that there are people that can accept a “shell” that takes 5-10 seconds to start. There are apps written in c# but they aren’t all that common?
I honestly don’t know but there must some translation happening between .net objects and *nix.
And .net “core” started supporting Linux like a decade ago. I’m guessing they’re related ¯\_(ツ)_/¯
Quick, how do I do
for i in $(find . -iname '*.pdf' -mtime -30); do convert -density 300 ${i} ${i}.jpeg; donein a GUI, again?$time = (get-date).adddays(-30) gci -file -filter *.pdf ` | ? { $_.lastwritetime -gt $time } ` | % { convert -density 300 $_.fullname $($_.fullname + ".jpg") }🤷
I fail to see the GUI in that.
Like, if I open Dolphin, Thunar, or
winfile.I missed where the person you replied to said to never use the terminal again, now your comment makes perfect sense. I thought you were conflating the preference for powershell with windows and therefor more GUI.
My bad, and I completely agree with you, if I had to give up CLI or GUI it would be GUI hands down no competition at all, I would die without CLI.
Better would be to leave the 1970s and never interact with a terminal again…
I’m still waiting for someone to come up with a better alternative. And once someone does come up with something better, it will be another few decades of waiting for it to catch on. Terminal emulation is dumb and weird, but there’s just no better solution that’s also compatible with existing software. Just look at any IDE as an example: visual studio, code blocks, whatever. Thousands of hours put into making all those fancy buttons menus and GUIs, and still the only feature that is worth using is the built-in terminal emulator which you can use to run a real text editor like vim or emacs.
What else are you going to do, though?
If you have some particular and complicated task then sure you’d probably write a program for it in a specific high-level language. But that isn’t what the shell is for.
If you’ve already got a bunch of apps and utilities and want to orchestrate them together to do a task, that’s a good shell use case.
Or if you have a system that needs setup and install tasks doing on it to prepare for running your actual workload, that’s also a task which the shell is ideally suited to.
Shell scripting always has a place, and I can’t see it being made obsolete any time soon.
People keep on telling me that python is a “scripting language” but honestly I would rather use the shoddiest and most barebones shell you would give me than python if I had to make a script. It all boils to interfaces, and there are more programs, libraries, and daemons that have a shell interface as opposed to whatever other “better” language there is out there. Trying to write scripts with Python or ruby or whatever will just boils down to plumbing data between external programs in a less succinct syntax. What good is type safety, try/catch, and classes if all the tools that you’re using are taking in and spitting out raw text anyway?
What about embedded work? You gonna run a full graphical GUI on a network router?
This joke comes with more documentation than most packages.
I have written 5 shell scripts ever, and only 1 of them has been more complex than “I want to alias this single command”
I can’t imagine being an actual shell dev
only 1 of them has been more complex than “I want to alias this single command”
I have some literal shell aliases that took me hours to debug…
I do not envy you.
Nah, it’s in the past.
But people, if you are writing a command that detects a terminal to decide to color its output or not, please add some overriding parameters to it.
It really isn’t bad especially if you use ash
A lot of people call
set -euo pipefailthe “strict mode” for bash programming, as a reference to;in JavaScript.In other words, always add this if you want to stay sane unless you’re a shellcheck user.
People call
set -euo pipefailstrict mode but, it’s just another footgun in a language full of footguns. Shellcheck is a fucking blessing from heaven though. I wish I could forcibly install it on every developer’s system.
Powershell is the future
- Windows and office365 admins
I even use powershell as my main scripting language on my Mac now. I’ve come around.
That’s sounds terrible honestly
I’m so used to using powershell to handle collections and pipelines that I find I want it for small scripts on Mac. For instance, I was using ffmpeg to alter a collection of files on my Mac recently. I found it super simple to use Powershell to handle the logic. I could have used other tools, but I didn’t find anything about it terrible.
#!/usr/bin/python
No, sorry. I’m a python dev and I love python, but there’s no way I’m using it for scripting. Trying to use python as a shell language just has you passing data across
Popencalls with a sea of.decodeand.encode. You’re doing the same stuff you would be doing in shell, but with a less concise syntax. Literally all of python’s benefits (classes, types, lists) are negated because all of the tools you’re using when writing scripts are processing raw text anyway. Not to mention the version incompatibility thing. You use an f-string in a spicy way once, and suddenly your “script” is incompatible with half of all python installations out there, which is made worse by the fact that almost every distro has a very narrow selection of python versions available on their package manager. With shell you have the least common denominator of posix sh. With Python, some distros rush ahead to the latest release, while other hang on to ancient versions. Evenprint("hello world")isn’t guaranteed to work, since some LTS ubuntu versions still havepythonpointing to python2.The quickest cure for thinking that Python “solves” the problems of shell is to first learn good practices of shell, and then trying to port an existing shell script to python. That’ll change your opinion quickly enough.
Lua
#!/usr/bin/env python
Gotta love a meme that comes with a
manpage!People say that if you have to explain the joke then it’s not funny. Not here, here the explanation is part of the joke.
It is different in spoken form, written form (chat) and written as a post (like here).
In person, you get a reaction almost immediately. Written as a short chat, you also get a reaction. But like this is more of an accessibility thing rather than the joke not being funny. You know, like those text descriptions of an image (usually for memes).
This is much better than a man page. Like, have you seen those things?
I really recommend that if you haven’t, that you look at the Bash’s man page.
It’s just amazing.
I think you mean a
manzing.I’ll see myself out.
Come to think of it, I probably have never done so. Now I’m scared.
I’m divided between saying it’s really great or that it should be a book and the man page should be something else.
Good thing man has search, bad thing a lot of people don’t know about that.
Shell is great, but if you’re using it as a programming language then you’re going to have a bad time. It’s great for scripting, but if you find yourself actually programming in it then save yourself the headache and use an actual language!
Your scientists were so preoccupied with whether they could, they didn’t stop to think if they should
Honestly, the fact that bash exposes low level networking primitives like a TCP socket via /dev/TCP is such a godsend. I’ve written an HTTP client in Bash before when I needed to get some data off of a box that had a fucked up filesystem and only had an emergency shell. I would have been totally fucked without /dev/tcp, so I’m glad things like it exist.
EDIT: oh, the article author is just using netcat, not doing it all in pure bash. That’s a more practical choice, although it’s way less fun and cursed.
EDIT: here’s a webserver written entirely in bash. No netcat, just the /bin/bash binary https://github.com/dzove855/Bash-web-server
The few times I’ve used shell for programming it was in strict work environments where anything compiled was not allowed without a ton of red tape.
Wouldn’t something interpreted like python be a better solution?
For more complicated input/output file handling, certainly.
Little shell scripts do great though if all you need to do is concatenate files by piping them.
It’s like the Internet, it’s not one big truck but a series of tubes.
Yep, in my mind piping together other commands is scripting not programming, exactly what shell scripts are for!
With enough regex and sed/awk you might be able to make even complicated stuff work. I’m not a regex guru (but I do occasionally dabble in the dark arts).
I feel attacked. https://github.com/fmstrat/gam
Alpine linux, one of the most popular distros to use inside docker containers (and arguably good for desktop, servers, and embedded) is held together by shell scripts, and it’s doing just fine. The installer, helper commands, and init scripts are all written for busybox sh. But I guess that falls under “scripting” by your definition.
Aka busybox in disguise 🥸
So what IS the difference between scripting and programming in your view?
No clear line, but to me a script is tying together other programs that you run, those programs themselves are the programs. I guess it’s a matter of how complex the logic is too.
I didn’t know the locale part…
I learned about it the hard way lol.
seqused to generate a csv file in a script. My polish friend runs said script, and suddenly there’s an extra column in the csv…I had os-prober fail if locale was Turkish, the seperator thing could explain that.
I was never a fan of
set -e. I prefer to do my own error handling. But, I never understood why pipefail wasn’t the default. A failure is a failure. I would like to know about it!IIRC if you pipe something do head it will stop reading after some lines and close the pipe, leading to a pipe fail even if everything works correctly
Yeah, I had a silly hack for that. I don’t remember what it was. It’s been 3-4 years since I wrote bash for a living. While not perfect, I still need to know if a pipeline command failed. Continuing a script after an invisible error, in many cases, could have been catastrophic.
My only issue is
-u. How do you print help text if your required parameters are always filled. There’s no way to test for -z if the shell bails on the first line.Edit: though I guess you could initialise your vars with bad defaults, and test for those.
#!/bin/bash set -euo pipefail if [[ -z "${1:-}" ]] then echo "we need an argument!" >&2 exit 1 fiGod I love bash. There’s always something to learn.
my logical steps
- #! yup
- if sure!
- [[ -z makes sense
- ${1:-} WHAT IN SATANS UNDERPANTS… parameter expansion I think… reads docs … default value! shit that’s nice.
it’s like buying a really simple generic car then getting excited because it actually has a spare and cupholders.
That’s good, but if you like to name your arguments first before testing them, then it falls apart
#!/bin/bash set -euo pipefail myarg=$1 if [[ -z "${myarg}" ]] then echo "we need an argument!" >&2 exit 1 fiThis fails. The solution is to do
myarg=${1:-}and then testEdit: Oh, I just saw you did that initialisation in the if statement. Take your trophy and leave.
Yeah, another way to do it is
#!/bin/bash set -euo pipefail if [[ $# -lt 1 ]] then echo "Usage: $0 argument1" >&2 exit 1 fii.e. just count arguments. Related,
fishhas kind of the orthogonal situation here, where you can name arguments in a better way, but there’s noset -ufunction foo --argument-names bar ... endin the end my conclusion is that argument handling in shells is generally bad. Add in historic workarounds like
if [ "x" = "x$1" ]and it’s clear shells have always been Shortcut City
Side note: One point I have to award to Perl for using
eq/lt/gt/etcfor string comparisons and==/</>for numeric comparisons. In shells it’s reversed for some reason? The absolute state of things when I can point to Perl as an example of something that did it betterPerl is the original GOAT! It took a look at shell, realised it could do (slightly) better, and forged its own hacky path!
I was about to say, half the things people write complex shell scripts for, I’ll just do in something like Perl, Ruby, Python, even node/TS, because they have actual type systems and readability. And library support. Always situation-dependent though.
