[ale] Bash Scripts: When to break them into files

[Steve Litt]

Leam Hall via Ale said on Tue, 27 Apr 2021 05:58:11 -0400

>On 4/26/21 10:11 PM, David Jackson via Ale wrote:
>> Hey Everyone,
>> 
>> When do you guys feel that your bash scripts have gotten too long?
>>  
>When 
>> do they need to be broken out into individual files, and when does
>> doing so make them less easy to maintain or follow?
>> 
>> Also, how do you organize your scripts so they are easy for
>> newcomers to understand?
>> 
>> Your thoughts are appreciated!  
>
>
>Because of the growing number of shell options, and because I like 
>Python/Perl/Ruby, I will usually only write shell/bash when it is a 
>small system function (backups, etc) and I don't want the overhead of 
>something bigger. Also, for things like installing packages and 
>compiling a new language when my preferred language might not be on
>the machine.

I agree wholeheartedly. Because of their quirky syntax, shellscripts
become unwieldy at about 200 lines of code. I'll often do the prototype
in /bin/sh, and then if it starts getting unwieldy, I'll take the
knowledge gained from the shellscript and write one or more Python
programs.

As far as overhead, the minute a process runs a loop over a few hundred
iterations, shellscripts lose any performance advantage they might have
had. Even Python, which is a slow load and pretty slow run, saves tons
of time over a shellscript when such loops are involved.


>
>I prefer not to break shell scripts into multiple files; if that is 
>needed to organize it, then that's time to use something else. 

Breaking into two different files could mean different things. The
barfy (but not as barfy as systemd) sysvinit system has all its huge
startup and shutdown shellscripts access functions from a *huge*
function shellscript. Tracing is a mess. I don't recommend this type of
multifile programming.

The type of multifile shellscripting I *do* recommend is something like
the following:

cat access.log | grep "\.htm" | remove_fields | resort | count | output

Commands like the preceding are easy to troubleshoot using intermediate
files, easy to understand, generally short, and custom made for
multiple processors (assuming they process one/output one).

> I saw
>one place that had a 50,000 line shell script/awk because they didn't
>want to use anything else.

That's gross.

SteveT

Steve Litt 
Spring 2021 featured book: Troubleshooting Techniques of the Successful
Technologist http://www.troubleshooters.com/techniques