[ale] Making the argument for many scripts vs one big one.

Doug Hall doughalldev at gmail.com
Wed Jul 24 17:54:34 EDT 2013 

I haven't done this with bash scripts, but the general idea in the "real"
programming world, is to create libraries (files) of related code, and
loading that library in the scripts that use them. So, you might have a
library for encryption/decryption routines, another library of routines for
parsing xml, and yet another for establishing connections to other devices
using a particular protocol. So, you won't usually have files with
individual routines. They might have dozens of routines in each library.
Usually, when you have more than this, there's some naming mechanism that
you can use to further subdivide the routines. If you need only one of
them, it's still more efficient to load the library (since you only have to
do it once, at the beginning of the script), than to create hundreds of
script files with one routine each.

My favorite scripting language is ruby. With it, you can create
Modules::Classes::Subclasses (etc) in order to properly namespace routines
with a particular functionality. Proper ruby coding would have the
programmer create a folder with the name of the module. This module folder
would then hold files with the name of the classes that are part of that
module. This keeps everything well-organized. I'm not sure how bash
programmers do this, but I'd love to hear the conventional wisdom from
those of you who have done this for a while.

Doug


On Wed, Jul 24, 2013 at 4:16 PM, Wolf Halton <wolf.halton at gmail.com> wrote:

> I have tended to make little scripts that do one thing or maybe 3 or 4 but
> then call them with another head script where the user input is set up. or
> just let people call the little scripts one by one, such as in the case
> where a little script might take 30 hours to complete, and network
> conditions can break it.  I like coming back in and running the script a
> second time, first thing in the morning and check what was left out in the
> overnight run.
>
> Of course, I am still a newbie at this.
>
> Wolf Halton
>
> --
> This Apt Has Super Cow Powers - http://sourcefreedom.com
> Security in the Cloud - http://AtlantaCloudTech.com<http://atlantaCloudTech.com>
> Apache Developer wolfhalton at apache.org
>
>
> On Wed, Jul 24, 2013 at 3:09 PM, Jim Kinney <jim.kinney at gmail.com> wrote:
>
>> feet first or head first, at some point they just have to jump in. Little
>> files are easier for nubes to follow.
>>
>>
>> On Wed, Jul 24, 2013 at 2:57 PM, leam hall <leamhall at gmail.com> wrote:
>>
>>> Yeah, I used to have the %POST do a wget and grab the latest scripts and
>>> then run them. The issue I'm trying to also deal with is to be able to run
>>> a select subset of the scripts ad hoc.
>>>
>>> One big driver is to make it easier for others to start adding code. One
>>> script to rule them all makes it a bit more intimidating.
>>>
>>> Leam
>>>
>>>
>>> On Wed, Jul 24, 2013 at 2:48 PM, Jim Kinney <jim.kinney at gmail.com>wrote:
>>>
>>>> Ah. Now I understand what you're doing. In a prior incarnation, we did
>>>> the DoD security dance with a single script that ran just post install
>>>> (kickstart called it in %POST) to do the lockdown. The script had comments
>>>> for each lockdown and a test was run for each to determine if it was
>>>> needed. Post lock tests were done and all pre and post tests were logged to
>>>> prove it was done. As this was done before a human touched the machine, the
>>>> powers that be were happy.
>>>>
>>>> So, zillion scriptlets can be merged into monolithic script and PHBs
>>>> are happy.
>>>>
>>>>
>>>> On Wed, Jul 24, 2013 at 2:11 PM, leam hall <leamhall at gmail.com> wrote:
>>>>
>>>>> In this case it's less than a couple hundred Bash scripts. Each script
>>>>> deals with a specific task in DoD directives. Thus I can pull the ones our
>>>>> site needs and document why the others won't work.
>>>>>
>>>>> Or I can bang my head into my desk why duplicating what's already been
>>>>> done...
>>>>>
>>>>> Leam
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jul 24, 2013 at 2:01 PM, Michael B. Trausch <
>>>>> mbt at naunetcorp.com> wrote:
>>>>>
>>>>>>  On 07/24/2013 01:50 PM, Jim Kinney wrote:
>>>>>>
>>>>>> That would depend, to me at least, on whether the final deployment is
>>>>>> an internal or external tool. Internal gets the single blob. External gets
>>>>>> a zillion files. The logic is to make it confusing to the external (l)user
>>>>>> so they won't tinker with things. Bonus points if the zillion files all
>>>>>> look like obfuscated perl :-)
>>>>>>
>>>>>>
>>>>>> Hrm... we need a minification tool for Bash.
>>>>>>
>>>>>> Perl, obviously, doesn't need one.  Just remove the whitespace, it's
>>>>>> ugly and cryptic all by itself.  :-D
>>>>>>
>>>>>>
>>>>>> I saw a system once that had a shell application that called a
>>>>>> zillion files. The customer wanted the development team to go away but was
>>>>>> worried about what all the application did. So I went over it with a fine
>>>>>> tooth comb. Basically, the application consisted of 4 or 5 main script
>>>>>> files that each would call 20-30 of the crap files to do things like count
>>>>>> lines and characters but then dump the results without ever using them. So
>>>>>> there was 150-200 scripts that were culled from the process after some
>>>>>> careful refactoring in the 5 main ones. Then I ran into funny issues that
>>>>>> were sort of like race conditions but not quite. The zillion crap scripts
>>>>>> were created to slow down the main scripts so it was closer to the original
>>>>>> system that ran at 300MHz instead of the now 1.5 GHz. It was using many
>>>>>> serial ports to get data from lab systems. I replaced the lot with a few
>>>>>> sleep calls to allow the serial port data to accumulate and the developers
>>>>>> were dumped.
>>>>>>
>>>>>>
>>>>>> Nice.  :-)
>>>>>>
>>>>>> A tool like Closure would actually be really neat for things like
>>>>>> bash, python, perl, etc.
>>>>>>
>>>>>> Well, not Python, since Python actually relies on whitespace for
>>>>>> semantics.
>>>>>>
>>>>>> Of course, if you have a gazillion things that can cause deadlocks or
>>>>>> have race conditions, you absolutely want things in a single program, even
>>>>>> if the work is itself spread over different modules.  Sounds like the
>>>>>> perfect thing for C.
>>>>>>
>>>>>>
>>>>>>     — Mike
>>>>>>
>>>>>>
>>>>>> --
>>>>>>   [image: Naunet Corporation Logo]  Michael B. Trausch
>>>>>>
>>>>>> President, *Naunet Corporation*
>>>>>> ☎ (678) 287-0693 x130 or (888) 494-5810 x130
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ale mailing list
>>>>>> Ale at ale.org
>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Mind on a Mission <http://leamhall.blogspot.com/>
>>>>>
>>>>> _______________________________________________
>>>>> Ale mailing list
>>>>> Ale at ale.org
>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>> http://mail.ale.org/mailman/listinfo
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> James P. Kinney III
>>>> *
>>>> *Every time you stop a school, you will have to build a jail. What you
>>>> gain at one end you lose at the other. It's like feeding a dog on his own
>>>> tail. It won't fatten the dog.
>>>> - Speech 11/23/1900 Mark Twain
>>>> *
>>>> http://electjimkinney.org
>>>> http://heretothereideas.blogspot.com/
>>>> *
>>>>
>>>> _______________________________________________
>>>> Ale mailing list
>>>> Ale at ale.org
>>>> http://mail.ale.org/mailman/listinfo/ale
>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>> http://mail.ale.org/mailman/listinfo
>>>>
>>>>
>>>
>>>
>>> --
>>> Mind on a Mission <http://leamhall.blogspot.com/>
>>>
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> http://mail.ale.org/mailman/listinfo/ale
>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>> http://mail.ale.org/mailman/listinfo
>>>
>>>
>>
>>
>> --
>> --
>> James P. Kinney III
>> *
>> *Every time you stop a school, you will have to build a jail. What you
>> gain at one end you lose at the other. It's like feeding a dog on his own
>> tail. It won't fatten the dog.
>> - Speech 11/23/1900 Mark Twain
>> *
>> http://electjimkinney.org
>> http://heretothereideas.blogspot.com/
>> *
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
>>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130724/eee8f64e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dafjajhe.png
Type: image/png
Size: 1701 bytes
Desc: not available
URL: <http://mail.ale.org/pipermail/ale/attachments/20130724/eee8f64e/attachment-0001.png>

More information about the Ale mailing list