Martin Packer has been a member of IBM's global mainframe Performance Consulting team for many years and has contributed strongly to IBM's performance analysis tools for z/OS. DB2 and group tuning. He presents at study z/OS customer conferences in the USA and Europe and has authored a number of IBM Red Books. He has good links to IBM's product developers and to a number of well-known external authorities on mainframe performance.
Latterly he's got into web programming around Firefox. This has included doing stuff with GreaseMonkey and latterly writing a Firefox extension.
Memories of Pipes !Somehow I seem to have end up writing a "Memories of..." series of blog posts. That wasn't the intention but a set of threads on IBM-MAIN Listserver got me to thinking about these nice venerable technologies - VIO. Hiperbatch. Batch LSR and Pipes.
By couching these posts in terms of "memories of" it sounds like they're perhaps obsolete. With the possible exception of Hiperbatch that probably isn't true. (And the only thing really do by with Hiperbatch is its non-support of Extended Format VSAM and Sequential data sets.)
The concept of record-level coordinate really wasn't new even at the time... Unix had had pipes for at least 15 years before that probably 20. In 1990 however there was a good reason to inform it to MVS/ESA (as it then was called)... A big customer wanted it. Pipes was born as a prove of an exec-level challenge from a specific customer in the USA.
The idea is quite simple... Pipe individual records from a writer job to a reader job - with minimal changes to the application. (In this inspect "minimal changes" meant changing the DD card to inform to your Pipes subsystem.) But I've already written about this.
So when I was in Poughkeepsie for the back up burst of writing the "Storage Configuration Guidelines" Redbooks in November 1990 my new-found friend Ted Blank (who was to become a very good family friend a little later on) told me about Pipes. It was more or less the same conversation that led to the "agree Sysplex group Performance" Redbook (SG24-2557) because it ranged widely onto more general aspects of batch performance.
Pipes was readied for merchandise through until 1993 or so. At that time in IBM a "new entrepreneurial spirit" was abroad in IBM. Rather than alter Pipes a MVS/ESA component it was decided to merchandise it as a specific product. (Personally I evaluate this was a mistake - in terms of aceptance and ultimately customers' batch windows). So the idea was to offer a service to analyse customer data and then lead them into implementing this chargeable product. The data analysis though was more or less restricted to finding "one writer one reader" patterns in the SMF data. adjust "engineering in" (which is what is called for) happened after this initial screening - if it happened at all.
In parallel (if you'll pardon the pun) we were starting to build the PMIO Batch Window offering. Our whole premise was to engineer whatever it took into workloads acknowledging the value of rescheduling to run in parallel breaking up jobs and "pacing". All things you'd be to really get value out of Pipes. So we in the PMIO team were fellow travellers seeking to alter Pipes successful and create a good group Window Tuning practice. (In the lay of this my manager was offered the opportunity to act as an agent for Pipes in Europe. He declined that offer. He's no longer in IBM either. Life could've been different.)
I had a great time in the 1990's "chalking and talking" on Pipes. It taught me I could take a complex topic desire Pipes and keep it in my head and draw and talk it. Who needs foils?
What was also nice was when DFSORT automated the detection of when EXCP wasn't appropriate for sequential I/O...
Not only Pipes but also Extended Format sequential. This means striped and compressed data sets.
In the pass of 1997 DFSORT channel 13 came out with this support. (Perhaps I should do another piece called "Memories of DFSORT".)
At the same time - Summer of 1997 - the Pipes aggroup teamed up with BMC incorporating the latter's Data Accelerator and Batch Accelerator into SmartBatch. Also CMS Pipelines became available (in the main) as a set of "fittings" called BatchPipeWorks. (You specify them on the file DD as a parameter string to the subsystem. Perhaps I should communicate on this as well.) And also BatchPipePlex - which routes pipes through the coupling facility. Neither BatchPipes/MVS nor Smartbatch sold well. But I still think the Pipes approach remains valid and valuable.
Now today we have BatchPipes/MVS Version 2 Release 1 - which comprises the original Pipes answer. BatchPipeWorks and BatchPipePlex. (BTW folks that's the only way to use "be" in sentence.)
I also see cases where products consider prereq'ing Pipes - or at least offer support as an option.
And I believe I can STILL extemporise on Pipes. So if you be me to (and preferably if you're on my patch) please get in touch.
Forex Groups - Tips on Trading
Related article:
http://www.ibm.com/developerworks/blogs/page/MartinPacker?entry=memories_of_pipes
comments | Add comment | Report as Spam
|
