The COMMON statement in basic can be confusing as it has several variations. Per the jBASE 4.1 documentation;
COMMON
The COMMON statement declares a list of variables and matrices that can be shared among various
programs. There can be many common areas including a default, unnamed common area.
COMMAND SYNTAX
COMMON {/CommonName/} variable{, variable ... }
SYNTAX ELEMENTS
The list of variables should not have been declared or referenced previously in the program file. The compiler will detect any bad declarations and display suitable warning or error messages. If the common area declared with the statement is to be named then the first entry in the list should be a string, delimited by the / character.
NOTES
The compiler will not, by default, check that variables declared in COMMON statements are initialized before they have been used as this may be beyond the scope of this single source code check. The -JCi option, when specified to the jBASE BASIC compiler, will force this check to be applied to common variables as well. The initialization of named common is controlled in the Config_EMULATE file.
Variables declared without naming the common area may only be shared between the program and its subroutines (unless CHAIN is used). Variables declared in a named common area may be shared across program boundaries. When any common area is shared, all programs using it should have declared the
same number of variables within it. Dimensioned arrays are declared and dimensioned within the COMMON statement.
EXAMPLES
COMMON A, B(2, 6, 10), c
COMMON/Common1/ A, D, Array(10, 10)
To restate this in my words.
The COMMON or COM statement is used to define shared variables, either named or unnamed.
Named common variables are created by including /CommonName/ in the statement as shown above. Once created, named variables exist until you logout of the session. Any basic program, subroutine or function can access and modify the contents of the variables by including a common statement with /CommonName/ and
one or more of the variables. You do not need to include all of them and the order does no matter.
For example, a setup or menu type of program could contain:
COMMON /DONS/ A, B, C, D
and a program could contain:
COMMON /DONS/ C
which would allow you to use or modify the data in C.
Thus named common variables are flexible and persistent.
Unnamed common variables are created by not including the /CommonName/ in the common statement. Unnamed common variables exist only as long as the the program that creates them is running, that is, until a STOP or CHAIN statement is executed. The variables are identified by position instead of name so they must be defined in the same order in all programs.
For example, a main program could contain:
COMMON A, B, C, D
and a subroutine would need the same statement if it needs any of the variables.
Thus unnamed common variables are not as flexible but when the main program terminates, they disappears which frees memory.
Note that multiple commons can be used. You can name several groups of variables and use each group separately. Unnamed variables must be defined in the same order in all programs that use them. They can have different names in each program but please don't do this as it is a bad practice.
More examples:
COMMON /GROUP1/ A, B, C, D(40)
COMMON /GROUP2/ W, X, Y, Z
COMMON VAR1
COMMON VAR2
COMMON VAR3
COMMON VAR4, VAR5(20), VAR6
A good practice is to put the common statements in a separate file and INCLUDE them in the programs that need them. This way you only need to maintain them in one place. Of course, when you change the include, you will need to compile all the programs that contain the include.
One more thing, there are some variations between the various MV system so the above may not be 100% true in all cases. One difference to watch out for is the value the variables contain when they are first encountered. They may be uninitialized or initialized to null or 0, check your manual.
Elapsed Time by Jon Kristofferson
When I write programs I sometimes like to find out what the elapsed wall clock time of my programs execution is. To do this I use a simple formula to print the elapsed time from the start of my program to the time I capture or output the elapsed time. The formula is as follows…
The @DATE and @TIME variables are set to the date and time of the beginning of program execution. The DATE() and TIME() functions provide the current date and time. The ((DATE()-@DATE)*86400) provides the number of days of execution in seconds. The DATE() function and @DATE variable are both given in days, and there are 86400 seconds/day, so the number of days time the seconds per day will give us the number of days in seconds. This works well especially if the program runs over midnight! Then using the TIME() function and @TIME variable, both given in seconds, I add the time difference in seconds to the elapsed time. Then I convert the resulting elapsed time using the OCONV function with the “MTH” conversion code. The OCONV function with the “MTH” code requires the value to be converted to be in seconds. The results provide some feedback as to how long the program to perform its task.
Spring cleaning: By Tom Dodds
Do you want to add some speed to your processing, gather a few extra minutes out of the days processing. Spend a little time cleaning up your VOC or MD (master control) file. Over the years these control files get pretty messy. They get overloaded with test items, pointers to files that don’t exist any more or are no longer in use, but they are still taking up space in your master control file.
Each time you execute a process you have to look it up in the master control file, every LIST statement looks up each work in the master control file. If you have a bunch of junk in that file your system has to step over that junk to find the real data it is looking for. Storing large Procs or Paragraphs in the master control file cause even more problems. Most master control files were created a long time ago and were created very small, maybe 20/30K. The have grown and grown and need to be cleaned up and then resized.
To attach this task, I would start by sorting the file by descending size. The size dictionary item will vary depending if you are on a U2/jBASE product or a more traditional Pick type system. See below for sample dictionary items. A reasonable list statement might be “LIST VOC/MD BY-DSND SIZE TYPE”. This will show the number of bytes in the item and the type of item. If it is a Proc/Paragraph move the item to a PROCLIB and replace the master control item with a pointer to the original Proc/Paragraph. I.E. “[PROCLIB {item name}”/See below for Remote.
Monthly Technical Tip Archives
U2/jBASE style size definition
0001: I
0002: LEN(@RECORD)
0003: MD0
0004: Length
0005: 9R,
0006: S
Pick style size definition
0001: A
0002: 9999
0003: SIZE
0004:
0005:
0006:
0007:
0008:
0009: R
0010: 10
Bottom at line 10.
U2/jBASE style remote pointer
0001: R
0002: UNIVERSE.VOCLIB
0003: LONGNAMES
Bottom at line 3.
Pick style remote pointer
[PROCLIB JUNK]
Shutting Down UNIX Systems
By: Bob Wyatt
When it comes to shutting down UNIX systems gracefully but without warning, it is nice to know which of the logged-in users have been inactive and for how long. All flavors of UNIX come with the ‘who’ command, most of which can tell you how long someone has been idle or inactive. The –u option typically reveals the inactivity timer, and the –H option displays an appropriate header for the display, as demonstrated below.
AIX (Activity is a misnomer – it is the idle or Inactivity time)
>who -uH
Name Line Time Activity PID Hostname
root pts/2 Jul 28 20:10 18:10 20202 (:0.0)
userx pts/3 Aug 01 05:12 0:05 3544 (100.0.13.52)
Solaris
# who -uH
NAME LINE TIME IDLE PID COMMENTS
userx pts/17 Aug 1 06:31 0:04 8844 (192.168.150.209)
usery pts/28 Aug 1 06:31 2:02 8847 (192.168.150.197)
RedHat Linux
#who -uH
NAME LINE TIME IDLE PID COMMENT
userx pts/0 Aug 1 11:12 0:02 23493 (169.248.229.217)
jBASE Or Reality Last Record Updated Tip
By Tom Fabian
Sometimes when analyzing data the question arises “When was this record last updated.” Today? this month? or even this decade?, etc.
In Jbase or Reality this can easily be done with a simple dictionary definition which we’ll call LASTUPD
File DICT CUSTOMER , Record 'LASTUPD'
Command->
0001 A
0002 0
0003
0004
0005
0006
0007 D2/
0008 A;NU
0009 R
0010 8
With this we can use JQL (a.k.a. English) with statements such as:
LIST CUSTOMER WITH LASTUPD GE "08/01/07" LASTUPD
CUSTOMER............ LASTUPD.
0303 08/22/07
0001 08/21/07
2 Records Listed
In fact if the LASTUPD definition is placed in one dictionary it can be used to list any file.
If LASTUPD is placed in the MD, for example, we still cause that definition
to be used with any file.
LIST CUSTOMER WITH LASTUPD GE "08/01/07" LASTUPD USING DICT MD
Using “guide” to help administer UniData files
By: Michael Byrne, Sr. IT Specialist – IBM
You can automate UniData’s guide command to help you avoid ever having your file’s incorrectly sized, which can cause overflow and possibly performance problems. This article discusses using guide and offers some advice on how to apply some very powerful techniques to simplify your life.
What is guide?
The system-level guide command analyzes hashed files, generates statistics, and provides suggestions for optimizing file sizes and ensuring data integrity. Guide is available on both Windows and UNIX platforms. UniData must be running when you execute guide.
By default, guide will output it’s findings in several text files:
•GUIDE_ADVICE.LIS – Management advice
•GUIDE_ERRORS.LIS – File errors
•GUIDE_STATS.LIS – Detailed statistics
•GUIDE_BRIEF.LIST – Brief statistics
•GUIDE_FIXUP.DAT – Damaged groups
Making it useful
Rather than using the default text files, we are going to use the –r option to direct output to a UniData file instead so we can use it more easily. To do this, we need to create a file to hold the data; CREATE.FILE UDT_GUIDE giving it an appropriate modulus depending on your number of files. So we can execute UniQuery commands against the file, we need to copy D_UDT_GUIDE from udthome\sys\D_UDT_GUIDE to this account, overwriting the existing dictionary.
The dictionary has a multitude of statistics that can be useful in managing your administration. Averages, minimums, maximums, file sizes, overflow statistics,
and much more. Please LIST DICT UDT_GUIDE to discover all the fields that are captured with this process.
Running Guide
Once the UDT_GUIDE file is created, go ahead and run guide once to get a feeling for what it will contain. You could either run it at the system level or from within UniData with the bang (!) operator.
:!guide –r UDT_GUIDE
By default, guide will analyze all files found in that account. Using the –i option, you can specify a text file that has a list of all the files you want to analyze, helping you to focus only on specific files you want to watch. As you’ll notice, guide creates a record for each file, making the filename the @ID for each record. Once this data is in our UniData file, you could do any type of calculations or logic to figure out how you want to use this information. Below we do a simple UniQuery statement that would show if any of your files are in overflow.
To further automate this prcess, you could create a cron job that ran this process at an appropriate interval, then run another program that searched for values out of your limits and perhaps send an email to someone that may act on this. It could be as simple as running specific UniQuery reports for occasional review or maybe creating a BASIC program that analyzed the data more in depth and took actions accordingly. Essentially, you could have your database alert you when it thinks it needs to be looked at for a potential file resize or other conditions!
More Information
To find more information and details about other options with guide, please read the UniData Commands Reference.
What Flavor is this UniVerse Account?
Steve Peterson, C-7 Reporting Solutions, LLC
From time to time we have all run into compatibility issues between different MultiValue Systems. Some commands work differently or may only be available on certain systems. UniVerse allows us to create different accounts with the ability to be compatible with various other systems. But then we need to know what “FLAVOR” account we are working in. We have come across a couple of ways to tackle this.
There is of course the “UPDATE.ACCOUNT” verb that copies NEWAC items in for your particular flavor, which also tells you your account flavor as it is updating the account. But you may not want to copy all those NEWAC items in.
So here is a very quick basic program that I call “WHAT.FLAVOR” to interrogate the SYSTEM() function and return the desired result.
Program WHAT.FLAVOR
* Program to display UniVerse Account Flavor
FLAVOR = SYSTEM(1001)
ACT = "Account Flavor is "
BEGIN CASE
CASE NOT(FLAVOR)
CRT "NOT a UniVerse Account"
CASE FLAVOR = 1
CRT ACT:"Ideal"
CASE FLAVOR = 2
CRT ACT:"Pick"
CASE FLAVOR = 4
CRT ACT:"Prime Information"
CASE FLAVOR = 8
CRT ACT:"Reality"
CASE FLAVOR = 16
CRT ACT:"IN2"
CASE FLAVOR = 64
CRT ACT:"PI/Open"
CASE 1
CRT ACT:"Other"
END CASE
STOP
END
The good thing about this quick little program is that if you run it on a non-UniVerse account, the SYSTEM(1001) function returns either a 0 or Null depending on the system. So it won’t cause any problems on non-UniVerse systems.
The above is a modified version of code provided as a courtesy by Binary Star Development Corporation. It is an excerpt from the Nucleus environment; a full featured 4GL http://www.binarystar.com,
For more information please feel free to contact Steve Peterson at SteveP@C-7Reporting.
Are you using Source Code Management Software?
If not, you should be!
By Don Robinson, Ashwood Computer Technician
This software may be referred to as Version Control or Source Code Control software. This software is designed to control and track changes to Source Code and files of any type. The two products I will discuss here are Perforce from www.perforce.com and Subversion from subversion.tigris.org.
Why should you use a third party product instead of doing it yourself? While you can certainly do something useful yourself, these products have years of design and work that you are unlikely to match. There is even a product available based on Multivalue but I don’t have any experience it.
I have worked with Perforce since 2002. Recently someone suggested I was dumb for paying for Perforce when Subversion is open source and free. This prompted me to check into Subversion.
There appears to be a big difference between the way they work. Perforce controls and tracks individual and groups of programs (files) whereas Subversion works on the entire repository. Some of the comments I read suggest that Subversion is much harder to use. For example, Subversion does not have an obliterate command although one was proposed in 2001! For more info on this subject, see this link: http://subversion.tigris.org/issues/show_bug.cgi?id=516. It seems that no one can figure out how to make obliterate work! However, Subversion does seem to be popular with the open source crowd.
Of course, I used this example because Perforce does have an obliterate command. This command is necessary to get rid of files added to the repository accidentally or files that are obsolete and just taking up space.
There are benefits to using Source Control whether you are a one man shop or are a world wide organization with dozens of programmers. With Perforce, this includes but is not limited to;
Prevents the modification of programs by two people at the same time or allows concurrent changes by providing a merge function.
Tracks who made a change and when the change was made.
Tracks what files were changed and the actual changes down to a single character.
Stores the current version of each file and the info necessary to automatically recreate any prior version of the file.
Provides commands to create a branch whereby separate versions of a program can be maintained while retaining history for the original program and both versions.
Allows you to clean up your programs by removing all the “commented out” garbage and unnecessary comments like “I changed file name from CUST to CUSTOMER”. Instead of keeping all these notes in the program, let Perforce keep the history record. If you or some one else looks at the program and thinks “that should be CUST”, you can look at the change in Perforce and it will show you what changed and why it changed.
Along with this, are you using a separate copy of the database and programs for development and testing? Perforce helps here as well, since you can modify and test changes on your test system and when they are ready, submit the changes to Perforce, sync the changes on the production machine, compile and go. No more ftp’ing files between system and no more program names like PRINT.INV.020599, PRINT.INV.0701105, PRINT.INV.OLD, PRINT.INV.SAVED, etc. The Perforce repository called a depot, should contain only working production code with all changes from the beginning of time. Test programs, one time programs and copies of programs should not be stored in the repository.
Perforce is a client and server system with both available for a wide range of Operating Systems. It has GUI, browser and command line clients. The repository can be accessed via the Web from anywhere in the world with access to the server. It has security features that allows control over what files a person has access to and what functions they can do. A two user version of Perforce is free with a modest charge for more than two users.
Word and Excel documents can be stored as well but because they are treated as binary files, some featured don’t work. You still have a history of changes and the comments from the person making the changes. For example, we keep our ISO documents in Perforce so we have a history of changes and the auditor likes it.
Computer systems continue their trend to become smaller, yet more powerful, on what seems to be a daily basis. Going back as little as 15 years ago, most of our computer systems were taller, wider, deeper, and heavier than most members of the offensive line of a football team! With this trend, these computer systems continue to demand fewer watts of electricity and can tolerate a wider range of operating temperatures. This allows many people to consider placing the new computer system out amongst the people most likely to use the system on a daily basis, and removing it from a sheltered, protected environment that its predecessor enjoyed. This now exposes the system to low humidity (and the resulting ElectroStatic Discharge), more “dirty” power (some of which can be created by pencil sharpeners, space heaters, coffee cup warmers, etc.), and more power interruptions (such as the cleaning staff unplugging it in favor of their vacuum cleaner, etc.).
ElectroStatic Discharge can cause the disc controller or CPU to ‘hang’, such that one has to power off the system without benefit of a controlled shutdown; it can also accelerate component deterioration or cause immediate component failure, resulting in additional down time for repairs. Unfortunately, “dirty” power isn’t any easier on the system, its components, or the budget (for overtime to make up for the time lost due to the system being down). Increasing the system availability is a primary goal of every organization that relies on their computer system for their daily business. Most, if not all businesses can save more money by leaving the smaller systems in a sheltered and protected environment, much like that of the predecessor system, than by recycling that room and exposing the system to a harsher environment.
The electrical outlet(s) supplying power to the system should be of the Isolated Ground (orange in color, and requires a specific installation procedure that differs from standard outlets). These outlets should provide power to protective equipment, such as an active, on-line Uninterruptible Power System (UPS) (as opposed to an inactive, Stand-by unit that only steps in when power is lost) for this main system. Alternatively, you could have a UPS unit that protects the entire “computer room”, but still provide the orange isolated-ground receptacles for the equipment. Personal Computers in use by your staff should also be powered from isolated-ground receptacles, with that ground being connected to a common point that is also used for the computer system (if everyone is in the same building or on the same floor). There should be an air conditioner/humidifier that is specifically intended to protect electronic equipment (not people) for the room, set to provide greater than 30% Relative Humidity and less than 80% (50% is typical). The temperature range should be whatever is comfortable for any people that may work in the room, usually 68 degrees Fahrenheit to 78 degrees Fahrenheit (72 to 75, typically).
For most Multi-Value Database Systems, any interruption to the system that prevents the database or the underlying Operating System from shutting down in an orderly fashion is a significant event. Providing a reliable environment for the database system is ‘cheap insurance’ towards providing the greatest uptime and system availability for the company.
Guess what, that is what we are going to talk about. First off, the definitions of the MERGE.LIST command. This definition comes from the UniVerse version of U2, but the concept is the same all of the MultiValued implementation.
Use MERGE.LIST to merge two numbered select lists using relational set operations and put the result in a third select list.
Here is how it works. Let’s say that you have a product history file for the last 5 years of sales. Your sale of olive oil was down by 18% in the 4th quarter of this year and you want to know why. So you want to know which customers purchased olive oil in the 4th quarter of 2006 that did not purchase olive oil in the 4th quarter of 2007. Assuming that you have symbolic items that represent the quarter and the year the sale was made, and that the sales item contains the product sold and the customer number; you can use a combination of selects to find your answer.
Your first select may be for those that purchased olive oil in the 4th quarter of 2006.
SELECT SALES WITH QTR = “4” AND WITH YEAR = “2006” AND WITH PRODUCT = “OLIVE OIL” SAVING
UNIQUE CUST.NO
SAVE.LIST 4QRT.2006
Now list one has all of the customers that purchased olive oil in the 4th quarter of 2006. The “SAVING UNIQUE “says I don’t want to save the key to the SALES file, I want to save the Customer number of the company that made the purchase. And I only want to save the same customer number once, UNIQUE, even if the customer has 50 sales items in the SALES file. Now we have a list of our customers from 2006.
You can probably guess the select statement for the second list.
SELECT SALES WITH QTR = “4” AND WITH YEAR = “2007” AND WITH PRODUCT = “OLIVE OIL” SAVING UNIQUE CUST.NO
SAVE.LIST 4QTR.2007
The only difference is to change the year to 2007 and the list number to 2. We have saved the two lists to different saved lists names, 4QTR.2006 and 4QTR.2007 so we can use them later.
Here is the definition, from UniVerse 10.2, for the relational operations.
UNION list3 contains all elements from list1 and all
Elements from list2 that are not in list1.
INTERSECT[ION]
list3 contains all elements from list1 that are
also in list2.
DIFF[ERENCE]
list3 contains all elements from list1 that are
not in list2.
Looks like what we want is all of the items from list1 that are not in list2, because we want those that purchased in 2006 and not in 2007, so we want a DIFF relational operation. The syntax again is:
Now then what we need to do. We need to get list1:
GET.LIST 4QTR.2006 TO 1
Get list2
GET.LIST 4QTR.2007 TO 2
Merge the two lists into list3
MERGE.LIST 1 DIFF 2 TO 3
And then save list3 to a saved list name. Do not do anything between the MERGE.LIST and the SAVE.LIST because you will lose list3, I have done it many times.
SAVE.LIST 4QTR.2006.NOT.2007 FROM 3
Now our list of 2006 purchasers that were not 2007 purchasers is called 4QTE.2006.NOT.2007 and can be used and reused as desired.
You now have a list of the customers that purchased olive oil in 2006, but did not purchase it in 2007 and thus the cause of your 18% drop in the sale of olive oil last quarter. You can use this list to send out a mailer, develop a follow up list for your sales staff, or a list that you can use to replace some of your sales staff.
It is a simple concept, but a very powerful tool.
Tom Dodds
Tom is a MultiValue consultant based in Chicago that has worked with the MultiValued system since 1972. He spent the early part of his career as an analyst and manager at Escom in Seattle. He then spent sometime in developing the Prime Information System. In 1978 he became an independent consultant and has been traveling and consulting every since. Now his main concentration is the U2 products and migrations from one platform to another. Tom has a wide variety of experience on databases as well as hardware platforms. Contact Tom at tts@ix.netcom.com.
A Different Kind of Oil
By: Tom Dodds
One of the most unusual aspects of the MultiValue systems is their direct access capability. Every record has a key and it must be, somehow, addressed by that key. If you do not know the key to the item you are looking for, you have to select the file and find the key. This process can get very expensive, time wise, with a large file. An inventive person came up with the saved list concept - a way of making one pass through a file and finding all of the items you want to work with and saving that list of keys. Saved list can be very powerful.
The commands that are associated with manipulating saved list are things like SAVE.LIST, GET.LIST DELETE.LIST, EDIT.LIST and MERGE.LIST. Most everyone is familiar with the first four, but many users have never used the most powerful of the saved lists command, the MERGE.LIST.
What we are going to be talking about today is how to clean out these directories from within UniVerse and UniData MV systems, because I don’t have any of the other MV’d databases to work with, And we will talk mainly about UniVerse running on a AIX/UNIX platforms. The concepts shown here can be applied to all of the MV databases and most all of the OS implementations with a little creative thought on your part.
In the UniVerse environment these files are called &HOLD&, &SAVEDLISTS&, &PH&, and &TEMP&. In the UniData environment they are referred to as _HOLD_, _PH_ and SAVEDLISTS. In most environments you can search the VOC file for anything that starts with & for UniVerse or anything that starts with _ for UniData and get the directories we are seeking. Of course UniData had to make the SAVEDLISTS file easier to type so they left the “&’s” and “ _’s” off of the name and just called it SAVEDLISTS.
We have 4 combinations of environments to discuss
1UniVerse and Windows
2UniVerse and AIX/UNIX
3UniData and Windows
4UniData and AIX/Unix
Space does not allow us to go into very much detail in this article so we will keep this at a high level and if you need more help you can contact either Ashwood Computers or me for assistance.
In order to acquire the information we will need to complete our task we will need to write a basic program and address the content of these directories at the OS level. This will be accomplished through the EXECUTE statement using the CAPTURE option. For a UniVerse underUnix environment a statement such as:
EXECUTE ‘SH –c “ls –ld “&SAVEDLISTS&”’ CAPTURING MY.LIST. (You must quote the &’ed string)
-rwxrwx--- 1 doddst doddst 25 Mar 19 2007 PROG.FILES
In that basic program, I extract each like of MY.LIST to a simple variable THIS.LINE. Then I TRIM(THIS.LINE), then convert the space to field marks so I can access each column of the listing as a field. The fields we are interested in are field 6,7, and 8 and field 9. We need fields 6, 7, and 8 to determine how old this item is and whether we want to delete it or not. Notice in the “Later in the list” part of the display that the format of field 6, 7, and 8 has changed. UNIX’s ls processor will display the month, day, and time of update for files that have been updated in the last 6 month and will replace the time with the year for items updated more that 6 month ago. The way I figure this out is to index field 8 for a “:”. If there is a colon than it is within 6 month and you have to generate the year portion of the update date. When you do that be careful of the year bindery. If your in month 4 and the update date of the file is in Nov then the year is 2007 not 2008.
So once you have this information you can then decide to delete the item, move it to an archive file and then delete it or whatever is appropriate for you installation.
I have included a copy of a program, written for UniVerse and AIX/UNIX that could serve as an example of what has to be done. I am not holding this program up as the answer to all questions nor the example of a perfect program, but just an example of what would get the job done.
GOOD CLEANING
* Program to parse a directory file for deletion of old items
* Modification Log
* Date By Comment
* 03/02/07 Tom Dodds Initial coding
* Variables
ERRMSG = ''
ERR = ''
DONE = 0
THIS.DATE = OCONV(DATE(),'D4/')
THIS.MO = FIELD(THIS.DATE,'/',1)
THIS.YEAR = FIELD(THIS.DATE,'/',3)
GOSUB GET.PARAMETER
GOSUB SELECT.ITEM
END.OF.ROUTINE:
* Normal or abnormal termination of routine
IF ERRMSG THEN
LOOP
REMOVE ERR FROM ERRMSG SETTING MORE.ERR
CRT ERR
WHILE MORE.ERR REPEAT
END
STOP
GET.PARAMETER:
* GEt the data necessary to run this routine
TEST.DATE = ''
LOOP
CRT 'Enter date before which files will be deleted ':
INPUT TEST.DATE
IF TEST.DATE = upcase('END') THEN
RETURN TO END.OF.ROUTINE
END
TEST.DATE = ICONV(TEST.DATE,'D')
IF STATUS() THEN
CRT 'Invalid date entered'
TEST.DATE = ''
END
WHILE NOT(TEST.DATE) REPEAT
SELECT.ITEM:
* Select the items from the directory
CMD = 'SH -c "ls -lt ':SQUOTE('&SAVEDLISTS&'):'"'
* crt CMD:;INPUT QQ
EXECUTE CMD CAPTURING MY.LIST
LINE.CNT = DCOUNT(MY.LIST,@FM)
IF LINE.CNT < 0 THEN
ERRMSG = 'No items found'
RETURN TO END.OF.ROUTINE
END
MORE.LIST = 1
LOOP
REMOVE THIS.LINE FROM MY.LIST SETTING MORE.LIST
WHILE MORE.LIST DO
THIS.LINE = TRIM(THIS.LINE)
CONVERT ' ' TO @FM IN THIS.LINE
MO = THIS.LINE<6>
DA = THIS.LINE<7>
UPDATE.DATE = ICONV(MO:'/':DA:'/2008','D')
UPDATE.DATE = OCONV(UPDATE.DATE,'D4/')
MO = FIELD(UPDATE.DATE,'/',1)
ITEM.NAME = THIS.LINE<9>
YEAR = THIS.LINE<8>
IF INDEX(YEAR,':',1) THEN
IF MO > THIS.MO THEN
YEAR = THIS.YEAR-1
END ELSE
YEAR = THIS.YEAR
END
END
THIS.DATE = ICONV(MO:'/1/':YEAR,'D')
* CRT THIS.DATE:' ':TEST.DATE :' ':ITEM.NAME:;INPUT QQ;IF QQ = 'Q' THEN RETURN TO END.OF.ROUTINE
IF THIS.DATE < TEST.DATE THEN
* CMD = 'DELETE &SAVEDLISTS& ':ITEM.NAME
CRT THIS.LINE<9>
* INPUT QQ
END
REPEAT
END
Spring Cleaning - By: Tom Dodds
It’s that time of year again. Time to start thinking about doing some spring cleaning, time to start cleaning out those pesky MV directories that just keep accumulating more and more forgotten items and keep slowing down some of your most critical processes. These general purpose directories are just that, they are storage units defined as directories in the underlying operating system. As such, every time you add data items to those directories its entry goes to the end of the allocation table. Each time you want to access this item the OS/MV has to search down the list of items in that directory to find the items you were looking for. As that list gets longer and longer the access gets slower and slower. Operating Systems, both UNIX and WINDOWS are not designed to house thousands of items in a directory. They are meant to be lean and clean, quick access type storage units.
Builiding the Killer App
By: Jim Paul
Here are some tips, mostly from the IT department of a small company; we will call the MCO that is whooping the pants off larger competitors. I had the pleasant experience of participating in my client’s acquisition of a similarly sized company, both in the business
of managing workers compensation claims. Interestingly the acquired company had purchased several companies and at one time was much larger.
IT seems to have been their Achilles heel. Though my client is one of the smaller players in the market, they are consistently ranked number one against their competitors, and I suspect they are one of the most profitable. Excellences in all areas are responsible for this success, including their home grown software, which has proved to be a strategic
asset. We have been following an old school path, and that has turned
out to be a winner so far.
My first observation on working for this company - it is a lot more fun to work for a company where the IT department is more than just a cost center. One would think that software companies are the place to work if you want some respect. But in this case IT was crucial for providing the flexibility for integration with the operations of their main business partner, the state of Ohio. In a crowded market, IT missteps can easily be enough to do a company in. Eight years ago, a change in the way the state of Ohio handled workers compensation caused about 30 companies to enter what was then a new market. About half survive and the number of survivors is expected to eventually reach a handful. It appears that the companies that relied on packaged software were the ones that didn’t make it.
What I can say authoritatively is that 3 principles drive the management of the IT function at this company and that the weapon under the covers is a MutliValue database. Living in the MultiValue world, we tend to have similar beliefs that at times fly against conventional wisdom. Maybe you'll relate to some of the ideas below.
* Service Users Directly
Have no middle man, no paper work, maybe no cubical wall, between the users and the programmer. It works at the MCO because we respect each others time and we share the goal of keeping everyone productive and the company profitable. It helps that people like working there and that they see the company prospering as their best avenue for a fulfilling work life. At this company the "work/change order" tracking system is just a spreadsheet. I have worked in a department that has a more formal system but they not require users to fill out a form to get help or request a change. The IT staff fills out the request because it is a tool for IT to manage itself. In that company the manager does set priorities and direction but usually puts the programmer face to face with the user. Users are the source of ideas of how to streamline processes and that's what IT is all about.
What about the potential loss of programmer productivity if they are highly available? For one, unless the programmer is working for a software company he is not producing income as directly as many other employees. And two, people wasting other people’s time, if that problem exists, is a company culture problem. If a user describes a problem saying "the computer is broken", which is probably the most common abuse of IT resources, I will usually walk through the problem with them, talking out loud so they can see the process of elimination used in diagnosing problems. I often use the words "you can..." stressing the “you”. I might also talk about how accurate communication helps me get my work done. The conversations can be touchy but it is better than the usual approach of isolating users from IT.
* Simplify Everything
Fewer machines, fewer operating systems, fewer software products, fewer dependencies, fewer layers…less is more. Less is more is one of the hardest places to stand because it will pit you against vendor/consultants looking for places to use their hammers, and programmers who might want to do resume driven development. You will find yourself fighting "conventional wisdom" left and right. I remind myself that some IT professionals come from a different planet, one where lots of stuff and people are good things. For example, I find myself defending that my apparent disregard for internal network security is do to the fact that our users call IT when a menu bar seems to disappear when they docked it, and so would rather budget for some training than defend against a spoofing attack from them.
* Don't Throw Technology at Problems
Did Bob Marley sing "Computerize it"? No he did not. While computerizing processes brought huge gains in the past, the easy stuff is done. What's left needs to be weighed against the dependencies, support and training burden computer solutions entail. Document imaging? You're probably going to have to upgrade staff to save space in the file room. And when you have a problem, your summer high school hire will not be able to fix it, and almost assuredly more than one person will be stopped from doing their job. Projects like this are not necessarily a bad idea, but make sure benefit to TCO is sound.
Another situation I've seen that can result in a bad project is one where a manager is facing a personnel issue he doesn't want to deal with. Appreciating the talent in the IT department he suggests a project he hopes will simplify things to a point where a narcoleptic monkey will be able to run it. Unfortunately the need for attentiveness and accuracy usually increases when a process is more automated.
* Conclusion
There was a recent survey of CIO's by Robert Half where a number of CIO's said not only that they were increasing IT staff and training in the coming year, but wish they had spent more on beefing up internal capabilities in the past. The strategic advantage that a dexterous IT department will deliver is the ability to refine your processes. The payoff is in the accumulation of savings from many small gains. "Sandy no longer has to key in orders that arrive by fax." "John no longer has to enter numbers from three reports into a spreadsheet to close the month." I suspect that the technology story of many winners is not sexy - not something you'd read about in Wired Magazine. We need to keep in mind that the n-tiered applications, the service oriented architecture, the virtualization just about everything, have the common simple goal of saving someone a little time.
Jim Paul is a free lance consultant/programmer living in Cincinnati. He can be reached at JimPaul at fuse net.
Regular Expressions
by Kevin Finley
You just received a requirement to read an ID number from users or from a comma separated data feed. This ID number can be either an SSN or EIN. They haven't asked yet; however, your users will likely ask you to accept a data field that reads “ID: X###### db” where db is a 2 letter suffix indicating the database from which is came. The origin database is important because the company sending this feed duplicates id numbers between databases.
Question: What's the best way to read this field?
Answer: Regular expressions.
Regular expressions are supported in all kinds of technology ranging from the major UNIX text editors (vi and emacs) to the UNIX command line tool “grep” to many GUI editors on windows to nearly all edit masking libraries to most programming languages, including Java, Perl, Ruby, C#, bash (through grep), and many others. A very incomplete list of supporting implementations can be found at http://en.wikipedia.org/wiki/List_of_regular_expression_software.
Let's satisfy the first requirement: differentiate SSN from EIN. Our software can have a list of acceptable expressions and branches. In pseudo code, this looks like:
IF ssn_regular_expression.matches( input ) THEN
CALL ssn_processing( input )
ELSE IF ein_expression.matches( input ) THEN
CALL ein_processing( input )
ELSE
CALL inform_user_of_error( input )
END IF
The 2 regular expressions listed above can then do the heavy lifting of validation. Let's look at their basic implementations in pseudo code:
Various characters are special. In this case, we used [] which indicate any one character listed inside. When used inside [], the - character indicates a range so 0-9 indicates any one character 0123456789. Any character that is not special is interpreted literally so the - used outside [] indicate the literal -. Even though the above expressions work, we can make them easier to read. The following does exactly the same work:
You'll notice that we used numbers inside {}. This indicates the immediately preceding expression – in this case, a character class, the formal term for [] – repeated that number of times. This makes a bit easier to understand that we assigned 3 numbers followed by – followed by 2 numbers followed by dash followed by 4 numbers to ssn_regular_expression.
Next month, the users want that nasty data field “ID: X###### db”. It simply cannot use the comma separation logic we use for the rest of the feed so we'll learn how regular expressions can separate this field for us.
About the Tech
Kevin Finley is an independent consultant specializing in architecture integration including data models, relational databases, object databases, network communication, metadata, and parsing. He can be reached at ashwood-blurbs@tomorrowenterprises.com.
Regular Expressions Part II
By: Kevin Finley
Last month, we received the following requirement:
...read an ID number from users or from a comma separated data feed. This ID number can be either an SSN or EIN. They haven't asked yet; however, your users will likely ask you to accept a data field that reads “ID: X###### db” where db is a 2 letter suffix indicating the database from which is came. The origin database is important because the company sending this feed duplicates id numbers between databases.
We also learned how to solve the first part, identifying a field that contains either a SSN or an EIN depending on the placement of dashes using regular expressions. This month we will learn to validate the field from the data feed as well and then to extract the important pieces.
The validation expression looks much like what we learned last month:
db_regular_expression = ID: X[0-9]{6} [a-zA-Z]{2}
where this expression says we want to find the exact string (case-sensitive) “ID: X” followed by 6 numbers followed by a single space followed by 2 upper- or lowercase letters. Remember,
1.regular expressions are case-sensitive by default
2.[] create a character class for one character position
3.{} say to repeat the prior element a specific number of times; we used this to state “repeat the prior character class (numbers) 6 times” and “repeat the prior character class (2 times)”
Now that we have valid input, how do we extract the important stuff to use in a program? That's just as easy! Modify the expression as follows:
What changed? We added some parentheses. Parentheses define a capture group. Capture groups tell the regular expression parser that you want whatever is contained between them IF the input is valid. Here is pseudo code to use this expression:
IF db_regular_expression.matches( input ) THEN
id = db_regular_expression.group( 1 )
db = db_regular_expression.group( 2 )
CALL deal_with_id( db, id )
END IF
A few things to note:
●There are lots of library flavors of regular expressions so not all expressions work identically everywhere.
●Capture group 0 is normally the entire input string.
●There are many books describing regular expressions.
●You can become proficient with regular expressions in an hour, but it will probably take years to unlock the full power.
●Regular expressions can be computationally expensive to construct if you need to do it thousands of times per second. Most languages/options provide a means to construct the state machine and keep it around for use on future input strings.
●Support is widespread:
built into languages like Perl, Ruby, and Javascript,
available as a library (sometimes standard, sometimes downloadable) in C, C++, VB, Java, etc.,
accessible from the command line using grep, egrep, fgrep, sed, awk, etc., and
many editors such as vi, emacs, Eclipse, Visual Studio, etc.
Kevin Finley is an independent consultant specializing in architecture integration including data models, relational databases, object databases, network communication, metadata, and parsing. He can be reached at ashwood-blurbs@tomorrowenterprises.com.
