Date: Mon, 18 Jul 1994 07:45:36 -0500 From: BITNET list server at SUVM (1.8a) Subject: File: "LISTDB MEMO" To: David Hanson LISTSERV historical documentation, release 1.5n ----------------------------------------------- Copyright Eric Thomas 1986,1987,1988 +---------------------------------------------------------+ | Revised LISTSERV: Database Functions | +---------------------------------------------------------+ | | | Document number: U01-012-0 (September 3rd, 1988) | | | | Document fileid: "LISTDB MEMO" (from "Info DATABase") | +---------------------------------------------------------+ Preface This manual is an introduction to the LISTSERV database functions. It is intended to be a reference document for general users with little or no knowledge of database systems. It does not contain any tech- nical information that general users do not need to worry about. In particular, the interactive database access protocols are not docu- mented in this manual; they can be found in the "Revised LISTSERV: Application Programmer's Guide" , document number A01-004. This document will discuss the syntax and operational characteristics of the LISTSERV database subsystem. It is assumed that the reader either o Has obtained a copy of the interactive database access user inter- face, LDBASE, and is familiar enough with his operating system to know how to install and invoke it. o Is familiar enough with the the "LISTSERV Command Job Language Interpreter" to build up a simple job and send it to LISTSERV for execution. A sample job "skeleton" is described later on. Conventions ----------- The following typographical conventions have been made in this docu- ment to improve its readability: | o Recent changes in the publication are indicated by a vertical bar | in the left margin. ! o Intermediate changes between two releases of the document ("Pre- ! releases") are flagged with an exclamation point in the left ! margin. Features described in this fashion should be considered ! as not documented and not officially supported until the exclama- ! tion point is removed. > o Temporary restrictions or circumventions are marked with a > "greater than" sign in the left margin. This sign may also be used > to signal obsolete features for which support will be dropped in > the next release. + + Paragraphs marked with a '+' sign in the left margin contain detailed + explanations for experienced users and can be skipped at first + reading. + **************** * Introduction * **************** The LISTSERV database functions have been developed in an attempt to make it possible for users to extract relevant information from list archives without having to retrieve a large "notebook" file and scan it locally. Users will send commands to LISTSERV, requesting it to perform search operations locally and to send out only the selected items from the list archives. The following goals have been kept in mind all throughout the development of the new facilities: o The functionalities provided must be general enough to allow for databases other than list archives (i.e. electronic mail) to be used if needed. o Users with little or not database experience must be able to learn how to use the LISTSERV database in a few minutes. o The syntax should be as close to "natural english" as possible, and should be easy to remember. o The commands must be powerful enough to be functional, but they should not be overly complex so as not to discourage beginners. o Interactive access to the database through the network is primor- dial. Once the search has been carried out, the user should have the option to retrieve the results in a file rather than as inter- active output. Because their main application is the scanning of list archives, the LISTSERV database functions are document-oriented and therefore quite different from "usual" commercial database systems. Accessing the database ---------------------- The database can be accessed either interactively or in "batch" mode. In the former case, you must obtain the LDBASE user interface by sending the following commands to your nearest LISTSERV: o For VM/SP CMS systems: - TELL LISTSERV AT nodeid GET LDBASE EXEC - TELL LISTSERV AT nodeid GET LSVIUCV MODULE The command to start the user interface is simply LDBASE to access your "home" server, or LDBASE nodeid to access the LISTSERV server at nodeid. o For VAX/VMS systems: - SEND LISTSERV@nodeid GET LDBASE COM The command to start the user interface is @LDBASE. This will install the required files in your directory and display more detailed instructions about the program. o Other systems may not presently access the database in interactive mode. Interactive access The exec is self-documented, and will ask you the userid and nodeid of the server you wish to access, after which it will try to establish a network connection to its database. This may fail if a line is down or if interactive database access has been disabled at the installa- tion you are trying to reach. In that case, you would be forced to send requests in "batch" mode. In interactive mode, you must enter your commands directly (without any embedded CJLI orders) when, and only when, you are prompted to do so. Entering a command out of sequence is interpreted as a request to cancel the session and leave the user interface. You should do so only if you have reasons to think that the network connection has been broken, or if you have to leave your terminal, etc. Batch access When accessing the database in "batch" mode, you must construct a CJLI job which you must then submit to the appropriate server for execution. This means that you must know in advance what you want to do exactly. If you are not familiar with CJLI, you can use the following "job skeleton" to build up your database search job: +--------------------------------------------------------------------+ | // JOB Echo=No | | Database Search DD=Rules | | //Rules DD * | | command 1 | | command 2 | | ... | | /* | | | | Figure 1. Sample database job skeleton | +--------------------------------------------------------------------+ You will then receive a "DATABASE OUTPUT" file containing the results of your search. This file might look like this: +--------------------------------------------------------------------+ | > Select * in TEST-L | | --> Database TEST-L, 4 hits. | | | | > Index | | Item # Date Time Recs Subject | | ------ ---- ---- ---- ------- | | 000001 87/10/18 13:09 12 This is a test looking for upcasing | | 000002 87/08/24 09:18 9 | | 000003 87/10/18 13:09 8 Test - please acknowledge receipt | | 000004 87/10/18 13:09 7 Does Reply-To=Both work correctly? | | | | Figure 2. Sample DATABASE OUTPUT: Each of the commands in the | | original job is echoed in the output file (unless | | specifically disabled). | +--------------------------------------------------------------------+ If you realize that the items you were interested in are number 1 and 3, you will have to submit a new job to ask for a copy of them, and LISTSERV will have had to do part of the work twice. This is the main advantage of interactive access over batch. Organization of the manual -------------------------- The next chapter of this manual will introduce the concepts required to understand the LISTSERV database system. The various database commands will then be explained, starting with the simplest syntax forms. For each command, there will be a short tutorial describing the most important forms of the syntax, followed by a detailed, exacting description of the command. The last chapter will describe the more advanced functions like changing the CPU limit, output lines limit, etc. ************ * Concepts * ************ This chapter introduces the various concepts required to understand the LISTSERV database system. What is a database? ------------------- A database is a collection of items of homogeneous type, which can be stored in computerized form. For example, a telephone directory, the archives of a distribution list, a list of employees can all be considered to be databases. Database items will often be referred to as documents since they will, in most cases, be just that. The actual "physical" structure of the database is usually quite complex, but it is completely transparent to the end-user who sees only a series of documents. Finally, to each database is associated a database type. Databases of the same type can be expected to contain roughly the same type of documents. For example, all list archives have a database type of "Notebook". The user does not have to specifically know the type of a database in order to access it. However, the keywords (see definition below) that can be searched will normally depend on the database type. For example, you can search a list-archives database for a "Subject" of "Comments on the minutes of the last meeting", but not for a "Diam- eter" of "1/10th" (which might be valid for a database of electric wires). What are the attributes of a document? -------------------------------------- Attributes common to all database types Each document is assigned a document number, which is unique all throughout the database and can therefore be used to identify it. + + The structure of the LISTSERV database system does not guarantee that + document numbers will not change from one session to the other. If, + for example, a list archive file is edited to add or remove some + entries, other documents might be renumbered and assigned a different + document number. + In addition, to each document is associated a date/time field, whose exact meaning will depend on the type of the database. It will usually be either the document creation date or the date of the last change made to it. Other attributes Document keywords Depending on the type of database, one or more keyword might have been defined for the documents. Each of these keywords is a name/value pair, such as "REFCODE 2378237" or "SENDER WIZARD@TOWER.FaerieNet". Abbreviations or synonyms might have been defined for the keyword name. For example, you might be able to specify "Author" instead of "Sender" when performing a search operation. Document portions In any case, these "keywords" contain information which is external to the document, in much the same way as a label sticked on a book contains information (e.g. the price of the book) which cannot be found inside the book itself. A document portions, on the contrary, is a name that refers to a subset of the contents of the documents, like "Foreword" or "Epilogue" in our book example above. There is a built-in document portion, "ALL" , which is common to all the database types and corresponds to the complete document. What are the attributes of NOTEBOOK databases? ---------------------------------------------- The following "keywords" are available for list-archives databases (shortest abbreviation is Capitalized): o Subject corresponds to the contents of the "Subject:" field of the mail header (case is preserved). o SEnder or From identify the RFC822 address of the originator of the mailfile (case is preserved). You should note that "FROM" may be a reserved database keyword, depending on the context in which it appears. You should therefore avoid to refer to the mail-ori- ginator keyword as "FROM", unless you use an abbreviated form. The following "document portions" have been defined for list-archives databases (shortest abbreviation is Capitalized): o Header and HDR both identify the RFC822 header of the mailfile. o Body and Text correspond to the contents of the mailfile, i.e. anything which appears after the header. o And of course, ALL corresponds to the complete mailfile, header and body. What is a hit? -------------- Before you are able to display anything, you will have to perform a search on the database (if you have nothing to search in particular, there is a special form of the search command which lets you select all the items from the database). You will specify a series of search rules, which will be applied to all the documents in the selected database. Each document that matches these search rules is called a hit and is considered to have been selected for further processing. Most database commands will, by default, be applied to all the docu- ments that have been "hit" in the last search you have performed. This means that if your previous search did not yield any "hit", you will not be able to do anything until you issue another search command. How can I get a list of existing databases? ------------------------------------------- Although there is no global list of all the available databases, each LISTSERV can send you a list of all the databases that it keeps. If you are not allowed to access a given database, it will (usually) not be listed so that you do not waste your time trying to search it. To get this list, simply send a DATABASE LIST command to the desired server. +--------------------------------------------------------------------+ | Database Description | | -------- ----------- | | APPPRG-L Archives of "ADMCSC Application Programmers list" | | BITEARN Information on all the BITNET/EARN/NetNorth nodes | | LISTS Information on all the network-wide LISTSERV lists | | MANUALS Information about IBM manuals in the ADMCSC library | | PEERS Information on all the LISTSERV servers in the network | | SYSPRG-L Archives of "ADMCSC Systems Programmers list" | | SUPPORT LISTSERV software support database | | | | Figure 3. Sample DATABASE LIST command | +--------------------------------------------------------------------+ ********************** * The SEARCH command * ********************** This chapter will introduce you to the SEARCH command. This minimum abbreviation of this command is just "S", and there also exists a synonym of SELECT with a minimum abbreviation of "SEL". The syntax of this command is a bit complex, and will be introduced step by step. Basic search functions ---------------------- The two most important things you have to indicate when you search a database are: 1. The name of the database you want to search. 2. What you want to search the individual documents for. The name of the database to be searched is specified after the words or phrases to be sought and is prefixed with an IN keyword. For example, we might do this: Search Rosemary in MOVIES This would select all the entries from database "MOVIES" containing the string "ROSEMARY". Now if you just wanted to see the list of all the movies you can see this week, you could have used an asterisk as search argument to select all the entries in the database: Search * in MOVIES Note that the database name doesn't have to be uppercased. This is merely done to make the examples look better. If you want to "narrow" your previous search, i.e. perform additional tests on the documents that have been previously selected, you must omit the IN keyword. In that case, the search will be applied to the previous "hits" and will create a new "hit list". But in most cases, we will want to search for something longer than one word, for example part of a "key" sentence. Search Hardware problem with a 4381 in IBMFORUM Another problem is that we might not remember the exact original sentence. This is not very important, since LISTSERV will search each word indi- vidually: in the above example, any entry that contained the words "hardware", "problem", "with", "a" and "4381" would have matched the search, even if the words appeared in a different order. But what if the original document had "4381-13" in it, instead of "4381"? This is again no problem, as LISTSERV does not require the word to be surrounded by blanks to find a match. Case is also ignored when performing the search operation. That is, "problem" would have found a match on "problems"... and "with" would have found a match on "without" or "withstand"! This may sound like inconsistent behaviour, but you should keep in mind that it is always possible to "narrow down" a search operation. However, once a document has been excluded from the list of "hits", it is very difficult to bring it back. Now what if I want to search for an exact string? For example, I am interested in the string "in C". It is very likely that just any document in the database will contain both a "in" and the letter C. But what I am interested in is things which have been written, or programmed, or implemented, "in C". In that case, it is possible to force LISTSERV to group words together by quoting them, as in: Search 'in C' in UTILITY This method can also be used to insert extra blanks between or before words: leading and trailing blanks are normally removed automat- ically, but they are preserved inside quoted strings. Please note that quotes must be doubled when specified inside quoted strings, as in: Search 'Rosemary''s baby' in MOVIES The search for 'in C' resulted in over fifty hits, because a match was erroneously found against "in clear", "in core", etc. However, I do not want to search for 'in C ' because there might be hits with "in C." or "in C," in the database and I don't want to miss them. If the search respected the capital C, it would no longer find all those irrelevant hits. To do this, you must enclose your search string in double-quotes instead of single quotes, for example: Search "in C" in UTILITY Note that single quotes should not be doubled inside double-quoted strings, and vice-versa. Only quotes of the same type than the string should be doubled. It is important to understand the difference between the two types of quoting. If you request a search for 'TEXT', you will find a match on "TEXT", "Text", "text" or even "teXt". This is the same behaviour as unquoted text. However, if you request a search for "TEXT", it will only find a match on "TEXT", not on "text" nor "Text". + + Quoting is also the only way to search for a reserved keyword like + "IN": if you tried "Search in in UTILITY", LISTSERV would report that + database "IN" does not exist and would reject the command. This is + because the keyword IN indicates the end of your search arguments. If + you quote it, however, it will not be recognized and will be searched + as you wanted it done. Similarly, if you want to search for an + asterisk, you will have to quote it since "Search *" indicates that + all entries should be selected. + Now the problem is that there may be sentences starting with a capital I, e.g. "In C, it would be coded this way:". How can I catch these sentences? Actually, you have been using "complex search expressions" from the beginning without even being aware of it. When you specified a search on "Hardware problem with a 4381", you had in fact been asking LIST- SERV for: "Hardware AND problem AND with AND a AND 4381". The "AND" is implicit, but it may be overriden. You may even use parenthesis if needed: Search ("in C" or "In C") and program in UTILITY The "AND" can still be implied, as in: +--------------------------------------------------------------------+ | Search wooden chair (blue or green) in CHAIRS | | Search (wooden chair) or (plastic chair) in CHAIRS | | Search plastic chair (blue or green but not streaked) in CHAIRS | | | | The following commands are strictly equivalent: | | Search (wooden chair) or (plastic chair not blue) in CHAIRS | | Search chair (wooden or (plastic not blue)) in CHAIRS | | Search chair (wooden or (plastic but not blue)) in CHAIRS | | Search chair AND (wooden OR (plastic AND NOT blue)) in CHAIRS | | | | Figure 4. Sample SEARCH commands using complex document search | | arguments | +--------------------------------------------------------------------+ Date specifications ------------------- Since each document has been assigned a "date/time" field, it is possible to select documents based on this date field. This is accom- plished by appending "date search rules" to the search expression, as in: +--------------------------------------------------------------------+ | Search problem (serious or severe) in BBOARD since july | | Search problem in BBOARD since oct 85 | | Search symptom in BBOARD since 12/28 | | Search error report from 12 january to august in BBOARD | | Search user complaint until 18 sept in BBOARD | | Search data check since today 11:53 in EREP | | | | Figure 5. Sample SEARCH commands using date search arguments | +--------------------------------------------------------------------+ The default values for omitted arguments are always chosen so as to exclude as little entries as possible. For example, "July" would mean "1 July 00:00:00" in a SINCE specification, and "31 July 23:59:59" in an UNTIL clause. The only exception is the year field, which always defaults to the current year. Keyword search specifications ----------------------------- The last thing you may wish to search is the "keywords" list. For example, you might want to select those plastic chairs which cost less than 50 dollars. It is assumed that the price will vary often (maybe almost daily), and that it is therefore kept externally from the docu- ment describing the chair. Thus, you would have a "Price" keyword which you could search in the following way: Search plastic chair in CHAIRS where price < 50 You may of course use complex expressions (with parenthesis) in the WHERE clause. There are new comparison operators available for this clause, like IS, CONTAINS, all the usual arithmetical comparison oper- ators, and some more. However, the AND operation is no longer implied, but it can still be specified explicitly of course: Search plastic chair in CHAIR where price < 50 and avail > 4 The problem now is that, as the search commands become more and more complex, they will no longer fit in a single line. To solve this problem, it has been decided that any database command ending in a dash indicates that more is to follow on the next line. This process can be repeated several times if desired. This applies to both inter- active and batch commands. +--------------------------------------------------------------------+ | Search chair (wooden or (blue or green but not streaked)) - | | in CHAIRS - | | where price < 50 & avail > 4 | | | | Search chair (wooden or (blue or green but not streaked)) - | | in CHAIRS where price < 50 & avail > 4 | | | | Search chair (wooden or ( - | | blue or green but not streaked) - | | ) - | | in CHAIRS where price < 50 & avail > 4 | | | | Figure 6. Sample SEARCH commands with continuation lines: All | | these commands are strictly identical, although the | | first one is obviously more legible. | +--------------------------------------------------------------------+ The only "trick" about this continuation line business is that you should always keep quoted strings on a single line. The process of identifying continuation lines and concatenating them afterwards may cause unwanted blanks to be inserted in the command line, which is no problem outside a quoted string since blanks are ignored, but might cause erroneous results in a quoted string. If you want to search for several possible values in a given keyword, you do not have to repeat the keyword name and operator: +--------------------------------------------------------------------+ | > Search * in BBOARD where - | | > subject contains (PC or (Personal and computer)) | | | | is strictly equivalent to: | | | | > Search * in BBOARD where - | | > subject contains PC or - | | > (subject contains Personal and subject contains computer) | | | | Figure 7. Sample use of "factorization" | +--------------------------------------------------------------------+ However, it should be noted that this "factorization" is performed according to the rules of logic, which may not necessarily match those of english grammar. This removes any possible ambiguity as to the meaning of these clauses. Let's consider the following example: machine does not contain (IBM and DEC) This clause will get translated into: machine does not contain IBM and machine does not contain DEC In english you would probably say "machine contains neither IBM nor DEC" . This is how LISTSERV will understand it. However, if you read the clause aloud, you will probably not pronounce the parenthesis and will end up saying "machine does not contain IBM and DEC", in other words, "machine does not contain both IBM and DEC" , which is a totally different thing (and would most probably be true all the time). The "english meaning" could be obtained with the following clause: not (machine contains (IBM and DEC)) In the former case, the negative "does not contain" operator is inserted inside the parenthesis. In the latter, only "contains" is moved, and the negation remains outside. +--------------------------------------------------------------------+ | > Search gateway problem - | | > in BBOARD - | | > since sept 86 - | | > where sender contains (john or paul but not mick) - | | > and subject does not contain lost | | --> Database BBOARD, 5 hits. | | | | > Index | | Item # Date Time Recs Subject | | ------ ---- ---- ---- ------- | | 000012 87/10/18 13:09 12 The gateway has stopped working | | 000017 87/08/24 09:18 9 Glory glory alleluja! Again!!! | | 000018 87/10/18 13:09 8 You know what? It WORKS!!! | | 000024 87/10/18 13:09 7 Guess what happened today? | | 000205 87/10/04 16:59 9 Who's going to babysit it today? | | | | | | You might now wish to narrow your search down to exclude postings | | whose subject contains "work". You can do this by specifying a | | new WHERE clause with no associated IN. | | | | > Search * where subject does not contain work | | --> Database BBOARD, 3 hits. | | | | > Index | | Item # Date Time Recs Subject | | ------ ---- ---- ---- ------- | | 000017 87/08/24 09:18 9 Glory glory alleluja! Again!!! | | 000024 87/10/18 13:09 7 Guess what happened today? | | 000205 87/10/04 16:59 9 Who's going to babysit it today? | | | | Figure 8. Sample SEARCH commands with keyword search clauses | +--------------------------------------------------------------------+ Finally, the reason why the database name appears in each reply from LISTSERV is that you can specify several database in the IN clause: +--------------------------------------------------------------------+ | > Search user complaint in BBOARD1 BBOARD2 - | | > since august - | | > where sender is charles | | --> Database BBOARD1, 2 hits. | | --> Database BBOARD2, 8 hits. | | | | | | Figure 9. Sample SEARCH commands involving several databases | +--------------------------------------------------------------------+ Phonetic search --------------- There may be cases where you are looking for a certain value of a keyword, the exact spelling of which you cannot remember. In these cases, it may be useful to try a phonetic search. A phonetic search will yield a match for anything that "sounds like" your search string, as dictated by a predefined algorithm which is of course not perfect. It may give a hit for something which does not actually sound like your search string, or, more rarely, omit a keyword which did sound like what you entered. The main reasons for this are that the algo- rithm must be fast to execute on the machine and therefore not too sophisticated, and that the way a given word is pronounced depends on the idiom in which the word was written. For example, the phonetical transcription of the name "Landau" will be different in French, English, German and Russian. Thus, it is impossible to decide whether a word sounds like another if the language in which the words are pronounced is not known (and of course LISTSERV does not have, a priori, any way to know it). Phonetic searches are performed through the use of the SOUNDS LIKE and DOES NOT SOUND LIKE operators, which are syntactically similar to CONTAINS and DOES NOT CONTAIN. That is, you could do something like: Select * in PHONEBOOK where NAME sounds like WOLF There is a little trick with the SOUNDS LIKE operator that you should be aware of. If your search string (WOLF in our above example) is a single word, it will be compared individually to all the words in the reference string (i.e. the data from the database), and will be considered a hit if it "sounds like" any of the words in the reference string. Thus, the search word "Ekohl" sounds like the reference string "Ecole Normale Superieure" because it matches the first word. If the search string contains more than one word, the search and reference strings will be compared phonetically as a whole (and "Ekohl Dzentrahll" will therefore not match "Ecole Normale Superieure"). Note that any search string containing more than a single word must be quoted, as explained in the previous sections of this chapter. +--------------------------------------------------------------------+ | > Select * in BITEARN where site sounds like (COHRNEAL and LAPORRAD|RY) | --> Database BITEARN, 3 hits. | | | | > Index | | Ref# Conn Nodeid Site name | | ---- ---- ------ --------- | | 0292 87/03 CRNLASSP Cornell University Cornell Laboratory of Atomic| | 0301 87/03 CRNLION Cornell University Cornell Laboratory of Plasma| | 0307 87/06 CRNLNUC Cornell University Laboratory of Nuclear Studes| | | | > Select * in BITEARN where SITE sounds like HOPTIKK | | --> Database BITEARN, 2 hits. | | | | > Index | | Ref# Conn Nodeid Site name | | ---- ---- ------ --------- | | 0751 87/09 FRIHAP31 Assistance Publique - Hopitaux de Paris | | 2120 87/04 UOROPT University of Rochester The Institute of Optics| | | | > Select * in BITEARN where SITE sounds like SCHIKAGO | | --> Database BITEARN, 1 hit. | | | | > Index | | Ref# Conn Nodeid Site name | | ---- ---- ------ --------- | | 0140 86/03 BMLSCK11 Studiecentrum voor Kernenergie (SCK/CEN), Mol, | | | | Figure 10. Sample SEARCH commands involving phonetic match: The | | first command shows an example of accurate phonetic | | match, where the result is exactly what the user | | expected. In the second example, the user found what | | he was looking for ("Optics"), but an additional | | unwanted entry was selected. This is by far the most | | common case. The last command is a typical example of | | phonetic clash, where the algorithm did not translate | | the search string into phonetics as the user expected | | it, with the result that the desired name ("Chicago") | | was not found and that completely irrelevant entries | | were presented instead. | +--------------------------------------------------------------------+ + + The phonetic matching algorithm used by LISTSERV is a slightly modi- + fied version of SOUNDEX -- a well-known algorithm that provides + reasonably accurate matches at a very low CPU cost. Although it gives + best results with the English language, for which it was originally + designed, it is not too strongly tied to it and can still be used with + other languages. It is of course absolutely impossible to write an + program that would work for all the languages in the world, or even + for the most widley used ones, since their interpretation of the most + common combinations of letters are completely incompatible. + Exact syntax description ------------------------ This section describes the exact syntax of the "SEARCH" command in technical terms. You can skip it if you are not interested in learning about the details of this command. General syntax +--------+-----------------------------------------------------------+ | | | | Search | search-rules | | SELect | | | | | | | Optional rules are: | | | | | | date-rules | | | keyword-rules | | | db-list | +--------+-----------------------------------------------------------+ The optional "date-rules", "keyword-rules" and "db-list" arguments may appear in any order. Database list specification For each SEARCH command, you may specify a list of databases to be searched. The default is to narrow the search, i.e. to use the result of the previous search as input for the new search. This is of course possible only if the previous search yielded one or more "hits". The syntax of the "db-list" specification is the following: db-list: IN <(> dbspec1 > <)> The parenthesis are optional. If they are omitted, database names may not be reserved keywords like SINCE or WHERE. Finally, the syntax of "dbspec" is the following: db-spec: db-name<.<(>range1<,><...>><)>> where "db-name" is the name of the database to be searched, and "ranges" are optional parameters restricting the search to a sub-list of entries in this database. Each "range" may be either a single entry number like 1274, or a range of numbers like 12-17, 827- or -40. They may be either enclosed in parenthesis and separated by blanks, or separated by commas, in which case the parenthesis are optional. +--------------------------------------------------------------------+ | IN (BBOARD1 BBOARD2) | | IN BBOARD1.12-40 BBOARD2 MGMT.(30-35 200-) | | IN REXXBB.200-,12-13 VMSTAFF.(61 80-100 12) | | | | Figure 11. Sample IN clauses | +--------------------------------------------------------------------+ Date rules specification You may optionally restrict the search to only those entries that lay within a given interval of time. This is accomplished by specifying one of the following date rules: SINCE date-spec FROM date-spec1 TO date-spec2 UNTIL date-spec The format of a "date-spec" is quite complex because of the number of different ways date/time specifications are usually expressed: TODAY yy dd mm
<->monthname<-> mm/yy mm-yy yy/mm/dd yy-mm-dd Month names can be abbreviated to any length. If there is an ambi- guity, the first month in chronological order is retained. For example, "J" would mean "January", "JU" would be "June" and "JUL" would unambiguously select "July". The format of a "time-spec" is simply >. +--------------------------------------------------------------------+ | FROM 14 july TO oct 87 | | SINCE 86 | | UNTIL 23-JUN-87 | | SINCE today 11:30 | | | | Figure 12. Sample date clauses | +--------------------------------------------------------------------+ NOTE: Case is irrelevant in date specifications. The keywords (SINCE, UNTIL, etc) have been capitalized only for better legibility, and can be entered in lower case if desired. Keyword rules specification You may request the actual document search to take place only for those entries which match a set of "keyword comparison" rules. The syntax is the following: WHERE kwd-expression WITH "kwd-expression" is, generally speaking, an mathematical expression of keyword/value comparisons, possibly bound by logical operators. Comparison operators have a higher precedence than logical operators, that is, "A>10 AND B=20" is interpreted as "(A>10) AND (B=20)". The available comparison operators are listed below. All the operators appearing on a given line are synonyms. +--------------------------------------------------------------------+ | = IS | | ^= <> IS NOT | | > | | < | | >= | | <= | | CONTAINS | | DOES NOT CONTAIN | | SOUNDS LIKE | | DOES NOT SOUND LIKE | | | | Figure 13. Comparison operators for WHERE clauses | +--------------------------------------------------------------------+ All these operators are self-explanatory, except the last two which allow you to search the keyword value for a given "substring". That is, "Sender contains jeff" would be true if the value of the "Sender" keyword was "Jeff Smith" or "Jeffrey Donaldson". The case is ignored during the comparison unless the search operand is double-quoted. If no valid comparison operator is specified between two arguments, "IS" (identity) is assumed. The available logical operators are: +--------------------------------------------------------------------+ | ^ NOT | | & AND BUT | | | / OR | | | | Figure 14. Logical (boolean) operators | +--------------------------------------------------------------------+ Finally, keywords and operators can be "factorized" when the same comparison is to be applied to a given keyword and a series of compa- rands. For example, you might enter: Search * where sender contains ('CS Dept' and (Jack or Phil)) This is internally expanded to: SEARCH * WHERE sender CONTAINS 'CS Dept' AND - (sender CONTAINS Jack OR sender CONTAINS Phil) Please note that the expression must always be enclosed in paren- thesis, even if it is a simple one: Search * where sender contains (Joe or Morris) This stems from the fact that comparison operators have a higher priority than logical (boolean) ones. +--------------------------------------------------------------------+ | WHERE Sender is "Arthur Dent" - | | and Subject does not contain tea | | | | WITH Refcode 8467272 and Location Roubaix | | | | WITH (QTY > 100 | PRICE > 1000) & MAT = COPPER | | | | Where Sender is (Atiaran@Land or Elena@Land) - | | and Subject contains ('Be true' but not Ur-Lord) | | | | Figure 15. Sample WHERE clauses | +--------------------------------------------------------------------+ Search rules specification Finally, you must specify what is to be searched inside the document. If you do not want anything to be sought at all (e.g. if you are only selecting known items from the database), you can specify an asterisk as a placeholder to waive the search. Otherwise you must specify a mathematical expression where arguments are search strings, possibly bound by logical operators (see Figure 14for a comprehensive list). The default operator is AND, so that a search for "INTERPRET STEM PROBLEM" will select all entries where "INTERPRET", "STEM" and "PROBLEM" can be found (not necessarily in the same line). +--------------------------------------------------------------------+ | Search * | | | | Search 'I/O' Error | | | | Select Interpret (performance or tips - | | but not (bug or question)) | | | | Figure 16. Sample document-search clauses | +--------------------------------------------------------------------+ Reserved words and quoting When to quote strings Keyword names and search arguments need not be quoted, unless: o They are formed of more than one word (search arguments only). o They contain leading or trailing blanks (search arguments only). o Their name matches one of the "reserved keywords" of the LISTSERV database system, and appears in a context where it can be mistaken for such. The "reserved keywords" are: FROM, IN, SINCE, TO, UNTIL, WHERE, WITH. o They contain a parenthesis, logical operator or comparison oper- ator symbol. More generally, you should quote any string that contains one of the following characters: ( ) < > = | & ^ / Any non-quoted word will be stripped of leading and trailing blanks and converted to uppercase before the search. Single-quoted strings Strings quoted in single-quotes (') are converted to upper case and cause case to be ignored during the search. That is, they behave in the same manner as un-quoted strings as far as the search algorithm is concerned. As a rule of thumb, any string can be single-quoted if desired, even if it does not have to. Single quotes must be doubled inside single-quoted strings, but double quotes should not: Search '"T''amo, ripetilo, si caro accento' in OPERA Double-quoted strings Strings quoted in double-quotes (") are not converted to upper case. They result in a case-sensitive search, which means that you should never double-quote a string unless you want case to be respected during the search. Double quotes must be doubled inside double-quoted strings, but single quotes should not: Search """T'amo, ripetilo, si caro accento" in OPERA ********************* * The INDEX command * ********************* If you have understood everything in the previous chapter, you will have no problem with this one, nor with the following. The SEARCH command is by far the most complex database command you will have to use, and you will find INDEX and PRINT to be quite simplistic as compared to it. Introduction ------------ The INDEX command can be used to display a list of all the "hits" that your last search resulted in. For each such "hit", a single line of information will be displayed. This information will depend on the type of the database you were searching of course, but it will typi- cally contain the following columns: o Item number and possibly name of the database, although the latter is often suppressed to make room for more useful information. o Date and time the item was created or last modified (the exact meaning depends on the database type). o Number of records or, more generally speaking, some indication on the size of the document. o Any other information relevant to the type of database you are searching. To each database type is associated a unique, predefined index format. + + You will learn later how to define your own "formats" to produce + customized index displays. This knowledge is not necessary to perform + normal searches on the database. + Syntax ------ The syntax of the INDEX command is simply INDEX with no argument (minimum abbreviation is "I"). You will learn later that this is a lie, and that you can specify a flurry of horrendous parameters on the INDEX command, but for now you are better off without this knowledge. So, let's the syntax be simply INDEX for now. Examples -------- Since the index format is the same for all databases of the same type, we will simply give an example of "Notebook" index, and another example for a fancy database type of "Opera" which we will use exten- sively in the following chapters. +--------------------------------------------------------------------+ | > Select * IN EARNTECH where Sender contains HEARN | | --> Database EARNTECH, 6 hits. | | | | > Index | | Item # Date Time Recs Subject | | ------ ---- ---- ---- ------- | | 000013 87/06/20 13:04 13 Re: Prior 1 files from mailers | | 000046 87/07/09 11:32 10 phone number change | | 000079 87/07/28 15:30 17 Re: HW/SW at nodes | | 000172 87/10/07 10:02 16 portugal meeting ? | | 000173 87/10/12 08:58 16 again errors in xmailer names | | 000187 87/10/15 10:06 18 SNA - VTAM help needed. | | | | Figure 17. Sample INDEX for a "Notebook" database | +--------------------------------------------------------------------+ We will assume that the "Opera" database type is designed to process opera libretti, and defines the following keywords and document portions: AUTHOR The name of the composer (keyword). FROM The name of the opera from which the tune is extracted (keyword). SINGERS The name of the singers in "my own official preferred best record" (keyword). NAME The "name" generally given to the tune (keyword). TUNE A few lines out of the text, which are supposed to help the reader remember what it is exactly (portion). CHARACTERS The name of the character(s) singing the tune (portion). We could then have an index looking like this: +--------------------------------------------------------------------+ | > Select felice IN OPERA.250- where Singer is Bergonzi | | --> Database OPERA, 2 hits. | | | | > Index | | Item # From Singer Recs Name | | ------ ---- ------ ---- ---- | | 000254 Rigoletto Bergonzi 27 La donna e mobile | | 000259 La Traviata Bergonzi 34 Di quell'amor che palpito | | | | Figure 18. Sample INDEX for a (fancy) Opera database | +--------------------------------------------------------------------+ ********************* * The PRINT command * ********************* Tutorial -------- The PRINT command (minimum abbreviation "P") can be used to display the contents of documents which have been previously selected. Where INDEX displayed only one line of information, PRINT will typically display one or more pages. You should therefore be careful to not PRINT documents that you are not interested in, as this may generate a lot of unwanted output. Furthermore, if you are accessing the database in interactive mode from a remote node, you will not be allowed to generate more than 30 lines of interactive output per command. If you attempt to print too much information with a single command, your command will be termi- nated and only 30 lines will be shown. You may request larger output by means of the SENDBACK command, which will be explained later on. In any case, you should understand that interactive database access is expensive in network and computer resources, and that it has been allowed only to let you decide which are the documents you are inter- ested in. Once you know exactly what you need, you can request a copy to be sent to you as a file and terminate the interactive session. The PRINT command will, by default, apply to any and all selected documents. That is, it is assumed that everything you have selected is of interest to you. It is also assumed that, by default, you want the entire contents of the selected documents printed out (which will usually be the case anyway). You can, however, request only specific document portions to be printed. This is done by listing the name of the "portions" after the PRINT command, as in: Print Characters Tune A separator will be printed between each document portion (see exam- ples below). You can also request to print only specific documents among the ones that have been selected. To do this, simply specify the document numbers after the PRINT keyword. If you also wish to specify portion names, these must appear before the document numbers and may optionally be followed by an OF keyword: Print Tune OF 127 232 841 For your convenience, it has also been made possible to PRINT the contents of keywords associated with the selected documents. This will cause each keyword value to be displayed as if it were a one-line document portion. The output therefore takes three lines per entry (separator, keyword value and one blank line), and the INDEX command should obviously be preferred. However, there may be cases where you want to display the value of a particular keyword in a particular entry, which was not listed in the index because there is usually not enough room to display all keywords in a single 80-characters line: Print Author OF 254 + + You could also get this information through the use of the "LIST" + command, which will be explained later on. However, this command + would produce an output similar to the index, with one line per + selected entry. This would most likely be longer than the PRINT + output if you were interested in a few entries only. + Finally, you can indicate several print specifications on a single PRINT command by separating them with commas: Print Author of 254, Tune of 220 226-230 Examples -------- In the examples that will follow, we will assume that the following search has been executed prior to issuing the PRINT command: +--------------------------------------------------------------------+ | > Select fleur or diletto or belleza IN OPERA.100-150 - | | > where singer is not Domingo | | --> Database OPERA, 5 hits. | | | | > Index | | Item # From Singer Recs Name | | ------ ---- ------ ---- ---- | | 000103 Carmen Carreras 41 La fleur que tu m'avais jete| | 000118 Don Giovanni Sciepi 23 La ci darem la mano | | 000121 Rigoletto Fisher-Die+ 37 Veglia o donna | | 000127 Aida Moll 25 Mortal, diletto ai Numi | | 000142 La Traviata Bergonzi 52 Brindisi | | | | Figure 19. Search/Index upon which the sample PRINT commands are | | based | +--------------------------------------------------------------------+ + + The meaning of the "+" sign appearing after the truncated "Fisher- + Dieskau" will be explained later. + Let's first see what happens when you print document portions. To avoid making the example too long, we will print only the "Tune" portion which is not very large. +--------------------------------------------------------------------+ | > Print Tune | | >>> Item number 103, dated 87/10/18 13:09:30 -- TUNE | | La fleur que tu m'avais jetee | | Dans ma prison m'etait restee | | | | >>> Item number 118, dated 87/10/18 16:29:12 -- TUNE | | La ci darem la mano, | | La mi dirai di si... | | Vedi non e lontano, | | Partiam ben mio da qui. | | | | >>> Item number 121, dated 87/08/15 09:23:45 -- TUNE | | Ah, veglia o donna questo fior | | Che a te puro confidai | | | | >>> Item number 127, dated 87/02/12 19:26:02 -- TUNE | | Mortal, diletto ai Numi | | A te fidate son d'Egitto le sorti | | | | >>> Item number 142, dated 87/11/29 13:52:10 -- TUNE | | Libiamo, libiamo nell' lieti calici | | Che la belleza infiora | | | | Figure 20. Sample PRINT of a document portion | +--------------------------------------------------------------------+ The name of the singer of "Veglia o donna" has been truncated in the INDEX output, and you would like to see it in full, as well as the name of the character singing the tune. You also don't remember who is the author of "Carmen", and you would like it printed. Finally, you would like to print the whole of tune number 127. +--------------------------------------------------------------------+ | > Print singer character of 121, author of 103, all of 127 | | >>> Item number 121, dated 87/08/15 09:23:45 -- SINGER | | Fisher-Dieskau, Dietrich | | | | >>> Item number 121, dated 87/08/15 09:23:45 -- CHARACTERS | | Rigoletto | | | | >>> Item number 103, dated 87/10/18 13:09:30 -- AUTHOR | | Georges Bizet | | | | >>> Item number 127, dated 87/02/12 19:26:02 -- ALL | | Ramfis (a Radames) | | | | Mortal, diletto ai Numi | | A te fidate son d'Egitto le sorti | | Il sacro brando dal Dio temprato | | Per tua man diventi ai nemici | | Terror, Folgore, Morte! | | | | Figure 21. Sample PRINT of both document keywords and portions | +--------------------------------------------------------------------+ Syntax ------ This section will describe the exact syntax of the PRINT command. You may skip it if you are not interested in details. The syntax of the PRINT command is: +--------------------------------------------------------------------+ | Print <,print-spec2<,...>> | | | | Each "print-spec" | | has the following form: | | >> >> | | | | Figure 22. Syntax of the PRINT command | +--------------------------------------------------------------------+ Each "part" must be a valid "document portion" or "keyword" for the database against which the PRINT command is being issued. "ALL" is the default and is always valid, regardless of the database type. You may optionally specify "ranges" (xxxx, xxxx-yyyy, xxxx- or -yyyy) to further restrict the list of database items to be displayed. However, this does not allow you to display a document that has not been previ- ously selected by means of a SEARCH command. Finally, you may place more than one set of print specifications on the command line, provided you separate them with commas. ************************ * The SENDBACK command * ************************ The SENDBACK command allows you to request large database output to be sent back to you as a file. It is valid only when accessing the data- base in interactive mode. In batch mode, output will always be returned as a file. The syntax of the SENDBACK command is very simple: you just put the word SENDBACK before the command whose output you want returned in a file: +--------------------------------------------------------------------+ | > SENDBACK Print all | | File "DATABASE OUTPUT" has been sent to you in Netdata format. | | | | Figure 23. Sample SENDBACK command | +--------------------------------------------------------------------+ After you have ordered the desired output, you can terminate your interactive session and wait for the file to arrive. You can of course enter additional commands in interactive mode if you so desire. ********************** * The FORMAT command * ********************** The next two chapters will deal with the definition of "index formats", which allow you to view indexes different from the default ones. You may wish to skip them if you are not interested in details. You should now know enough about the database commands to be able to perform most kind of searches. Introduction ------------ The FORMAT command lets you define a named index format. An "index format" is a series of specifications which define one or more columns of output. For each column, you must specify the name of the keyword which is to be listed, along with some information like the number of characters (width) of the column, whether it must be left-justified, right-justified or centered, and optionally a title for the column. The default title is the (upper case) name of the keyword associated with the column. Please note that the FORMAT command itself does not cause any informa- tion to be displayed. This is done by the LIST command, which will be explained in the next chapter. Syntax ------ The exact syntax of the FORMAT command (minimum abbreviation "F") is: +--------------------------------------------------------------------+ | Format fmtname<:> fmtspec1 > | | | | Each "fmtspec" defines a column in the output chart, using the | | following syntax: | | | | fieldname<(<,end>)><. cols> <"heading"> | | | | Figure 24. Syntax of the FORMAT command | +--------------------------------------------------------------------+ This will create a column in the table, under the specified heading, where a substring (start,end) of the specified keyword (fieldname) will be displayed. The default values for start and end are 1 and 255, respectively. The width of the column is controlled by the cols specification, which defaults to 12 columns. Finally, the data in the column will be justified according to the just specification, whilst the header is justified as indicated by hd-just. Both justification parameters default to L, but can be any of the following: +--------------------------------------------------------------------+ | L -- Left justification | | R -- Right justification | | C -- Centered | | R0 -- Right justification with leading zeroes (for numerics) | | | | Note that the R0 justification type is invalid for headings. | | | | Figure 25. Justification keywords for use with the FORMAT command | +--------------------------------------------------------------------+ If the data does not fit entirely in the column space you have provided ("cols" parameter), i.e. if characters other than blanks (or leading zeroes for numeric fields) had to be removed in order for the keyword substring to fit into the column, a "+" sign will be placed in the following inter-column gap. This merely informs you that more data is available, but could not be displayed with the format you specified. It is also possible to specify a previously defined format name in lieu of column definition. This will insert the complete format spec- ification that had been recorded when the format was defined. Each "fieldname" must be a valid "keyword" name for the database against which the format will be used. No verification is made during the definition of the format. In addition, some common keywords are provided irrespective of the database type: +--------------------------------------------------------------------+ | DATABASE -- Name of the database (1-8 characters) | | DATE -- Date of the database entry (yy/mm/dd) | | TIME -- Time of the database entry (hh:mm:ss) | | #RECS -- Number of records in the database entry | | # -- Reference number of the database entry | | | | Figure 26. Reserved keywords for use with the FORMAT command | +--------------------------------------------------------------------+ Example ------- We will assume that you want to create a new index format for the "Opera" database, because you are not satisfied with the default one. You would want to see the name of the character singing the tune, not the "Singer" name which is what the person who keyed-in the database entries considers to be the best singer for the tune -- something which you might not agree about and don't want to see in the index. You also want to reduces the "Recs" column to 2 characters, since tunes are rarely longer than 99 lines. Finally, you want to change the heading of the "For" column to say "Excerpted from". The first thing to do is to prepare the "Item #" column, which you don't want to change. The "keyword" associated with the item number is #, the contents are numeric, must be right-justified with leading zeroes and there are 6 characters. Just to practise, we will change the header to "Nb" instead of "Item #", and will cause it to be centered in the column. The resulting specification is: #.6R0 "Nb"c For the second column, we only want to change the title. There were 13 characters displayed, left-justified (which is the default) and the title must now be "Excerpted from". Since the title is 14 characters long, we will make the field 14 characters long too: From.14 "Excerpted from" The "Recs" column must be shortened to 2 characters. The title must therefore be shortened too, and we will use "Sz". This is numeric data, so we want it right-justified but without leading zeroes: #Recs.2R "Sz" Now we want to create a new column for the character name. This could be up to, say, 12 characters, left-justified and with a title of "Character": Characters.12 "Character" Finally, we must prepare the "Name" column. We want it to take all the remaining room in the output display. The simplest way to do this is to assign it a length of 80, and let LISTSERV truncate the resulting output lines to 80 by itself: Name.80 "Name of the tune" Assuming we want to call this new format "MYINDEX", we would use the following command: +--------------------------------------------------------------------+ | > Format MYINDEX: #.6R0 "Item #" - | | > From.14 "Excerpted from" #Recs.2R "Sz" - | | > Characters.12 "Character" - | | > Name.80 "Name of the tune" | | | | > Select fleur or diletto or belleza IN OPERA.100-150 - | | > where singer is not Domingo | | --> Database OPERA, 5 hits. | | | | > Index MYINDEX | | Nb Excerpted from Sz Name of the tune | | -- -------------- -- ---------------- | | 000103 Carmen 41 La fleur que tu m'avais jetee | | 000118 Don Giovanni 23 La ci darem la mano | | 000121 Rigoletto 37 Veglia o donna | | 000127 Aida 25 Mortal, diletto ai Numi | | 000142 La Traviata 52 Brindisi | | | | Figure 27. Sample FORMAT command | +--------------------------------------------------------------------+ For your information, the default format provided for "Notebook" data- bases is the following: +--------------------------------------------------------------------+ | > Format INDEX: #.6R0 "Item #" Date.8L "Date"c - | | > Time(1,5).5 "Time" #Recs.4R "Recs" - | | > #.0 "" Subject.80 "Subject" | | | | Figure 28. Default format provided for "Notebook" databases | +--------------------------------------------------------------------+ ******************** * The LIST command * ******************** Introduction ------------ The purpose of the LIST command is to allow you to produce output indexes similar to those obtained through the use of the INDEX command, but with different fields. The syntax of the LIST command is a bit complex, but there is a very simple form when you are using a "format" which you have previously defined: List format-name The minimum abbreviation for the LIST command is "L". + + In fact, the INDEX command is nothing but a synonym for the LIST + command. The default "format" is the index provided with the database + type definition. + Syntax and examples ------------------- In fact, the syntax of the LIST command is very similar to that of the FORMAT command. It allows you to specify column definitions directly if you so desire, without having to define an explicit format. That is, if you do not plan to re-use a given format in the same session, you can specify it directly on the LIST command. For example, let's assume that you have selected a few tunes out of the "Opera" database, displayed an index, and that you realize that you also want to display the name of the author of the opera. You could do this with the following command: List Author "Author" However, this would produce only a list of authors, without anything else. What you want is whatever was on the previous index, plus the "Author" field. To do this, you can simply append INDEX to your LIST command, to cause the default index specifications to be appended to yours. ---------------------------------------------------------------------- > List Author.15 "Author" Index Author Item # From Singer Recs Name ------ ------ ---- ------ ---- ---- Georges Bizet 000103 Carmen Carreras 41 La fleur que + W.A. Mozart 000118 Don Giovanni Sciepi 23 La ci darem l+ Giuseppe Verdi 000121 Rigoletto Fisher-Die+ 37 Veglia o donn+ Vincenzo Bellin+000143 Norma Callas 25 Casta Diva W.A. Mozart 000187 Le Nozze di F+Fisher-Die+ 12 Se vuol balla+ G. Rossini 000203 Il Barbiere d+Kiri te Ka+ 38 Una voce poco+ Figure 29. Sample LIST command ---------------------------------------------------------------------- Note the plus sign after "La fleur que". The full title is "La fleur que tu m'avais jetee", and without the plus sign you might not have realized that some words were missing since the last displayed word was not truncated. ********************** * Advanced functions * ********************** This chapter contains information about "advanced" database functions. You should not read it until you have acquired a good knowledge of the basic database functions. You might in fact never need to use the functions described herein, althought database application programmers would certainly need to. Syntax of the DATABASE SEARCH command ------------------------------------- When performing database searchs in "batch" mode, you submit a job to LISTSERV containing a DATABASE SEARCH command and an inline dataset containing the search commands to be executed. The name of this dataset is specified through the use of a DD= keyword. There are, however, some additional keywords that may be specified on the DATA- BASE SEARCH command. The ECHO keyword The ECHO= keyword lets you choose whether you want search commands echoed in the job output or not. Echoed commands are preceded by a "greater than" sign in the job output. The default is to echo all commands, but this may be overriden by specifying an ECHO=NO keyword on the DATABASE SEARCH command: Database Search DD=Rules Echo=No Specifying a value other than YES or NO causes the search to be aborted with an error. The CPULIM keyword The amount of CPU time that a database search job may use is, by default, limited to 60 seconds. The amount of work that can be done in this time depends on the CPU model of course. You can raise or lower this limit by means of the CPULIM= keyword, whose exact syntax is: CPULIM=ss The search job is aborted (with a partial output being returned to you) if this limit is exceeded. The OUTLIM keyword The number of output lines that can be generated by database search jobs has been limited, by default, to 2000 lines. This is to avoid generating huge output because of an error in the search criteria (for example, an "AND" clause being inadvertently replaced with an "OR"). This default value may be overriden through the use of an OUTLIM=nnnn keyword. If the maximum number of output lines is exceeded, the job is imme- diately aborted and its output is returned to you. Limiting the output of a given command -------------------------------------- General description There might be cases where you want to limit the output of a partic- ular database command. For example, you might want to display the beginning of an index or document which you know to be quite large. This can be achieved by specifying an OUTLIM=nnnn keyword at the end of your command. This keyword can appear anywhere after the last quoted string in the command. This restriction allows you to perform a search on a string containing "OUTLIM=" by just quoting it, without changing the the maximum number of output lines. The keyword may not appear before the command verb itself. +--------------------------------------------------------------------+ | Examples of valid use of the "OUTLIM=" keyword: | | | | Print body of 220 outlim=30 | | Print outlim=100 all of 340-362 | | List Sender.17L "Sender"c Index Outlim= 20 | | List Sender.17L "Sender"c OUTLIM=20 index | | List Sender.17L "Sender outlim=10"c index | | | | Examples of invalid use of the "OUTLIM=" keyword: | | | | OUTLIM=20 Print all | | --> The command verb (PRINT) must appear before the keyword. | | | | List Sender.17L "Sender outlim=10"c index | | --> Here the keyword will be considered part of the heading. | | | | List Sender.17L outlim=20 "Sender"c Index | | --> There is a quoted string after the keyword. | | | | Figure 30. Example of valid and invalid use of the OUTLIM keyword | +--------------------------------------------------------------------+ If the maximum number of output lines is exceeded, an error message is issued (in the job output) and all output lines subsequently generated by the command are discarded. However, the job itself is not aborted, and following commands will continue to execute normally. +--------------------------------------------------------------------+ | > Select * in TEST-L | | --> Database TEST-L, 21 hits. | | | | > Print header Outlim=10 | | >>> Item number 1, dated 87/10/18 13:09:30 -- ALL | | Date: 29 Jun 1987, 10:00:00 | | Reply-To: Revised LISTSERV -- test list | | Sender: Revised LISTSERV -- test list | | Comments: Warning -- original Sender: tag was TEST-L@UGA | | From: PHIL@UIUCVMD | | Subject: this is a test looking for upcasing | | | | --> Maximum number of output lines reached, remaining command | | output flushed. | | | | > Search debugging | | --> No hit. | | | | Figure 31. Example of the use of the OUTLIM keyword | +--------------------------------------------------------------------+ Batch mode In batch mode, there are in fact two OUTLIM keywords: the "global" one, which is defined in the DATABASE SEARCH command, and the "local" one, which applies only to the individual command with which it has been specified. In batch mode, the default value of the local OUTLIM keyword is "infi- nite", i.e. it is internally equated to the value of the global keyword. The local OUTLIM keyword cannot be used to override the limitations set by the global keyword. That is, if the global keyword indicates that the job is to be aborted as soon as it generates more than 2000 lines, you cannot allow a particular command to generate 3000 lines of output by appending an "OUTLIM=3000" keyword to it. However, if the command would have generated 3000 lines and you had set a local "OUTLIM=100" keyword, only 100 lines would be actually generated and the job would not be aborted. Interactive mode In interactive mode, there is no global OUTLIM keyword. You can generate as much output as you want, provided that each command does not exceed the limit specified by the local OUTLIM keyword, which defaults to 2000 lines for "local" users (i.e. users on the same node as LISTSERV) and 30 lines for "remote" users. Furthermore, the OUTLIM keyword cannot be used to raise these limits. That is, a "remote" user cannot request 100 lines of output to be sent to him in interac- tive mode by specifying "OUTLIM=100" on the command line. The SENDBACK command should be used to retrieve large amounts of data. See "The SENDBACK command" for more information. Limiting the CPU time available to a given command -------------------------------------------------- General description There might be cases where you want to limit the amount of CPU time which may be spent on a particular command. This can be done by spec- ifying a CPULIM=ss keyword at the end of your command. It is similar to the "OUTLIM=" keyword in that it may appear anywhere after the last quoted string in your command, but may not appear before the command verb itself. If the CPU limit you have specified is exceeded, the job is aborted with an error message, and the (incomplete) job output is returned to you. Batch mode In batch mode, there are two CPULIM keywords, a global one and a local one. This is exactly the same as with the "OUTLIM" keyword. The default value of the local CPULIM keyword is "infinite". The local CPULIM keyword cannot be used to override the limitations set by the global keyword. Interactive mode In interactive mode, there is no global CPULIM keyword. You can use as much CPU time as you want, provided that each command does not exceed the limit specified by its local CPULIM keyword, which defaults to 20 seconds. However, this limit can be raised by just specifying a CPULIM keyword with the desired value. Performance considerations -------------------------- Search clauses LISTSERV tries to process search clauses in such a way as to minimize the number of I/O operations to the database. Each search operation is done in three steps: 1. The "date" clause is compared against the date of the database item, which is present in the database index and immediately available to LISTSERV at no extra cost. 2. The "WHERE" clause is executed against the document "keywords", which are available to LISTSERV at no I/O cost. 3. The document is read in from the database, and the actual search operation takes place. This step is waived if no search argument was specified, of course. It is therefore best to specify "date" or "WHERE" clauses which you know to be true for the items you are searching and false for "most" other items. This will not constitute extra work for LISTSERV, as one might first have thought. Response time You should always keep in mind that LISTSERV is not a multi-tasking system, and may only serve a single user at a time. There are two consequences to this: o Other users will have to wait for your database search to be completed before they can be served. You should therefore avoid sending database jobs that are likely to require more than one minute of wall-clock time to execute. It is always best to send several smaller jobs, especially since the output will be smaller and will therefore reach you faster. o You may have to wait for other users' commands to complete before you are served. This means you should be prepared to wait longer if you are addressing a server on a "hub" node with a lot of traffic. You should also remember that the "work unit" is the job when in batch mode, and the command when in interactive mode. If you have five different and independent search operations in a given job, they will all be executed sequentially before the next user is served. In interactive mode, LISTSERV will switch to the next user after each search command. ************************************* * Appendix A. The LISTSERV Library * ************************************* o User's guide . . . . . . . . . . . . . . . . . . . . (U01-001) o List Manager's guide . . . . . . . . . . . . . . . . (M01-002) o Installation guide . . . . . . . . . . . . . . . . . (S01-003) o Application Programmer's guide . . . . . . . . . . . (A01-004) o Maintenance guide . . . . . . . . . . . . . . . . . . (S01-005) o File Server Functions . . . . . . . . . . . . . . . . (U01-006) o Listserv-Punch Implementation . . . . . . . . . . . . (R01-007) o File Maintainer's guide . . . . . . . . . . . . . . . (M01-008) o BITNET-Oriented Presentation . . . . . . . . . . . . (P01-009) o Public Utilities Reference . . . . . . . . . . . . . (A01-010) o Licensed Utilities Reference . . . . . . . . . . . . (S01-011) --> o Database Functions . . . . . . . . . . . . . . . . . (U01-012) LISTSERV Document Numbers ------------------------- U 01 - 006 - 0 _ __ ___ _ | | | | Document Class -----------+ | | | | | | | | | | | | Product Number --------------+ | | | | | | | | Publication Number -------------------+ | | | | Revision Number ------------------------+ Document Class The Document Class indicates for which category of persons the publi- cation was written. The current classes are: A Documents intended for Application Programmers. These publica- tions are usually very technical. M Documents intended for Software Managers, i.e. operators, "list owners", "file maintainers", et al. P General Presentation documents intended for persons who do not have any particular knowledge in the product. These are gener- ally non-technical documents. R Reference documents defining protocols used by the product. These documents are very technical and are intended for people who have to write interfaces for the product or attempt to port it to an operating system or environment for which it was not originally written. S Documents intended for Systems Programmers, i.e. the persons responsible for the installation and operation of the product. U Documents intended for General Users. Product Number The Product Number is a unique number associated with the product to which the publication relates. Number 01 refers to LISTSERV, number 02 corresponds to the NETINFO sub-product, etc. Publication Number This is a unique number associated with the publication. Publication Numbers are assigned sequentially, disregarding the Document Class. There is a different set of Publication Numbers for each product. Revision Number This number is incremented at every release change in the publication. Fractional numbers indicate intermediate changes between two releases. <-