Date Command Dating for beginners.. (FreeBSD 6.2) |
Posted 22.06.2007 | Updated 22.06.2007 | Edited by Andy Mallett
PART I: Getting to know the DATE CommandNote the basic date command output contains SIX space-delimited fields. Day, Month, Date, Time (24hr), EST, Year. Yours will be different if you're not in the Eastern Standard Time zone..date Fri Jun 22 16:09:21 EST 2007 Notice also that the Day and Month are in abbreviated alpha format. Why am I mentioning this? Well sometimes it may be required for this information to be represented or output in a slightly different format, as for the previously mentioned text string searches in log files. The usual man page comes up with a wealth of switches just from typing man date and is always a good place to start. Isolating Date ElementsPull out the Abbreviated Alpha Day only, with..date +%a Fri Pull out the Full Alpha Day only, with.. date +%A Friday Pull out the Abbreviated Alpha Month only, with.. date +%b Jun Pull out the Numeric Month only, with.. date +%m 06 Pull out the Date only, with.. date +%d 22 Pull out the Abbreviated Year only, with.. date +%y 07 Pull out the Full Year only, with.. date +%Y 2007 Customising Date ElementsNow that we can isolate individual parts of the date, we can mix and match the output using custom delineators, like underscores, hyphens, spaces, etc. Check out the following examples..date +%d-%m-%Y 22-06-2007 date +%d" "%b" "%y 22 Jun 07 Time TravelThe date output can also be forwarded or put back in time using switches. Notes that this does not affect the system time, only the output of the command..Move the Date output back 20 days.. date -v -20d Sat Jun 2 19:59:09 UTC 2007 Move the Date output forward 20 days.. date -v +20d Thu Jul 12 19:59:09 UTC 2007 Note that none of these commands affect the time element of the output, just the date. PART II: Using the DATE Command in Shell ScriptsOK we're getting a bit specialist now and I'll come clean. This whole Date thing has been leading somewhere and that somewhere is the reason I wrote this stuff in the first place. In fact, it was during the experimentation for this article that (as anticipated) I found the answer to a problem I was having with an Apache log query script that I had written.Customising the Date output for string queriesI needed to use the date command to query an Apache file called error_log. The requirement was to be able to pull out any lines relating to a specified user, after a specific date (or umber of days from today).The error log conveniently outputs one error per line in the following format. The example is truncated as this behaviour makes for some very long lines at times.. more /usr/local/apache/logs/error_log
[Sun Nov 19 20:25:12 2006] [notice] Apache/1.3.17 (Unix) configured -- resuming normal.. [Mon Nov 20 19:05:57 2006] [error] [client 192.168.0.1] File does not exist: .. [Tue Nov 21 17:08:49 2006] [error] [client 192.168.0.1] Invalid method in request .. [Wed Nov 22 07:55:35 2006] [notice] SIGHUP received. Attempting to restart .. Notice that the lines begin with the date in the following format.. [Day Month Date Time(24hr) Year] This is very similar to the output of the standard date command. I used the cut command to get rid of the time field, as this was not relevant to the search.. more /usr/local/apache/logs/error_log | cut -f 1-3,5-20 -d " "
[Sun Nov 19 2006] [notice] Apache/1.3.17 (Unix) configured -- resuming normal.. [Mon Nov 20 2006] [error] [client 192.168.0.1] File does not exist: .. [Tue Nov 21 2006] [error] [client 192.168.0.1] Invalid method in request .. [Wed Nov 22 2006] [notice] SIGHUP received. Attempting to restart .. To get the standard date command to match up to this format, we can also use cut to remove extraneous fields like the Date and that superfluous EST entry.. date Fri Jun 22 20:30:54 UTC 2007 date | cut -f 1-3,6 -d " " Fri Jun 22 2007 Ah, a perfect match! Both the log file (modified output) and the modified date output contain the date string in the same format.. Day Month Date Year - four fields delimited by a single space. Seems straightforward enough.. Thus we can now query the log file for any line which contains a matching date string to the date we select. This formed the basis of an assignment given to some of our IT students. It involved creating a shell script which had to satisfy the following criteria..
cd /usr/local/apache/logs (assume we'll always be working in this directory from now on) more error_log | cut -f 1-3,5-20 -d " " | grep "Thu May 24 2007" ..giving an output of something like..
[Thu May 24 2007] [error] [client 192.168.0.1] File does not exist: .. [Thu May 24 2007] [error] [client 192.168.0.1] File does not exist: .. [Thu May 24 2007] [error] [client 192.168.0.20] File does not exist: .. [Thu May 24 2007] [error] [client 192.168.0.20] File does not exist: .. ..and then grep this for the required name (not shown in the truncated lines above). Don't forget that it is also necessary to pull out all relevant records dated between 30 days ago and today. More on that later.. Setting a Date variableInitially, I decided to calculate the search date using the method shown in Time Travel above. The script asks the user to enter a username to search for and then calculates the date to start searching from using the following notation...date -v -30d | cut -f 1-3,6 -d " " Thu May 24 2007 For the script to search for this date, the Date string is best defined as a variable.. searchdate=`(date -v -30d) | cut -f 1-3,6 -d " "` Note the expression sits inside backticks (shares the tilde ~ button on many keyboards), and the date calculation sits inside round brackets. The cut command is told to assume space separated values using the -d " " parameters. Let's see if it worked by echoing the search date variable.. echo $searchdate Thu May 24 2007 We can now use grep to pull out lines matching the two criteria of the required date and the required username.. more error_log | grep $searchdate | grep smithf There is a problem..At this point, I won't go into how I got the script to prompt for the username and date, put these into variables and then search for every day between the stated date and the present date. That can become the subject of a further article; we need to concentrate on the Date aspect of the thing. Take it from me, I got the script working smoothly and was very happy until a certain error started cropping up.The error was only occurring sporadically and was only obvious as I provided a summary at the end of the shell script, echoing various variables, partly for debugging purposes. Going back 30 days provided no problems.. echo $searchdate Thu May 24 2007 However when I went back a different number of days, mainly in order to capture different information, the variable sometimes stuffed up big time. This highlights how crucial it is to thoroughly test one's code, especially with unanticipated formats. searchdate=`(date -v -50d) | cut -f 1-3,6 -d " "` echo $searchdate Fri May UTC Going back 50 days instead of the original 30 was supposed to declare a date variable in the format of Fri May 4 2007. It became apparent that there was some inconsistency in how the date command was showing a space separated delimiter for its fields. In short, when the date was a single digit this seemed to throw out the cut command, which was delimiting using spaces. The problem is that there is no discernible difference between to the two outputs. Consider the following commands and try to see a difference in the space fields separating the date elements.. searchdate=`(date -v -30d)` echo $searchdate Thu May 24 16:12:01 UTC 2007 searchdate=`(date -v -50d)` echo $searchdate Fri May 4 16:12:07 UTC 2007 And yet, when you use exactly the same cut parameters on either command, the inconsistency emerges.. searchdate=`(date -v -30d) | cut -f 1-3,6 -d " "` echo $searchdate Thu May 24 2007 searchdate=`(date -v -50d) | cut -f 1-3,6 -d " "` echo $searchdate Fri May UTC Remember the delimiter can only be a single tab, a single space, a single comma, an alphanumeric character and some other special characters, as defined by the -d switch. This inconsistency occurred both under bash and csh. I couldn't find any references to this strange situation on the web and I don't think I'm missing anything obvious. Here is a solutionI had to define the date command so it could be modified backwards or forwards in time, with consistent results. Here's the eventual solution..date +%d" "%b" "%y 23 Jun 07 This gives us a consistent date variable by appending a 0 to single-digit dates.. date -v -30d +%d" "%b" "%y 24 May 07 date -v -50d +%d" "%b" "%y 04 May 07 Additionally the end result does not need to be cut as it appears as three fields, separated by commas; grep can handle this nicely. There is no need to include a Day field as this is really suplerfluous in a date search. The error_log and most other standard Unix log files also appropriately append a zero to single character dates..
[Tue Mar 07 12:25:00 2006] [error] [client 10.0.1.224] File does not exist: ... [Tue Mar 07 12:25:00 2006] [error] [client 10.0.1.224] File does not exist: ... [Tue Mar 07 12:25:01 2006] [error] [client 10.0.1.224] File does not exist: ... Watch out though, as some log files (such as samba logs) use a yyyy/mm/dd format (having said that, this is really the correct - and universal - order for a date layout - but damn those forward slashes..!). You always need to read the raw log file to check out the field layouts of dates and other data, before you can compose a meaningful query script. |
|