Analog 6.0: Other reports

This section deals with the non-time reports. There are quite a lot of commands which control these reports, although we've seen some of them already.

First, these reports have COLS commands, just like the time reports. (See the section on Time reports for how to use these commands.) But for these reports, several additional columns are available. Here is the full list of columns for the non-time reports

R: Number of requests
r: Percentage of the requests
S: Number of requests in the last 7 days
s: Percentage of the requests in the last 7 days
P: Number of page requests
p: Percentage of the page requests
Q: Number of page requests in the last 7 days
q: Percentage of the page requests in the last 7 days
B: Number of bytes transferred
b: Percentage of the bytes
C: Number of bytes transferred in the last 7 days
c: Percentage of the bytes in the last 7 days
d: Date of last access
D: Date and time of last access
e: Date of first access
E: Date and time of first access
N: The number of the item in the list

So, for example,

REQCOLS NRSD

counts the files in the Request Report, listing the number of requests for each, the number of requests for each in the last 7 days, and the time when each was last requested. The full list of COLS commands for non-time reports is HOSTCOLS, REDIRHOSTCOLS, FAILHOSTCOLS, ORGCOLS, DOMCOLS, REQCOLS, DIRCOLS, TYPECOLS, SIZECOLS, PROCTIMECOLS, REDIRCOLS, FAILCOLS, REFCOLS, REFSITECOLS, SEARCHQUERYCOLS, SEARCHWORDCOLS, INTSEARCHQUERYCOLS, INTSEARCHWORDCOLS, REDIRREFCOLS, FAILREFCOLS, BROWREPCOLS, BROWSUMCOLS, OSCOLS, VHOSTCOLS, REDIRVHOSTCOLS, FAILVHOSTCOLS, USERCOLS, REDIRUSERCOLS, FAILUSERCOLS and STATUSCOLS. Not every column is allowed in every report, but if you specify an illegal one, analog will warn you about it.

Next you need to know how use a SORTBY command to specify how the reports should be sorted. There are ten possible ways of sorting reports:

REQUESTS: total number of requests
REQUESTS7: requests within the last 7 days
PAGES: total requests for pages
PAGES7: requests for pages within the last 7 days
BYTES: total bytes transferred
BYTES7: bytes transferred within the last 7 days
FIRSTDATE: time of first request
DATE: time of most recent request
ALPHABETICAL: alphabetically
RANDOM: unsorted, sometimes useful for speed in very long reports

For example, the command

HOSTSORTBY ALPHABETICAL

will sort the Host Report alphabetically. The full list of SORTBY commands is HOSTSORTBY, REDIRHOSTSORTBY, FAILHOSTSORTBY, ORGSORTBY, DOMSORTBY, REQSORTBY, DIRSORTBY, TYPESORTBY, REDIRSORTBY, FAILSORTBY, REFSORTBY, REFSITESORTBY, SEARCHQUERYSORTBY, SEARCHWORDSORTBY, INTSEARCHQUERYSORTBY, INTSEARCHWORDSORTBY, REDIRREFSORTBY, FAILREFSORTBY, BROWREPSORTBY, BROWSUMSORTBY, OSSORTBY, VHOSTSORTBY, REDIRVHOSTSORTBY, FAILVHOSTSORTBY, USERSORTBY, REDIRUSERSORTBY, FAILUSERSORTBY and STATUSSORTBY. Again, not every sort method is possible in every report, but you'll be warned if you choose an illegal one.

There is one known bug concerned with SORTBY ALPHABETICAL. The report is sorted before any output alias is applied. This means that if an output alias has been specified for the report, then the report may appear not to be sorted correctly.

You can also specify a FLOOR for most reports, saying how much activity an item needs before it is listed on the report. (Other items will just be accumulated together in the "not listed" line at the bottom of the report.) There are lots of possible ways of specifying floors, which I'll list here, using the DOMFLOOR (Domain Report FLOOR) command as an example. Essentially each one consists of a number indicating the level of the floor, followed by a letter indicating the floor criterion.

DOMFLOOR 1000r       # all domains with at least 1000 requests
DOMFLOOR 100s        # at least 100 requests within the last 7 days
DOMFLOOR 1000p       # at least 1000 requests for pages
DOMFLOOR 100q        # at least 100 requests for pages within the last 7 days
DOMFLOOR 1000000b    # at least 1,000,000 bytes transferred
DOMFLOOR 1kb         # at least 1 kilobyte (1024 bytes)
DOMFLOOR 10.5Mc      # at least 10.5Mb within the last 7 days
DOMFLOOR 0.5%r       # 0.5% of the total requests in the Domain Report
                     # (ditto %s, %p etc.)
DOMFLOOR 0.5:r       # 0.5% of the maximum number of requests for any domain
                     # (ditto :s, :p etc.)
DOMFLOOR 970701d     # last access since 1st July 1997
DOMFLOOR 970701e     # first access since 1st July 1997
DOMFLOOR -00-01-00d  # last access in last month (see
                     # documentation on FROM and TO commands)
DOMFLOOR -100r       # domains with top 100 number of requests
                     # (ditto -100s, p, q, b, c, d, or e)

The full list of FLOOR commands is HOSTFLOOR, REDIRHOSTFLOOR, FAILHOSTFLOOR, DOMFLOOR, ORGFLOOR, REQFLOOR, DIRFLOOR, TYPEFLOOR, REDIRFLOOR, FAILFLOOR, REFFLOOR, REFSITEFLOOR, SEARCHQUERYFLOOR, SEARCHWORDFLOOR, INTSEARCHQUERYFLOOR, INTSEARCHWORDFLOOR, REDIRREFFLOOR, FAILREFFLOOR, BROWREPFLOOR, BROWSUMFLOOR, OSFLOOR, VHOSTFLOOR, REDIRVHOSTFLOOR, FAILVHOSTFLOOR, USERFLOOR, REDIRUSERFLOOR, FAILUSERFLOOR and STATUSFLOOR. Once again, not every floor method is legal for every report, but you'll be warned if you try and choose an illegal one.

I've already told you about how to turn each report on and off from the command line using its code letter. In fact, you can specify the SORTBY and the FLOOR in the same command. Take the example of the Referrer Report. If you follow the +f (to turn the report on) with a letter, it represents the sort method according to the following code:

r: REQUESTS
s: REQUESTS7
p: PAGES
q: PAGES7
b: BYTES
c: BYTES7
d: DATE
e: FIRSTDATE
a: ALPHABETICAL
x: RANDOM

You can then, or alternatively, use one of the above FLOOR formats to specify the floor. If you specify a SORTBY, you can also leave off the last letter of the floor, and analog will guess it according to the sort method: the floor will be the same as the sort method, or by requests if the sort method is ALPHABETICAL or RANDOM. Here are four examples:

+fp: means turn the Referrer Report on and sort it by page requests, but says nothing about the floor;
+f100s: means list all referrers with at least 100 requests in the last 7 days, but says nothing about the sort method;
+fb10000: means list all referrers with at least 10,000 bytes, sorted by bytes;
+fa-000101d: means list all referrers with accesses this year, sorted alphabetically.

Each of these reports can have a pie chart drawn at the top of it. The charts can be turned on and off, or plotted by a different criterion, using the CHART commands. For example,

REQCHART OFF

will stop you getting a pie chart on the Request Report, whereas

REQCHART ON

will turn it back on. You can also use the following arguments to plot the chart by a specific variable.

REQUESTS: total number of requests
REQUESTS7: requests within the last 7 days
PAGES: total requests for pages
PAGES7: requests for pages within the last 7 days
BYTES: total bytes transferred
BYTES7: bytes transferred within the last 7 days

By default, the chart will be plotted by the SORTBY if that's one of the above six variables, or else by the FLOOR if that's one of the six, or failing both of those by REQUESTS. Usually you just want to stick to the default, and analog will warn you if you could have used the SORTBY or FLOOR and requested something different.

The full list of CHART commands is HOSTCHART, REDIRHOSTCHART, FAILHOSTCHART, ORGCHART, DOMCHART, REQCHART, DIRCHART, TYPECHART, REDIRCHART, FAILCHART, REFCHART, REFSITECHART, SEARCHQUERYCHART, SEARCHWORDCHART, INTSEARCHQUERYCHART, INTSEARCHWORDCHART, REDIRREFCHART, FAILREFCHART, BROWREPCHART, BROWSUMCHART, OSCHART, VHOSTCHART, REDIRVHOSTCHART, FAILVHOSTCHART, USERCHART, REDIRUSERCHART, FAILUSERCHART, STATUSCHART, SIZECHART and PROCTIMECHART. Again, not every chart method is available for every report. You can also use

ALLCHART ON
ALLCHART OFF

to turn them all on or off simultaneously.

The pie charts are normally written to the same directory as the OUTFILE. But you can specify a different location using the CHARTDIR and LOCALCHARTDIR commands. If the OUTFILE is standard output, you must use these commands, or you will not get any pie charts. Also, if you are writing two output files to the same directory, you must use these commands, or one set of images will overwrite the other. And you also need to use these commands if you are using the form interface, but there are additional issues in that case.

You have to use both of the commands before they have any effect. The CHARTDIR is the location of the pie chart directory on your server, similar to the IMAGEDIR; it's used for putting in the <img> tag to include the image. The LOCALCHARTDIR is the location on your local disk; it's where the image is written to. For example, you might have

CHARTDIR /images/
LOCALCHARTDIR /usr/local/apache/htdocs/images/

to put the pie charts in locations like /usr/local/apache/htdocs/images/dom.png and link to them like <img src="/images/dom.png">. On Windows, you will need to put the LOCALCHARTDIR in quotes, to stop the backslash at the end of the line being interpreted to mean that the command continues on the next line. For example,

LOCALCHARTDIR "\webimages\analog\"

Actually, the CHARTDIR and LOCALCHARTDIR are just prefixes to the filename, so you can specify something like

CHARTDIR /stats/rep1
LOCALCHARTDIR /usr/local/htdocs/stats/rep1

to put the pie charts in locations like /usr/local/htdocs/stats/rep1dom.png and link to them like <img src="/stats/rep1dom.png">. Also the names can contain date codes in the same way as the LOGFILE and OUTFILE.

If you compiled your own version of analog, and you used your version of libgd instead of mine (see the Makefile for how to do this), then you can use the command

JPEGCHARTS ON

to make the charts in JPEG format instead of PNG. (See the PNGIMAGES command for how to make the bar charts in the time reports into gif's instead of png's.)

Here are the exact rules for which wedges are plotted in the pie chart. Up to ten wedges, plus "Other", are drawn, but wedges are only drawn if they are large enough. Also, wedges are only drawn if the item is listed in the main table for the report. And the whole chart will not be plotted if it would contain only one wedge.

If you want to plot more detail on the pie charts, for example Windows 98 & Windows NT instead of just Windows on the Operating System Report chart, see the next section.

You can list the time period covered by each report. This is off by default because it uses a lot of memory to calculate it, but if different reports cover different time periods (which can happen if your log format has changed at some point), it's useful to turn it on with the command

REPORTSPAN ON

There is also a command called REPORTSPANTHRESHOLD (which can be abbreviated RSTHRESH). This says that each report span should only be listed if it differs from the overall span of the whole report -- listed at the top of the page -- by at least this many minutes at one end. For example,

REPORTSPANTHRESHOLD 60

will only list a report span if that report starts at least an hour after the start of the logfile, or ends at least an hour before the end of the logfile. You can set

REPORTSPANTHRESHOLD 0

to make sure that the report span is listed on all the reports.

We've already seen some other commands affecting what was listed in the non-time reports. The output INCLUDE and EXCLUDE commands specified lines to omit from each report, and the output alias commands specified some aliasing to do on the names before they were listed. There were also LINKINCLUDE and LINKEXCLUDE commands to control what was linked to in the reports. You might want to have another look at these paragraphs.

There's one other command which affects the links in the Request Report, Redirection Report and Failure Report. The command BASEURL prepends an additional string to the URLs in the target of the link. For example, after the command

BASEURL http://www.statslab.cam.ac.uk

/~sret1/ will be linked to http://www.statslab.cam.ac.uk/~sret1/, not just to /~sret1/. This is very useful if you want to display the statistics on a different server from the server they refer to. If you want the file to be listed as http://www.statslab.cam.ac.uk/~sret1/, rather than just to be linked to that address, you need to use the second argument to the LOGFILE command instead.

In the next section, we'll look at commands for generating hierarchical reports, which are closely related to the commands in this section.

Go to the analog home page.

Stephen Turner
19 December 2004

Need help with analog? Use the analog-help mailing list.

[ Top | Up | Prev | Next | Map | Index ]