Webalizer with extended reports feature

May 19th, 2009

Now you can generate variety of reports using Webalizer with its custom report generation module.
Using this you can generate RSS feed, sitemaps, html pages for top urls, or low performing urls, or HTML page showing your top viewed wallpapers or least viewed wallpapers or any kind of special report for your analysis in format that you want and configure.

Using the different filter keywords you can specify exactly what you want to see in the report and in a format you love because the report generation is templatized. You specify the the structure and content of Reports through some keywords in a config file and Webalizer with this custom report release can generate the reports for you.

You should to read the README file first to start generating custom reports
Download webalizer with customer report source code

The changes has been done in the latest webalizer 2.21-02 source code available

 

Tags: ,

Techtalk, Webalizer 1 Comment »

Webalizer patch to generate top url RSS feed and to fix the extra host id field in access log of some hosting server like 1and1.com

May 5th, 2009

This patch is obsolete now. Check the report generation patch.

The webalizer patch has the following fixes:

  1. Extends webalizer to generate top url RSS feed.
  2. For 1and1.com server - skip host ID found in access log to correctly process the logs
  3. Increase referrer buffer size to eliminate truncation of referrer field

Download the patch

1. Extend webalizer to generate top url RSS feed

Now you can generate RSS feed for top urls with this patch. Through different config settings (mentioned below) in your webalizer config file, you can narrow down the urls you want to consider for the RSS feed for top urls.

You can generate as many RSS feed, each with different specification. For each RSS feed you need to provide following information in config file:

  1. RssFileName: RSS feed file name e.g topurl.xml
  2. RssTopUrlCount: Number of urls in RSS feed
  3. RssUrlRootFolders: Specify parent folders, the urls under which to be considered for RSS feed. Folders are separated by semicolon. e.g: /forum;/blog;
  4. RssIncludeUrlExts: Extensions of urls to be included for RSS feed. Specify noext if you want urls without extensions. e.g: *.htm;*.php;noext
  5. RssExcludeUrls: Specify parent folders, the urls under which to be excluded for RSS feed. e.g. /images;/photo
  6. RssEndDateAndDuration: Takes the access log date range in <Endate>,<duration> format. With this you can specify any kind of range. Please note it’s not start date, it’s end date followed by duration.For example if you want to generate top url from 1st May 2009 to 10th May 2009 then you specify 05/10/2009,10.Dynamic range: If you want to generate top url for last seven days whenever you run webalizer then it should be: today,7Specify none if you want to consider access logs of any date
  7. RssHeaderFile: Specify the path for the file that contains the header of RSS feed. It should contain the data that you want to have before the first item node of RSS feed. Click here for sample header file.
  8. RssFooter: Specify the footer that you want to have after the last item node of RSS feed. E.g. </channel></rss>

Example

The following config settings will generate top url RSS feed /root/rssfeed/topurl.xml, with max 10 urls that are under the /forum or /blog folder, and will include only htm, php or url with no extensions, excluding urls under /images folder.

RssFileName /root/rssfeed/topurl.xml
RssTopUrlCount 10
RssUrlRootFolders /forum;/blog
RssIncludeUrlExts *.htm;*.php;noext
RssExcludeUrls /images;
RssEndDateAndDuration none
RssHeaderFile /root/webalizer/rsstheader.txt
RssFooter </channel></rss>

Optional settings

The RSS feed needs title and description of the url, for which it downloads the url to parse title and description from it. You can specify the download timeout period in milliseconds using RssUrlDownloadTimeout config setting.

E.g

RssUrlDownloadTimeout 1000

Above setting will cause download to time out if it does no happen in 1000 milliseconds.

2. For 1and1.com server - skip host ID found in access log to correctly process the logs

1and1.com has a modified version of apache access_log file, with an added data field ( domain-name of your domain ). Hence webalizer works up to the field transfer size and then breaks due to the added NON-default field in the log file.

The very same patch also can be used for any other host or access_log format with any INSERTED data-field between any 2 fields of your default access_log.

Just look at the modularized build of the parser.c file and compare the sequence of data fields with the actual log format of your host and insert the skip-patch according to your location of the added data-field !!

3. Referrer buffer size increased to eliminate truncation of referrer field

The referrer buffer size increase to eliminate the warning message “Warning: Truncating oversized referrer field”

Download the patch

To apply the patch you need to have the webalizer-2.21-02 version. To apply the patch just run the following command in your webalizer 2.21-02 folder.

> patch -p1 -i webalizer-rssfeed.patch

Build note:

  1. To use the RSS feed feature you need configure webalizer with –enable-rssfeed option
    > ./configure --enable-rssfeed
  2. To fix extra host id issue in access log, configure webalizer –enable-skiphostid option
    >./configure --enable-skiphostid
 
Techtalk, Webalizer No comments »