Geeklog Usage Stats Plugin [v1.7.0]
by Andy Maloney [imol00@gmail.com]
09 Apr 2006
Table of Contents
Description
The Geeklog Usage Stats [GUS] plugin collects statistics on who visits your site, what browser and operating system
they are using, which pages they view, and which links they are clicking to get there. It allows the administrator
to browse these stats through a series of tables, getting right down into the data. It is based on the
venerable stats plugin v1.3
by Tom Willett and John Hughes. Some of this documentation is based on their work as well. I was using
the stats plugin for a long time and eventually decided it needed some cleanup and
updating to work with the latest version of geeklog. Tom mentioned he wasn't going to continue supporting it, so I
thought I'd make a few changes myself and release them. I figured this was going to be a quick week's work - wrong!
As I got into it, I found I wanted to fix this and... oh that while I'm here, and optimise this
a bit... oh and tweak that...
Eventually I decided to do a major overhaul.
Differences Between GUS and 'Stats'
I made the following changes to the database:
- moved user agents to their own table
- use the SQL date and time formats
- removed redundant fields
I also changed the name of the plugin from 'stats' to 'gus', eliminated the
'name the plugin whatever you want' functionality, put all the documentation in this readme,
added the ability to import stats from the stats plugin, and changed the installation/upgrade mechanism.
I probably did some other things here and there too...
The changes to the database will improve performance and require less space. I don't have any
real statistics on this, but from the feedback I got from the testers, it reduces databases to less than
half their size and is noticeably faster when browsing the stats.
Since the initial changes to the stats plugin, many other features and tweaks have been added to GUS.
Developed Using These Tools
- Geeklog 1.3.11 & Geeklog 1.4
- PHP 5.0.4 [works under 4.3 or greater]
- MySQL 4.1.16 [works in 4.0.x or greater - some users have reported that it works under 3.23]
- FireFox 1.5.0.1
- MacOS X 10.4.6
If you find GUS useful for monitoring and managing your site, please consider making a donation to support development.
How about the price of a pizza or even just a beer? Check out the Things You Can Do section.
Version History
- v1.7.0 [09 Apr 2006]
- [fix] fix paging when register_globals is off
- [fix] fix several other issues when register_globals is off [thanks Randall Hand]
- [fix] removes temporary tables created if MySQL TEMPORARY is not used
- [new] if you are using phpbbbridge, GUS will display [most] topic and forum names
- [new] if you are using easyfile, GUS will display file names for downloads
- [new] made day.php more concise [based on suggestions by Adrian Crook]
- [new] added config option for hiding referrers in day.php [suggested and financed by Adrian Crook]
- controlled by the $_GUS_CONF['hide_in_day_summary'] option in config.php
- set an array of referrers which should be hidden when displaying day.php
- the data is still collected, it is just hidden in the day summary
- [new] added a similar config option for hiding referrers in the Who's Online block
- controlled by the $_GUS_CONF['wo_hide_referrers'] option in config.php
v1.6.0 [29 Jan 2006]
- [fix] save on an SQL query if there is no referrer
- [fix] eliminate some warnings and a few minor buglets
- [new] now installs and works with register_globals off
- [new] changed the browser stats to make them more useful by breaking it down by browser and then version
- [new] optionally show bots by name [Google, Yahoo!, Ask Jeeves, Inktomi, and MSN Bot] in the Who's Online block instead of showing them as Guests [based on code by ByteEnabled]
- controlled by the $_GUS_CONF['wo_show_bots'] option in config.php
v1.5.0 [13 Oct 2005]
- [security fix] fixed a problem which might allow non-authorised users to view some stats pages
- [fix] if you had a file outside the document root which included lib-common.php, the path would not be correct when the stats were entered in the database [introduced in 1.4]
- [fix] if your $CONF['site_url'] was 'http://foo.com' and someone arrived via 'http://www.foo.com', then 'foo.com' would show up as a referrer in the Who's Online block
- [fix] sorting columns on today's stats was broken [reported by Adrian Crook]
- [new] added new config option $_GUS_CONF['allow_ignore_anonymous'] to optimise a bit if you do not need the ability to ignore the user 'Anonymous'
- [new] added filtering of referrers [suggested and financed by Adrian Crook]
- [new] added some more navigation options to some pages [required a change to the header.thtml template file, so if you've changed yours, look at this]
v1.4.0 [14 Sep 2005]
- [new] new interface for the admin page [uses Javascript]
- if Javascript is disabled it will still work - it's just not as nice and may produce quite a long page
- [new] added filtering of user agent strings [including wildcards]
- [new] added filtering of host names [including wildcards]
- [new] IP address, user, and page filtering now allow wildcards
- if you were using addresses such as '123.0.1.' to block a range, GUS will update your tables to the new format ['123.0.1.%'] automatically
- GUS uses MySQL's LIKE syntax for matching
- [new] changed host name lookup to be more flexible
- replaced config option $_GUS_CONF['use_nslookup'] with $_GUS_CONF['host_lookup'] which can be either 'host', 'nslookup', or 'gethostbyaddr'
- if your system is set up such that you cannot execute shell commands, you can use the built-in gethostbyaddr() function again
- [new] added new config option $_GUS_CONF['SQL_use_TEMPORARY']
- if you know your installation of MySQL does not allow the use of TEMPORARY tables, then set this to false for a slight optimisation
- if you don't know what I'm talking about, then just leave it and everything will still work fine
- [new] added new config options $_GUS_CONF['show_left_blocks'] and $_GUS_CONF['show_right_blocks'] to control the visibility of left and right blocks when viewing GUS pages
- [new] more info in tables showing 'Page/Title/URL'
- searches using the amz plugin will now show the keyword used in the search
- gallery 1.x album titles and photo names will now be shown
- user name for profile links will now be shown
- [new] removed extra path from page names
- if you had a site located at 'http://foo.bar.com/something/blat/', then all the pages in the database would start with 'something/blat/...'
- this is a waste of space [screen and db], so it's outta there
- this also speeds up the creation of the page columns by eliminating the need for some regular expression matches
- the updater will automatically fix the entries already in the database
- [fix] when importing from the stats plugin, the data would not be updated to the current GUS format correctly
- if you imported your stats before version 1.3.0, then it's not an issue
- if you did it after 1.3.0, then everything will still work, you just won't have the breakout of Windows versions for old data
v1.3.3 [29 Jul 2005]
- [fix] error on IP page when using MySQL 3.x
[reported by Ajay]
v1.3.2 [25 Jul 2005]
- [fix] query strings were being ignored [bug introduced in 1.3.0]
v1.3.1 [22 Jul 2005]
- [fix] installation failed on MySQL < 4.1.0
[reported by Curly]
- [new] update the user agents table
- reflect the better detection of Windows varients from 1.3.0 [i.e. change all occurrences of 'Windows NT 5.0' to 'Windows 2000', 'Windows NT 5.1' to 'Windows XP', etc.]
- recognise Windows Vista
- add WebTV, SymbianOS, and some bot detection
v1.3.0 [16 Jul 2005]
- [fix] index now has correct numbers for links followed
[reported by Ajay]
- [fix] 'robustified' the installation process
[thanks to windancer and Tino for working with me on this]
- [fix] use $_GUS_table_prefix when constructing TEMPORARY TABLE names and, if it fails to
properly create it, use a regular table [suggested by B. Judson]
- [fix] remove keyword TABLE from TRUNCATE statements so that they work on MySQL versions 3.23.28 to 3.23.32
[see MySQL manual]
- [fix] don't allow blanks to be entered for IPs, names, and pages on the admin page
- [new] added new gTLDs to the country page
- [new] link in the user block now includes a link to today's stats
- [new] treat 404 page specially - now show what page the user was trying to access
[inspired by machinari's question about 404s]
- this only works if you use a 404 page called 404.php which includes lib-common.php, such as the GL-supplied 404.php
- it changes the way 404 pages are added to the database, so previous 404 pages won't have the additional info
- [new] host name lookup is controlled by two new config options $_GUS_CONF['use_nslookup'] and $_GUS_CONF['host_lookup_timeout']
- Windows users will probably have to set $_GUS_CONF['use_nslookup'] to 'true' [I don't run any web servers on Windows, so I can't verify this]
- [new] add links to whois and netcraft on the IP page
- [new] show list of user agents associated with an IP on the IP page
- [new] added config option to show only a count of registered users instead of their names
[suggested by Martinez, based on the built-in Who's Online block]
- [new] remove config options $_GUS_sub and $_GUS_Whois
- [new] changed some configuration option names - if you've customised yours, you will have to change it
- [new] more specific detection of Windows versions [thanks to Mark van der Heijden]
- [new] translations
- Swedish [thanks to Markus Berg]
- Dutch [thanks to Mark van der Heijden]
v1.2.0 [08 May 2005]
- [fix] should now install properly on Geeklog 1.3.12cvs
[reported by ldfoo]
- [fix] import now works on PHP < 4.2.0
[reported by Ajay]
- [fix] fixed typo preventing cleanup in one case when the install failed
[reported by ldfoo]
- [fix] various HTML cleanups and fixes
- [new] clicking an IP address now takes you to a page where you can:
- lookup the IP using WHOIS
- turn data collection on and off for the IP
- you can ban the IP if you have the ban plugin installed
- see the list of hostnames associated with the IP
- [new] the tables now use CSS
- note that the themes have changed significantly, so if you've made changes
you should look at them carefully
- GUS will look in the <public_html>/gus/css directory for a file named {theme}.css and use it
if it exists. If not, it will use default.css.
- if you have created a CSS file for one of the common themes,
send it to me and I will
include it in future versions of GUS
v1.1.0 [14 Apr 2005]
- [security fix] fixed a potential javascript injection through the referrer URL
Note: The format of the referrers has been changed to fix this, so all data from 1.0 is invalid
and will have to be removed and re-imported [use the remove data capability on the admin page].
- fixed a problem with URLs not being decoded properly when looking at page views
- fixed a problem with importing user agents containing single quotes
- added proper upgrade functionality
v1.0.0 [10 Apr 2005]
Thanks!
- Tom Willett and John Hughes for writing the original stats plugin
- Brian Bevel for help testing and the slick icons [www.deliciousironing.com]
- Rick Robinson, Richard Rodriguez, Georg Kowallek, Joseph Anthony, and Tom Willett for help testing the pre-release versions of GUS 1.0.0
- All the others listed in the release notes for reporting problems or making suggestions
Installing The Plugin
- Backup your Geeklog database because the GUS plugin adds and modifies tables.
You can do this with the built in database backup facility in the Admin menu.
- Uncompress the GUS plugin archive while in the <geeklog_dir>/plugins directory.
The archive will create a directory called gus.
- In your <public_html> directory, create a directory called gus. Under your
<public_html>/admin/plugins/ directory create a directory called gus.
- Change to your <geeklog_dir>/plugins/gus/ directory.
- Copy the files in the admin directory to the admin/plugins/gus/ directory you created in step 3.
- Copy the files and directories in the public_html directory to the public_html/gus/
directory you created in step 3.
- Go to the plugin administration page. The gus plugin
should show up with a link to install it. Click it. The install page will tell you if the install
was successful or not. If not, examine the Geeklog system errorlog in <geeklog_dir>/logs to see
what the problem was. Fix it and re-install.
- The install page will offer you a link to the import page if it finds a suitable stats plugin to import from.
If you choose not to do this right away, a link will appear on the admin page.
Configuring The Plugin
- In the Admin block you will find a link to the GUS admin page where you may set up ignored
IP addresses, pages, and users. By default the user and IP of the user who installed the plugin is
entered in the ignore list and statistics capture is turned on. The admin page lets you enter additional
users and IPs for which usage information will not be collected. It also allows you to enable and disable
statics capture.
- Set up security. Upon install, only the root users have access to statistics admin and viewing.
You can delegate control for either of these functions through the user and group editors.
- Review the privacy policy. The installation also provides a privacy policy page.
This is controlled by the $_GUS_priv variable in the config.php file. If your
site already has a privacy policy available to your site users, I would encourage you to review it and
see that it is worded so that your site users know what information is being collected about them. If you
allowed anonymous access, a link to the visitor statistics viewer is also added.
- There are several variables in the config.php file which control the display of information. These are documented in the file itself.
- As of version 1.2, GUS uses CSS when displaying the tables. It only includes the CSS file when you are viewing a
GUS page. It first looks in <public_html>/gus/css for a file called {theme}.css - where {theme} is the name of the
current theme. If it doesn't find it, it will use the default.css file included with the plugin. If you have created a
CSS file for one of the common themes, send it to me
and I will include it in future versions of GUS.
Importing Data From Stats
GUS will give you the option to import your data and ignored lists from the stats plugin version 1.3.
You may do this either when you are installing GUS, or from the admin page. If you are going to do this,
I recommend doing it as soon as you install GUS because if there are problems with the import, it may be
difficult to recover.
Note that GUS does not use any of the tables from the stats plugin after the data has been imported,
so it is safe to remove the stats plugin and all its tables.
Upgrading The Plugin
Upgrading GUS is very similar to installation:
- Backup your Geeklog database because upgrading may add or modify database tables.
You can do this with the built in database backup facility in the Admin menu.
- Disable GUS using the admin interface's plugin editor [Admin->Plugins->gus]
- Make a backup of your config.php so you can update the new one with your settings.
- Uncompress the GUS plugin and copy the new files over the old ones as if it were a new installation.
- Enable GUS using the admin interface [Admin->Plugins->gus]
- Go back to the plugin editor for GUS [Admin->Plugins->gus] and click the 'Update' button.
Removing The Plugin
- Because the plugin modifies several database tables, it should always be removed through the interface provided.
Log in to your Geeklog as a root user and go to the plugin administration page. Click the 'gus'
link and then click the 'Delete' button. It will ask you to confirm it. Confirm it.
- Delete the three plugin directories created in the install process:
- <geeklog-dir>plugins/gus/
- <public_html>/gus/
- <admin>/plugins/gus/
- There is no step three.
Things You Can Do
- Check my PHP & SQL and tell me if there are better ways to do any of this.
[email me]
- Let me know if you use it or have any suggestions.
[email me]
- Link to my site [http://imol.gotdns.com] somewhere on yours.
- Support development by making a donation:
or by using one of the following links to shop at amazon:
License
The GUS plugin is licensed under the GPL even though I don't really understand it and all its implications.
Why do we let legalese control the world?