WebLog: weblog.pl

"Fossies" - the Fresh Open Source Software Archive

Member "weblog_files/weblog.pl" (23 Aug 2007, 155484 Bytes) of package /linux/www/old/weblog_files.zip:

As a special service "Fossies" has tried to format the requested source page into HTML format using (guessed) Perl source code syntax highlighting (style: standard) with prefixed line numbers and code folding option. Alternatively you can here view or download the uninterpreted source code file. For more information about "weblog.pl" see the Fossies "Dox" file reference documentation.

A hint: This file contains one or more very long lines, so maybe it is better readable using the pure text view mode that shows the contents as wrapped lines within the browser window.

1 ############################################ 2 ## ## 3 ## WebLog ## 4 ## by Darryl Burgdorf ## 5 ## (e-mail burgdorf@awsd.com) ## 6 ## ## 7 ## version: 2.54 ## 8 ## license modified: 4/13/06 ## 9 ## last modified: 4/1/06 ## 10 ## copyright (c) 2006 ## 11 ## ## 12 ## latest version is available from ## 13 ## http://awsd.com/scripts/ ## 14 ## ## 15 ############################################ 16 17 # COPYRIGHT NOTICE: 18 # 19 # Copyright 2006 Darryl C. Burgdorf. 20 # Copyright 2007 Stefan Lemcke 21 # 22 # This program is free software. You can redistribute it and/or 23 # modify it under the terms of either: 24 # 25 # a) the GNU General Public License as published by the Free Software 26 # Foundation, either version 1 or (at your option) any later version, 27 # 28 # or 29 # 30 # b) the "Artistic License" which comes with this program. 31 # 32 # You should have received a copy of the Artistic License with this 33 # module, in the file artistic.txt. If you did not, I'll be glad to 34 # provide one. 35 # 36 # You should have received a copy of the GNU General Public License 37 # along with this program. If you did not, write to the Free Software 38 # Foundation, Inc., 59 Temple Place, Suite 330, Boston MA 02111-1307. 39 # 40 # This program is distributed "as is" and without warranty of any 41 # kind, either express or implied. (Some states do not allow the 42 # limitation or exclusion of liability for incidental or consequential 43 # damages, so this notice may not apply to you.) In no event shall 44 # the liability of Darryl C. Burgdorf and/or Affordable Web Space 45 # Design for any damages, losses and/or causes of action exceed the 46 # total amount paid by the user for this software. 47 48 # VERSION HISTORY: 49 # 50 # 2.54 04/01/06 Subdivided "Spider/Robot" category in agents list 51 # Added Firefox and Safari to agents list 52 # Added Navigator 8 to agents list 53 # Stripped "gibberish" characters from keywords 54 # Set agents list to log 403 accesses as spiders 55 # Set refs list and sessions count to ignore 403s 56 # Improved "access summary" info to isolate 403s 57 # 2.53 12/31/02 Added entrance/exit page lists 58 # Split Navigator 6 & 7 in agents list 59 # Trapped for a few new "link checkers" 60 # Stripped HTML tags from keywords 61 # 2.52 08/28/02 Fixed parsing of Win 2000 vs. Win XP 62 # Updated top-level domains list 63 # $IncludeQuery now recognizes WebBBS "SEF" URLs 64 # Squashed small bug in EOM $time setting 65 # 2.51 09/15/01 Added "About" to keywords list 66 # Improved HotBot/Lycos differentiation 67 # Added Windows XP to platform breakdown 68 # Separated OpenVMS from "Unix/Linux" category 69 # Fixed reading of user IDs from MS log files 70 # Closed rare "div by 0" error in session counting 71 # 2.50 07/05/01 Updated spider/robot filter lists 72 # Eliminated reading of details report into memory 73 # Allowed details report to keep summaries longer 74 # Corrected "Navigator v5" to "Navigator v6" 75 # Allowed "full" lists to be put in separate report 76 # Lowered referrers/keywords "filter" thresholds 77 # Added "master list" of recognized search engines 78 # Eliminated use of "file list filter" with top 404 79 # Made "details filter" apply to error URLs 80 # Fixed $IncludeOnlyDomain handling of domain names 81 # Log file name wildcards now only match at end 82 # Allowed for "foreign" characters in keywords 83 # Squashed "top 404 to files" bug reading old report 84 # Fixed unclosed <BLOCKQUOTE> tag in details report 85 # 2.41 02/21/01 Replaced @Accesses & @Sessions with temp files 86 # Squashed minor bug in summary "spider count" 87 # 2.40 02/07/01 Added support for old MS-IIS log file format 88 # Allowed for MS extended logs with multiple headers 89 # DoW & hourly reports now average, not cumulative 90 # Allowed for "summary only" details report 91 # Added "spider count" to details summary 92 # Eliminated $TotalReset flag 93 # Set script to automatically use best available DBM 94 # Allowed keywords report without refs report 95 # Added $TopNKeywords option 96 # Replaced $RefsMinHits with $RefsFilterLists 97 # Added "Direct Hit" and "FAST" to keywords list 98 # Correctly separated Google and Yahoo 99 # Updated "Snap!" to NBCi 100 # Made other minor adjustments to keyword parsing 101 # 2.30 12/25/00 NOW REQUIRES PERL 5! 102 # Added Windows 2000 and CE to platform breakdown 103 # Removed offline browsers, etc. from spiders list 104 # Otherwise expanded agents and platform lists 105 # Added "master list" of recognized agents/platforms 106 # Made maintenance of full agents list optional 107 # Revised graph bars to use fewer image calls 108 # Added "Courier" font specifications to <PRE>s 109 # Trapped for invalid resolved domain names 110 # Empty keywords report no longer created w/o refs 111 # Allowed inclusion/exclusion of unresolvable IPs 112 # Made various minor tweaks 113 # 2.20 11/08/99 Added support for MS extended log format 114 # Added errors to "details" report 115 # "Trimmed" IPs/domains to improve visitor count 116 # (This also makes the IP database smaller!) 117 # Added agent/platform info to details report 118 # "Singularized" all agent/platform list entries 119 # Fixed recognition of newer versions of Opera 120 # Split robots and harvesters in platform list 121 # Trapped for obviously corrupted agent names 122 # Updated and greatly expanded keywords parsing 123 # Squashed bug in "Top Referers" calculation 124 # Improved EOM File creation criteria 125 # Added $TotalReset flag 126 # 2.12 05/13/99 Corrected minor bug in $DefaultPageName def 127 # "Single-Quoted" several variable definitions 128 # Allowed for possibility of spaces in user IDs 129 # Added "Non-Standard Domain" to top domains list 130 # Added $bodyspec, $headerfile and $footerfile 131 # Fixed bug preventing re-gzipping of files 132 # 2.11 04/20/99 Put IP resolution back into main processing loop 133 # 2.10 03/30/99 Replaced resolved IPs list with DBM file 134 # Removed IP resolution from main processing loop 135 # Eliminated copying of @Accesses array 136 # Improved "memory management" of disabled options 137 # Added stripping of "/../" from file names 138 # Expanded "Verbose" notifications 139 # Took care of a few minor "format bug" fixes 140 # Added "E-Mail Harvesters" to agents list 141 # Corrected counting of some AOL as MSIE 142 # Corrected "Netscape" to "Netscape Navigator" 143 # Improved parsing of "ambiguous" Win platforms 144 # Added MSN to search engines in keywords list 145 # Removed final spaces from keywords lists 146 # Allowed *all* ref domains to be "lower-cased" 147 # Various other minor tweaks 148 # 2.03 07/04/98 Added "top referers" list 149 # Added Windows 98 to platform list 150 # Allowed for better EOM processing 151 # Fixed handling of "mixed" type log files 152 # Added date to details report listings 153 # Improved second-level parsing of non-US domains 154 # Allowed for NT-style "\" file path delimiters 155 # 2.02 05/02/98 Trapped "run on" URLs 156 # Added $HourOffset configuration variable 157 # Added "$Verbose" flag 158 # 2.01 04/06/98 Fixed path bug in "wildcarded" file names 159 # Fixed bug in unzipping of multiple zipped files 160 # Fixed bug preventing print of "null" reports 161 # Fixed bug in hourly visit & page view counting 162 # Corrected multi-file reset of %agentcounter 163 # Added "Snap!" to keywords breakdown 164 # Separated "Excite" and "Netfind" in breakdown 165 # 2.00 03/09/98 Incorporated referer & agent logging 166 # Added "search keywords" logging 167 # Added resolution of IP numbers 168 # Added ability to "wildcard" log file name 169 # Added "page views" logging to main report 170 # Allowed user selection of info to be graphed 171 # Fixed bug causing graphs to flake with large logs 172 # Simplified configuration of file locations 173 # Eliminated need to "pre-create" report files 174 # Eliminated separate "country codes" file 175 # Allowed "autodetection" of log file type 176 # Set $NoSessions to disable *all* sessions logging 177 # Expanded list to include all HTTP/1.1 status codes 178 # Script now re-compresses gzip'd log files 179 # Made the usual collection of minor format tweaks 180 # 1.12 01/01/98 Added tracking of user IDs and associated domains 181 # Made complete file listing optional 182 # Cleaned up minor glitch in EOM listings 183 # Adjusted regex to allow single-digit log file dates 184 # 1.11 08/06/97 Fixed minor bug in $EndDate calculation 185 # 1.10 08/04/97 Finally eliminated "one day's log per day" limit 186 # Added "log only new" option 187 # Improved handling of various server error codes 188 # Removed unnecessary "summary" report 189 # Fixed minor bug in session counting 190 # 1.05 05/24/97 Added comma delineation to all numbers 191 # Added "record book" report 192 # Fixed new bug in "top" domain logging 193 # 1.04 05/03/97 Changed month/day logs from domains to sessions 194 # Added sessions to day-of-week log 195 # Allowed disabling of above sessions logging 196 # Allowed "details" log of only one day's accesses 197 # Enabled "top" domain logging even if domains off 198 # Added ability to log "second level" domains 199 # Fixed display bugs if no accesses on first day 200 # The typical selection of minor bug fixes 201 # 1.03 02/02/97 Added support for combined log files 202 # Added option to include file query information 203 # Corrected handling of zipped files 204 # Corrected bug in EOM report wrapping (Dec/Jan) 205 # 1.02 12/11/96 Added "running total" of user sessions 206 # 1.01 12/08/96 Added "user sessions" to details report 207 # Corrected day counting in details report 208 # 1.00 12/02/96 Initial "public" release 209 210 sub MainProg { 211 use Fcntl; 212 BEGIN { @AnyDBM_File::ISA = qw (DB_File GDBM_File SDBM_File ODBM_File NDBM_File) } 213 use AnyDBM_File; 214 if ($Verbose) { print "Log Analysis for $SystemName\n"; } 215 &Initialize; 216 if ($PrintFullAgentLists) { &PrintFullAgentLists; } 217 if ($LogFile =~ /\*/) { 218 if ($LogFile =~ /(.*)[\/\\]([^\/\\]+)/) { 219 $dir = $1; 220 $LogFile = $2; 221 } 222 else { 223 $dir = ""; 224 } 225 $LogFile =~ s/(\W)/\\$1/g; 226 $LogFile =~ s/\\\*/\.\*/g; 227 $LogFile =~ s/^/\^/g; 228 opendir (DIR,$dir) || die " Error: Unable to read log files directory\n"; 229 @files = readdir (DIR); 230 closedir (DIR); 231 foreach $file (@files) { 232 if ($file =~ $LogFile) { 233 if ($dir) { $fullfile = $dir."/".$file; } 234 else { $fullfile = $file; } 235 push (@LogFiles,$fullfile); 236 } 237 } 238 } 239 else { 240 push (@LogFiles,$LogFile); 241 } 242 foreach $LogFile (sort @LogFiles) { 243 $LogFile =~ s/.gz$//; 244 if (-e "$LogFile.gz") { 245 $ZipLock = 1; 246 system ("gunzip -q $LogFile"); 247 } 248 $OldFile = $ReportFile; 249 $time = time; 250 $RealDate = (localtime($time+($HourOffset*3600)))[3]; 251 $time = (stat("$LogFile"))[9]; 252 ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = 253 localtime($time+($HourOffset*3600)); 254 $year += 1900; 255 if ($EOMFile && (($mday==1) || ($RealDate < $mday))) { 256 $ReportFile = $EOMFile; 257 if ($mday == 1) { 258 ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = 259 localtime($time-86400+($HourOffset*3600)); 260 $year += 1900; 261 } 262 $sec = 59; 263 $min = 59; 264 $hour = 23; 265 $eom = $mon+1; 266 if (length($eom) == 1) { $eom = "0".$eom; } 267 $EOMDate="$year $eom $mday 23 59 59"; 268 &ResetVars; 269 &ReadOldFile; 270 &ReadLog; 271 if ($LogLineCount || ($EndDate gt $InitialEndDate)) { 272 unless ($NoSessions) { &GetSessions; } 273 if ($Verbose) { print " Generating EOM Report\n"; } 274 &PrintReport; 275 } 276 $ReportFile = $OldFile; 277 $EOMDate = ""; 278 $time = (stat("$LogFile"))[9]; 279 ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = 280 localtime($time+($HourOffset*3600)); 281 $year += 1900; 282 } 283 &ResetVars; 284 &ReadOldFile; 285 &ReadLog; 286 if ($LogLineCount || ($EndDate gt $InitialEndDate)) { 287 if ($AgentsFile) { &PrintAgentsReport; } 288 if ($RefsFile) { &PrintRefsReport; } 289 if ($KeywordsFile) { &PrintKeywordsReport; } 290 unless ($NoSessions) { &GetSessions; } 291 if ($Verbose) { print " Generating Main Report\n"; } 292 &PrintReport; 293 if ($DetailsFile) { &PrintHostDetailsReport; } 294 } 295 if ($ZipLock) { 296 system ("gzip $LogFile"); 297 $ZipLock = 0; 298 } 299 } 300 unlink "$FileDir/tempaccesses.txt"; 301 unlink "$FileDir/tempsessions.txt"; 302 if ($Verbose) { print "Report Complete\n"; } 303 } 304 305 sub Initialize { 306 $version = "2.54"; 307 @months=('Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec'); 308 %MonthToNumber= 309 ('Jan','01','Feb','02','Mar','03', 310 'Apr','04','May','05','Jun','06', 311 'Jul','07','Aug','08','Sep','09', 312 'Oct','10','Nov','11','Dec','12'); 313 %NumberToMonth= 314 ('01','Jan','02','Feb','03','Mar', 315 '04','Apr','05','May','06','Jun', 316 '07','Jul','08','Aug','09','Sep', 317 '10','Oct','11','Nov','12','Dec'); 318 %DoWCounter= 319 (1,'Sun',2,'Mon',3,'Tue',4,'Wed',5,'Thu',6,'Fri',7,'Sat'); 320 $DefaultPageName = '(index\.(((s|p)*html*|cgi))|welcome\.(((s|p)*html*|cgi)))'; 321 if ($NoSessions) { 322 $DetailsFile = ""; 323 if ($GraphBase eq "visits") { 324 $GraphBase = ""; 325 } 326 } 327 if (($DetailsDays<1) && ($DetailsSummaryDays<1)) { 328 $DetailsFile = ""; 329 } 330 if ($DetailsFile) { 331 if ($DetailsDays>36) { $DetailsDays=36; } 332 if ($DetailsSummaryOnly) { 333 if ($DetailsSummaryDays<1) { 334 $DetailsSummaryDays = $DetailsDays; 335 } 336 $DetailsDays = 0; 337 } 338 if ($DetailsDays == 0) { $DetailsSummaryOnly = 1; } 339 if ($DetailsSummaryDays < $DetailsDays) { 340 $DetailsSummaryDays = $DetailsDays; 341 } 342 if ($DetailsSummaryOnly) { 343 $HRFlag = 1; 344 $AccessDetailsReport = "Access Summary Report"; 345 } 346 else { 347 $AccessDetailsReport = "Access Details Report"; 348 } 349 } 350 if ($NoResolve) { $IPLog = ""; } 351 %RespCodes = ( 352 '100','Code 100 Continue', 353 '101','Code 101 Switching Protocols', 354 '200','Code 200 OK', 355 '201','Code 201 Created', 356 '202','Code 202 Accepted', 357 '203','Code 203 Non-Authoritative Information', 358 '204','Code 204 No Content', 359 '205','Code 205 Reset Content', 360 '206','Code 206 Partial Content', 361 '300','Code 300 Multiple Choices', 362 '301','Code 301 Moved Permanently', 363 '302','Code 302 Moved Temporarily', 364 '303','Code 303 See Other', 365 '304','Code 304 Not Modified', 366 '305','Code 305 Use Proxy', 367 '400','Code 400 Bad Request', 368 '401','Code 401 Unauthorized', 369 '402','Code 402 Payment Required', 370 '403','Code 403 Forbidden', 371 '404','Code 404 Not Found', 372 '405','Code 405 Method Not Allowed', 373 '406','Code 406 Not Acceptable', 374 '407','Code 407 Proxy Authentication Required', 375 '408','Code 408 Request Time-Out', 376 '409','Code 409 Conflict', 377 '410','Code 410 Gone', 378 '411','Code 411 Length Required', 379 '412','Code 412 Precondition Failed', 380 '413','Code 413 Request Entity Too Large', 381 '414','Code 414 Request-URI Too Large', 382 '415','Code 415 Unsupported Media Type', 383 '500','Code 500 Internal Server Error', 384 '501','Code 501 Not Implemented', 385 '502','Code 502 Bad Gateway', 386 '503','Code 503 Service Unavailable', 387 '504','Code 504 Gateway Time-Out', 388 '505','Code 505 HTTP Version Not Supported' 389 ); 390 %CountryCodes = ( 391 'ac','Ascension Island', 392 'ad','Andorra', 393 'ae','United Arab Emirates', 394 'af','Afghanistan', 395 'ag','Antigua and Barbuda', 396 'ai','Anguilla', 397 'al','Albania', 398 'am','Armenia', 399 'an','Netherlands Antilles', 400 'ao','Angola', 401 'aq','Antarctica', 402 'ar','Argentina', 403 'as','American Samoa', 404 'at','Austria', 405 'au','Australia', 406 'aw','Aruba', 407 'az','Azerbaijan', 408 'ba','Bosnia and Herzegovina', 409 'bb','Barbados', 410 'bd','Bangladesh', 411 'be','Belgium', 412 'bf','Burkina Faso', 413 'bg','Bulgaria', 414 'bh','Bahrain', 415 'bi','Burundi', 416 'bj','Benin', 417 'bm','Bermuda', 418 'bn','Brunei Darussalam', 419 'bo','Bolivia', 420 'br','Brazil', 421 'bs','Bahamas', 422 'bt','Bhutan', 423 'bv','Bouvet Island', 424 'bw','Botswana', 425 'by','Belarus', 426 'bz','Belize', 427 'ca','Canada', 428 'cc','Cocos (Keeling) Islands', 429 'cd','Congo (formerly Zaire)', 430 'cf','Central African Republic', 431 'cg','Congo', 432 'ch','Switzerland', 433 'ci','Cote dIvoire (Ivory Coast)', 434 'ck','Cook Islands', 435 'cl','Chile', 436 'cm','Cameroon', 437 'cn','China', 438 'co','Colombia', 439 'cr','Costa Rica', 440 'cs','Czechoslovakia (Former)', 441 'cu','Cuba', 442 'cv','Cape Verde', 443 'cx','Christmas Island', 444 'cy','Cyprus', 445 'cz','Czech Republic', 446 'de','Germany', 447 'dj','Djibouti', 448 'dk','Denmark', 449 'dm','Dominica', 450 'do','Dominican Republic', 451 'dz','Algeria', 452 'ec','Ecuador', 453 'ee','Estonia', 454 'eg','Egypt', 455 'eh','Western Sahara', 456 'er','Eritrea', 457 'es','Spain', 458 'et','Ethiopia', 459 'eu','Europe', 460 'fi','Finland', 461 'fj','Fiji', 462 'fk','Falkland Islands (Malvinas)', 463 'fm','Micronesia', 464 'fo','Faroe Islands', 465 'fr','France', 466 'fx','France (Metropolitan)', 467 'ga','Gabon', 468 'gb','Great Britain (UK)', 469 'gd','Grenada', 470 'ge','Georgia', 471 'gf','French Guiana', 472 'gg','British Channel Islands (Guernsey)', 473 'gh','Ghana', 474 'gi','Gibraltar', 475 'gl','Greenland', 476 'gm','Gambia', 477 'gn','Guinea', 478 'gp','Guadeloupe', 479 'gq','Equatorial Guinea', 480 'gr','Greece', 481 'gs','S. Georgia and S. Sandwich Islands', 482 'gt','Guatemala', 483 'gu','Guam', 484 'gw','Guinea-Bissau', 485 'gy','Guyana', 486 'hk','Hong Kong', 487 'hm','Heard and McDonald Islands', 488 'hn','Honduras', 489 'hr','Croatia (Hrvatska)', 490 'ht','Haiti', 491 'hu','Hungary', 492 'id','Indonesia', 493 'ie','Ireland', 494 'il','Israel', 495 'im','Isle of Man', 496 'in','India', 497 'io','British Indian Ocean Territory', 498 'iq','Iraq', 499 'ir','Iran', 500 'is','Iceland', 501 'it','Italy', 502 'je','British Channel Islands (Jersey)', 503 'jm','Jamaica', 504 'jo','Jordan', 505 'jp','Japan', 506 'ke','Kenya', 507 'kg','Kyrgyzstan', 508 'kh','Cambodia', 509 'ki','Kiribati', 510 'km','Comoros', 511 'kn','Saint Kitts and Nevis', 512 'kp','North Korea', 513 'kr','South Korea', 514 'kw','Kuwait', 515 'ky','Cayman Islands', 516 'kz','Kazakhstan', 517 'la','Laos', 518 'lb','Lebanon', 519 'lc','Saint Lucia', 520 'li','Liechtenstein', 521 'lk','Sri Lanka', 522 'lr','Liberia', 523 'ls','Lesotho', 524 'lt','Lithuania', 525 'lu','Luxembourg', 526 'lv','Latvia', 527 'ly','Libya', 528 'ma','Morocco', 529 'mc','Monaco', 530 'md','Moldova', 531 'mg','Madagascar', 532 'mh','Marshall Islands', 533 'mk','Macedonia', 534 'ml','Mali', 535 'mm','Myanmar', 536 'mn','Mongolia', 537 'mo','Macau', 538 'mp','Northern Mariana Islands', 539 'mq','Martinique', 540 'mr','Mauritania', 541 'ms','Montserrat', 542 'mt','Malta', 543 'mu','Mauritius', 544 'mv','Maldives', 545 'mw','Malawi', 546 'mx','Mexico', 547 'my','Malaysia', 548 'mz','Mozambique', 549 'na','Namibia', 550 'nc','New Caledonia', 551 'ne','Niger', 552 'nf','Norfolk Island', 553 'ng','Nigeria', 554 'ni','Nicaragua', 555 'nl','Netherlands', 556 'no','Norway', 557 'np','Nepal', 558 'nr','Nauru', 559 'nt','Neutral Zone', 560 'nu','Niue', 561 'nz','New Zealand (Aotearoa)', 562 'om','Oman', 563 'pa','Panama', 564 'pe','Peru', 565 'pf','French Polynesia', 566 'pg','Papua New Guinea', 567 'ph','Philippines', 568 'pk','Pakistan', 569 'pl','Poland', 570 'pm','St. Pierre and Miquelon', 571 'pn','Pitcairn', 572 'pr','Puerto Rico', 573 'ps','Palestinian Authority', 574 'pt','Portugal', 575 'pw','Palau', 576 'py','Paraguay', 577 'qa','Qatar', 578 're','Reunion', 579 'ro','Romania', 580 'ru','Russian Federation', 581 'rw','Rwanda', 582 'sa','Saudi Arabia', 583 'sb','Solomon Islands', 584 'sc','Seychelles', 585 'sd','Sudan', 586 'se','Sweden', 587 'sg','Singapore', 588 'sh','St. Helena', 589 'si','Slovenia', 590 'sj','Svalbard and Jan Mayen Islands', 591 'sk','Slovak Republic', 592 'sl','Sierra Leone', 593 'sm','San Marino', 594 'sn','Senegal', 595 'so','Somalia', 596 'sr','Suriname', 597 'st','Sao Tome and Principe', 598 'su','USSR (Former)', 599 'sv','El Salvador', 600 'sy','Syria', 601 'sz','Swaziland', 602 'tc','Turks and Caicos Islands', 603 'td','Chad', 604 'tf','French Southern Territories', 605 'tg','Togo', 606 'th','Thailand', 607 'tj','Tajikistan', 608 'tk','Tokelau', 609 'tm','Turkmenistan', 610 'tn','Tunisia', 611 'to','Tonga', 612 'tp','East Timor', 613 'tr','Turkey', 614 'tt','Trinidad and Tobago', 615 'tv','Tuvalu', 616 'tw','Taiwan', 617 'tz','Tanzania', 618 'ua','Ukraine', 619 'ug','Uganda', 620 'uk','United Kingdom', 621 'um','US Minor Outlying Islands', 622 'us','United States', 623 'uy','Uruguay', 624 'uz','Uzbekistan', 625 'va','Vatican City State (Holy See)', 626 'vc','Saint Vincent and the Grenadines', 627 've','Venezuela', 628 'vg','Virgin Islands (British)', 629 'vi','Virgin Islands (US)', 630 'vn','Viet Nam', 631 'vu','Vanuatu', 632 'wf','Wallis and Futuna Islands', 633 'ws','Samoa', 634 'ye','Yemen', 635 'yt','Mayotte', 636 'yu','Yugoslavia', 637 'za','South Africa', 638 'zm','Zambia', 639 'zr','Zaire', 640 'zw','Zimbabwe', 641 'aero','Dot-Aero', 642 'arpa','Old-Style Arpanet', 643 'biz','Dot-Biz', 644 'com','US Commercial', 645 'coop','Dot-Coop', 646 'edu','US Educational', 647 'gov','US Government', 648 'info','Dot-Info', 649 'int','International', 650 'mil','US Military', 651 'museum','Dot-Museum', 652 'nato','NATO Field', 653 'name','Dot-Name', 654 'net','US Network', 655 'org','US Organization', 656 'pro','Dot-Pro', 657 'xxx','Unresolved Domain', 658 'ooo','Unrecognized Domain' 659 ); 660 $harvester_list = 'bullseye|cherrypicker|crescent|emailcollector|emailsiphon|emailwolf|extractor|microsoft url|mozilla/3.mozilla/2.01|newt|nicerspro|webbandit|brutus'; 661 $download_list = 'da \d|dnload|download|fetch|flashget|ftp|getright|gozilla|jetcar|leach|leech'; 662 $linkchecker_list = 'analyze|check|internetseer|link|netmechanic|netmind|powermarks|redalert|tooter|validat|verif|walk|webhostingratings'; 663 $offline_list = 'avantgo|batch|copier|httrack|msiecrawler|msproxy|netattache|netscape-proxy|offline|spacebison|teleport|webcapture|webzip'; 664 $spider_list = 'aport|archive|ask jeeves|behold|borg|bot|catch|crawl|digger|elitesys|enfish|esense|euroseek|ferret|grab|griffon|gulliver|harvest|htdig|hubat|hunt|infoseek|java|leia|lwp-|lwp:|mantraagent|mapper|mata hari|mercator|netants|perl|quest|reader|reaper|roamer|rover|scooter|search|slurp|snatch|spider|spinne|spyder|sweep|t-h-u-n-d-e-r-s-t-o-n-e|ultraseek|url|utopy|webcollage|webster pro|webwhacker|wfarc|wget|whatuseek'; 665 } 666 667 sub ResetVars { 668 &date_to_count(($mon+1),$mday,($year-1900)); 669 $DayBreak = $perp_days; 670 $CurrDate="$mday $months[$mon] $year"; 671 $CurrMonth="$months[$mon]"; 672 $CurrYear="$year"; 673 $InitialEndDate="0000 00 00 00 00 00"; 674 $EndDate="0000 00 00 00 00 00"; 675 $FileEndDate="0000 00 00 00 00 00"; 676 $FileStartDate="9999 99 99 99 99 99"; 677 $AveragePages = 0; 678 $DailyTotalBytesCounter = 0; 679 $DailyTotalHitsCounter = 0; 680 $DailyTotalPViewsCounter = 0; 681 $DailyTotalVisitsCounter = 0; 682 $GoDaily = 0; 683 $GoMonthly = 0; 684 $HourlyTotalBytesCounter = 0; 685 $HourlyTotalHitsCounter = 0; 686 $HourlyTotalPViewsCounter = 0; 687 $HourlyTotalVisitsCounter = 0; 688 $LocalPercent = 0; 689 $MonthlyTotalBytesCounter = 0; 690 $MonthlyTotalHitsCounter = 0; 691 $MonthlyTotalPViewsCounter = 0; 692 $MonthlyTotalVisitsCounter = 0; 693 $OutsidePercent = 0; 694 $TodayBanned = 0; 695 $TodayBytes = 0; 696 $TodayErrors = 0; 697 $TodayHits = 0; 698 $TodayHosts = 0; 699 $TodayKB = 0; 700 $TodayLocal = 0; 701 $TodayMB = 0; 702 $TodayOutside = 0; 703 $TodayPagesErrors = 0; 704 $TodayPagesHits = 0; 705 $TodayPagesTotal = 0; 706 $TopDomain = ""; 707 $endhour = 0; 708 $logsegment = ""; 709 $refscounter = 0; 710 $sessionstime = 0; 711 $usersessions = 0; 712 %BytesDay = (); 713 %BytesFileCounter = (); 714 %BytesHour = (); 715 %BytesUserIDCounter = (); 716 %DayFilesCounter = (); 717 %DomainsBytesCounter = (); 718 %DomainsDay = (); 719 %DomainsFilesCounter = (); 720 %HitsFileCounter = (); 721 %HitsUserIDCounter = (); 722 %HourFilesCounter = (); 723 %LastAccessDomain = (); 724 %LastAccessFile = (); 725 %LastAccessUserID = (); 726 %MonthlyBytesCounter = (); 727 %MonthlyCounter = (); 728 %MonthlyFilesCounter = (); 729 %MonthlyPViewsCounter = (); 730 %MonthlySessionsCounter = (); 731 %PViewsDay = (); 732 %PViewsHour = (); 733 %Resolved = (); 734 %TargetCounter = (); 735 %TodayDomains = (); 736 %TodayHosts = (); 737 %TopDomainAccess = (); 738 %TopDomainBytesCounter = (); 739 %TopDomainFilesCounter = (); 740 %VisitsHour = (); 741 %WhichDayCounter = (); 742 %WhichDayFiles = (); 743 %WhichDayBytes = (); 744 %WhichDayDomains = (); 745 %WhichDayPViews = (); 746 %aboutcom = (); 747 %agentcounter = (); 748 %agentreport = (); 749 %altavista = (); 750 %askjeeves = (); 751 %cnet = (); 752 %combinedreport = (); 753 %dates = (); 754 %day_counts = (); 755 %dayusersessions = (); 756 %directhit = (); 757 %dogpile = (); 758 %excite = (); 759 %euroferret = (); 760 %euroseek = (); 761 %fast = (); 762 %firstpages = (); 763 %fnfBytesFileCounter = (); 764 %fnfHitsFileCounter = (); 765 %fnfLastAccessFile = (); 766 %google = (); 767 %goto = (); 768 %hostlist = (); 769 %hotbot = (); 770 %hourusersessions = (); 771 %infoseek = (); 772 %lastpages = (); 773 %looksmart = (); 774 %lycos = (); 775 %magellan = (); 776 %mamma = (); 777 %metacrawler = (); 778 %monthusersessions = (); 779 %msn = (); 780 %netfind = (); 781 %netscape = (); 782 %northernlight = (); 783 %othersearch = (); 784 %planetsearch = (); 785 %platformreport = (); 786 %prevday = (); 787 %previouspage = (); 788 %prevvisit = (); 789 %refdomain = (); 790 %savvysearch = (); 791 %snap = (); 792 %topkeywords = (); 793 %webcrawler = (); 794 %yahoo = (); 795 @files = (); 796 @hosts = (); 797 } 798 799 sub ReadOldFile { 800 if ($Verbose) { print " Reading Old Report File: $OldFile\n"; } 801 return unless (-e "$FileDir/$OldFile"); 802 open (OLD,"$FileDir/$OldFile") || die " Error Opening File: $OldFile\n"; 803 while (<OLD>) { 804 chomp; 805 unless ($endhour || $logsegment) { 806 ($endhour,$endminute,$endsec, 807 $endday,$endmonth,$endyear) = 808 m#^.*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d)#o; 809 if ($endhour) { 810 $EndDate= 811 "$endyear $MonthToNumber{$endmonth} $endday $endhour $endminute $endsec"; 812 $InitialEndDate = $EndDate; 813 } 814 } 815 if (/<A NAME="([^"]*)">/oi) { 816 $logsegment=$1; 817 if ($logsegment ne 'records') { <OLD>; <OLD>; <OLD>; } 818 next; 819 } 820 next unless ($logsegment); 821 if ($logsegment eq "monthly") { 822 ($Accesses,$Bytes,$Domains,$PViews,$Month) = 823 m#^\s*([\d,]+)\s+([\d,]+)\s+([\d,]+|-)\s+([\d,]+)\s+(\w.{7})#o; 824 $Accesses =~ s/,//g; 825 $Bytes =~ s/,//g; 826 $Domains =~ s/,//g; 827 $PViews =~ s/,//g; 828 $MonthlyFilesCounter{$Month}=$Accesses; 829 $MonthlyBytesCounter{$Month}=$Bytes; 830 $MonthlySessionsCounter{$Month}=$Domains; 831 $MonthlyPViewsCounter{$Month}=$PViews; 832 $MonthlyTotalBytesCounter+=$Bytes; 833 $MonthlyTotalHitsCounter+=$Accesses; 834 $MonthlyTotalVisitsCounter+=$Domains; 835 $MonthlyTotalPViewsCounter+=$PViews; 836 next; 837 } 838 elsif ($logsegment eq "daily") { 839 ($Accesses,$Bytes,$Domains,$PViews,$Day,$Month) = 840 m#^\s*([\d,]+)\s+([\d,]+)\s+([\d,]+|-)\s+([\d,]+)\s+(\d\d)\s+(\w\w\w)#o; 841 if ($Month) { 842 $Accesses =~ s/,//g; 843 $Bytes =~ s/,//g; 844 $Domains =~ s/,//g; 845 $PViews =~ s/,//g; 846 $Year=$CurrYear; 847 if (($Month =~ "(Nov|Dec)") && 848 ($CurrMonth =~ "(Jan|Feb)")) { 849 $Year=$Year-1; 850 } 851 $Today="$Year $MonthToNumber{$Month} $Day"; 852 $DayFilesCounter{$Today}=$Accesses; 853 $BytesDay{$Today}=$Bytes; 854 $DomainsDay{$Today}=$Domains; 855 $PViewsDay{$Today}=$PViews; 856 &date_to_count(int($MonthToNumber{$Month}),$Day,($Year-1900)); 857 if ($DayBreak-$perp_days < 36) { 858 $DailyTotalBytesCounter+=$Bytes; 859 $DailyTotalHitsCounter+=$Accesses; 860 $DailyTotalVisitsCounter+=$Domains; 861 $DailyTotalPViewsCounter+=$PViews; 862 } 863 $Today="$Today 00 00 00"; 864 $EndDate=$Today if ($Today gt $EndDate); 865 } 866 next; 867 } 868 elsif ($logsegment eq "houraverage") { 869 ($Accesses,$Bytes,$Sessions,$PViews,$Hour) = 870 m#^\s*([\d,]+)\s+([\d,]+)\s+([\d,]+|-)\s+([\d,]+)\s+(\d\d)#o; 871 $Accesses =~ s/,//g; 872 $Bytes =~ s/,//g; 873 $Sessions =~ s/,//g; 874 $PViews =~ s/,//g; 875 $HourFilesCounter{$Hour}=($Accesses*3); 876 $BytesHour{$Hour}=($Bytes*3); 877 unless ($NoSessions) { $VisitsHour{$Hour}=($Sessions*3); } 878 $PViewsHour{$Hour}=($PViews*3); 879 unless ($HourFilesCounter{$Hour} < 1) { $HourlyDayCounter{$Hour} = 3; } 880 next; 881 } 882 elsif ($logsegment eq "records") { 883 $line=$_; 884 if ($line =~ m#Most Hits#o) { 885 ($RecordHits,$RecordHitsDate) = 886 $line=~m#([\d,]*)\s$([^)]*)$#o; 887 $RecordHits =~ s/,//g; 888 } 889 elsif ($line =~ m#Most Bytes#o) { 890 ($RecordBytes,$RecordBytesDate) = 891 $line=~m#([\d,]*)\s$([^)]*)$#o; 892 $RecordBytes =~ s/,//g; 893 } 894 elsif ($line =~ m#Most Visits#o) { 895 ($RecordVisits,$RecordVisitsDate) = 896 $line=~m#([\d,]*)\s$([^)]*)$#o; 897 $RecordVisits =~ s/,//g; 898 } 899 elsif ($line =~ m#Most PViews#o) { 900 ($RecordPViews,$RecordPViewsDate) = 901 $line=~m#([\d,]*)\s$([^)]*)$#o; 902 $RecordPViews =~ s/,//g; 903 } 904 next; 905 } 906 elsif ($logsegment eq "files") { 907 ($Hour,$Minute,$Second,$Day,$Month,$Year, 908 $Accesses,$Bytes,$FileName)= 909 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\S.*)#o; 910 next unless ($Month =~ /$CurrMonth/); 911 next if ($FileName =~ /\/[^\/]*\/\.\.\//); 912 $Accesses =~ s/,//g; 913 $Bytes =~ s/,//g; 914 $HitsFileCounter{$FileName}=$Accesses; 915 $BytesFileCounter{$FileName}=$Bytes; 916 $LastAccessFile{$FileName}= 917 "$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 918 next; 919 } 920 elsif ($logsegment eq "404") { 921 ($Hour,$Minute,$Second,$Day,$Month,$Year, 922 $Accesses,$Bytes,$FileName)= 923 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\S.*)#o; 924 next unless ($Month =~ /$CurrMonth/); 925 next if ($FileName =~ /\/[^\/]*\/\.\.\//); 926 $Accesses =~ s/,//g; 927 $Bytes =~ s/,//g; 928 $fnfHitsFileCounter{$FileName}=$Accesses; 929 $fnfBytesFileCounter{$FileName}=$Bytes; 930 $fnfLastAccessFile{$FileName}="$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 931 next; 932 } 933 elsif ($logsegment eq "entrancepages") { 934 ($Accesses,$FileName)= m#^\s*([\d,]+)\s+(\S.*)#o; 935 $Accesses =~ s/,//g; 936 $firstpages{$FileName} = $Accesses; 937 next; 938 } 939 elsif ($logsegment eq "exitpages") { 940 ($Accesses,$FileName)= m#^\s*([\d,]+)\s+(\S.*)#o; 941 $Accesses =~ s/,//g; 942 $lastpages{$FileName} = $Accesses; 943 next; 944 } 945 elsif ($logsegment eq "userids") { 946 ($Hour,$Minute,$Second,$Day,$Month,$Year, 947 $Accesses,$Bytes,$UserID)= 948 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\S.*)#o; 949 next unless ($Month =~ /$CurrMonth/); 950 $Accesses =~ s/,//g; 951 $Bytes =~ s/,//g; 952 $HitsUserIDCounter{$UserID}=$Accesses; 953 $BytesUserIDCounter{$UserID}=$Bytes; 954 $LastAccessUserID{$UserID}= 955 "$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 956 next; 957 } 958 elsif ($logsegment eq "topleveldomain") { 959 ($Hour,$Minute,$Second,$Day,$Month,$Year, 960 $Accesses,$Bytes,$TopDomain)= 961 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\w+)\s+=\s+#o; 962 next unless ($Month =~ /$CurrMonth/); 963 $Accesses =~ s/,//g; 964 $Bytes =~ s/,//g; 965 $TopDomainFilesCounter{$TopDomain}=$Accesses; 966 $TopDomainBytesCounter{$TopDomain}=$Bytes; 967 $TopDomainAccess{$TopDomain}="$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 968 next; 969 } 970 elsif ($logsegment eq "domains") { 971 ($Hour,$Minute,$Second,$Day,$Month,$Year, 972 $Accesses,$Bytes,$Domain)= 973 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\S.*)#o; 974 next unless ($Month =~ /$CurrMonth/); 975 $Accesses =~ s/,//g; 976 $Bytes =~ s/,//g; 977 $DomainsFilesCounter{$Domain}=$Accesses; 978 $DomainsBytesCounter{$Domain}=$Bytes; 979 $LastAccessDomain{$Domain}="$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 980 next; 981 } 982 } 983 close (OLD); 984 if ($FullListFile) { 985 if ($Verbose) { print " Reading Old \"Full List\" File: $FullListFile\n"; } 986 return unless (-e "$FileDir/$FullListFile"); 987 open (OLD,"$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 988 while (<OLD>) { 989 chomp; 990 if (/<A NAME="([^"]*)">/oi) { 991 $logsegment=$1; 992 <OLD>; <OLD>; <OLD>; 993 next; 994 } 995 next unless ($logsegment); 996 if ($logsegment eq "files") { 997 ($Hour,$Minute,$Second,$Day,$Month,$Year, 998 $Accesses,$Bytes,$FileName)= 999 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\S.*)#o; 1000 next unless ($Month =~ /$CurrMonth/); 1001 next if ($FileName =~ /\/[^\/]*\/\.\.\//); 1002 $Accesses =~ s/,//g; 1003 $Bytes =~ s/,//g; 1004 $HitsFileCounter{$FileName}=$Accesses; 1005 $BytesFileCounter{$FileName}=$Bytes; 1006 $LastAccessFile{$FileName}= 1007 "$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 1008 next; 1009 } 1010 elsif ($logsegment eq "404") { 1011 ($Hour,$Minute,$Second,$Day,$Month,$Year, 1012 $Accesses,$Bytes,$FileName)= 1013 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\S.*)#o; 1014 next unless ($Month =~ /$CurrMonth/); 1015 next if ($FileName =~ /\/[^\/]*\/\.\.\//); 1016 $Accesses =~ s/,//g; 1017 $Bytes =~ s/,//g; 1018 $fnfHitsFileCounter{$FileName}=$Accesses; 1019 $fnfBytesFileCounter{$FileName}=$Bytes; 1020 $fnfLastAccessFile{$FileName}="$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 1021 next; 1022 } 1023 elsif ($logsegment eq "entrancepages") { 1024 ($Accesses,$FileName)= m#^\s*([\d,]+)\s+(\S.*)#o; 1025 $Accesses =~ s/,//g; 1026 $firstpages{$FileName} = $Accesses; 1027 next; 1028 } 1029 elsif ($logsegment eq "exitpages") { 1030 ($Accesses,$FileName)= m#^\s*([\d,]+)\s+(\S.*)#o; 1031 $Accesses =~ s/,//g; 1032 $lastpages{$FileName} = $Accesses; 1033 next; 1034 } 1035 elsif ($logsegment eq "userids") { 1036 ($Hour,$Minute,$Second,$Day,$Month,$Year, 1037 $Accesses,$Bytes,$UserID)= 1038 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\S.*)#o; 1039 next unless ($Month =~ /$CurrMonth/); 1040 $Accesses =~ s/,//g; 1041 $Bytes =~ s/,//g; 1042 $HitsUserIDCounter{$UserID}=$Accesses; 1043 $BytesUserIDCounter{$UserID}=$Bytes; 1044 $LastAccessUserID{$UserID}= 1045 "$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 1046 next; 1047 } 1048 elsif ($logsegment eq "domains") { 1049 ($Hour,$Minute,$Second,$Day,$Month,$Year, 1050 $Accesses,$Bytes,$Domain)= 1051 m#^\s*(\d\d):(\d\d):(\d\d) (\d\d) (\w\w\w)\w* (\d\d\d\d) \s*([\d,]+)\s+([\d,]+)\s+(\S.*)#o; 1052 next unless ($Month =~ /$CurrMonth/); 1053 $Accesses =~ s/,//g; 1054 $Bytes =~ s/,//g; 1055 $DomainsFilesCounter{$Domain}=$Accesses; 1056 $DomainsBytesCounter{$Domain}=$Bytes; 1057 $LastAccessDomain{$Domain}="$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 1058 next; 1059 } 1060 } 1061 close (OLD); 1062 } 1063 } 1064 1065 sub ReadLog { 1066 if ($Verbose) { print " Reading Log File: $LogFile\n"; } 1067 if ($IPLog) { 1068 if ($DBMType==1) { 1069 tie (%Resolved,'AnyDBM_File',"$IPLog",O_RDWR|O_CREAT,0666,$DB_HASH); 1070 } 1071 elsif ($DBMType==2) { 1072 dbmopen(%Resolved,"$IPLog",0666); 1073 } 1074 else { 1075 tie (%Resolved,'AnyDBM_File',"$IPLog",O_RDWR|O_CREAT,0666); 1076 } 1077 } 1078 open (LOGFILE,"$LogFile") || die " Error Opening File: $LogFile\n"; 1079 $LogLineTotal = 0; 1080 $LogLineCount = 0; 1081 $ResolutionCount = 0; 1082 $logtype = ""; 1083 open (TEMPACCESSES,">$FileDir/tempaccesses.txt"); 1084 while(<LOGFILE>) { 1085 $LogLineTotal++; 1086 chomp; 1087 s/\s+/ /go; 1088 next if m/format\=\%/i; 1089 if (m/^#fields:/i) { 1090 $logtype = "microsoft"; 1091 @logtype = split(/\s/,$_); 1092 foreach $field (1..(@logtype-1)) { 1093 if ($logtype[$field] eq "date") { $MSdate = $field-1; } 1094 if ($logtype[$field] eq "time") { $MStime = $field-1; } 1095 if ($logtype[$field] eq "c-ip") { $MSdomain = $field-1; } 1096 if ($logtype[$field] eq "cs-method") { $MSrequest1 = $field-1; } 1097 if ($logtype[$field] eq "cs-uri-stem") { $MSrequest2 = $field-1; } 1098 if ($logtype[$field] eq "cs-uri-query") { $MSrequest3 = $field-1; } 1099 if ($logtype[$field] eq "sc-status") { $MSstatus = $field-1; } 1100 if ($logtype[$field] eq "sc-bytes") { $MSbytes = $field-1; } 1101 if ($logtype[$field] eq "cs(User-Agent)") { $MSagent = $field-1; } 1102 if ($logtype[$field] eq "cs(Referer)") { $MSreferer = $field-1; } 1103 if ($logtype[$field] eq "cs-username") { $MSusername = $field-1; } 1104 } 1105 if ($MSrequest1 && $MSrequest2 && $MSstatus) { 1106 unless ($MSagent) { $AgentsFile = ""; } 1107 unless ($MSreferer) { $RefsFile = ""; $KeywordsFile = ""; } 1108 } 1109 } 1110 next if m/^#/; 1111 $referer = ""; 1112 $Agent = ""; 1113 if ($logtype eq "microsoft") { 1114 @elements = split(/\s/,$_); 1115 ($Year,$Month,$Day) = $elements[$MSdate] =~ m/^(\d+)-(\d+)-(\d+)/o; 1116 $Time = $elements[$MStime]; 1117 $TimeDate = "$Day/$NumberToMonth{$Month}/$Year:$Time "; 1118 $Domain = $elements[$MSdomain]; 1119 if ($MSrequest1) { $Request = $elements[$MSrequest1]; } 1120 else { $Request = "GET"; } 1121 $Request .= " $elements[$MSrequest2]"; 1122 if ($MSrequest3) { 1123 if (length($elements[$MSrequest3]) > 1) { 1124 $Request .= "?$elements[$MSrequest3]"; 1125 } 1126 } 1127 $Request .= " HTTP/1.0"; 1128 $Status = $elements[$MSstatus]; 1129 $Bytes = $elements[$MSbytes]; 1130 if ($MSagent) { 1131 $Agent = $elements[$MSagent]; 1132 $Agent =~ s/\+/ /g; 1133 if (length($Agent) < 5) { $Agent = ""; } 1134 } 1135 if ($MSreferer) { 1136 $referer = $elements[$MSreferer]; 1137 if (length($referer) < 5) { $referer = ""; } 1138 } 1139 $rfc931 = "-"; 1140 if ($MSusername) { $authuser = $elements[$MSusername]; } 1141 else { $authuser = "-"; } 1142 } 1143 elsif (($Domain,$rfc931,$authuser,$TimeDate,$Request,$Status,$Bytes,$referer,$Agent) = 1144 /^(\S+) (\S+) (.+) \[(.+)\] \"(.+)\" (\S+) (\S+) \"(.*)\" \"(.*)\"/o) { 1145 $logtype = "combined"; 1146 if (length($referer) < 5) { $referer = ""; } 1147 if (length($Agent) < 5) { $Agent = ""; } 1148 } 1149 elsif (($Domain,$rfc931,$authuser,$TimeDate,$Request,$Status,$Bytes) = 1150 /^(\S+) (\S+) (.+) \[(.+)\] \"(.+)\" (\S+) (\S+)/o) { 1151 unless ($logtype) { $logtype = "standard"; } 1152 } 1153 elsif (($Domain,$xx,$date,$time,$xx,$xx,$xx,$xx,$xx,$Bytes,$Status,$xx,$Request,$file) = 1154 /^([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+),\s*([^\s,]+)/o) { 1155 unless ($logtype) { $logtype = "standard"; } 1156 ($Month,$Day,$Year) = $date =~ m/^(\d+)\/(\d+)\/(\d+)/o; 1157 if ($Year > 50) { $Year += 1900; } 1158 else { $Year += 2000; } 1159 if (($time < 10) && (length($time) < 8)) { $time = "0".$time; } 1160 $TimeDate = "$Day/$NumberToMonth{$Month}/$Year:$time "; 1161 $Request .= " $file HTTP/1.0"; 1162 $rfc931 = "-"; 1163 $authuser = "-"; 1164 } 1165 else { 1166 next; 1167 } 1168 unless ($Agent =~ /^\w/) { $Agent = ""; } 1169 if ($Status==403) { 1170 $Agent = "BannedAccess"; 1171 $referer = ""; 1172 } 1173 if ($Status==404) { 1174 $Agent = ""; 1175 } 1176 next if (!($Domain && $rfc931 && $authuser && $TimeDate && $Request && $Status)); 1177 ($Day,$Month,$Year,$Hour,$Minute,$Second)=$TimeDate=~ 1178 m#^(\d+)/(\w\w\w)/(\d\d\d\d):(\d\d):(\d\d):(\d\d) #o; 1179 if ($HourOffset) { 1180 &date_to_count(int($MonthToNumber{$Month}),int($Day),($Year-1900)); 1181 $Hour += $HourOffset; 1182 if ($Hour < 0) { 1183 $Hour += 24; 1184 $perp_days -= 1; 1185 &count_to_date($perp_days); 1186 $Year = $perp_year+1900; 1187 $Month = $perp_mon; 1188 if (length($Month) == 1) { $Month = "0".$Month; } 1189 $Month = $NumberToMonth{$Month}; 1190 $Day = $perp_day; 1191 } 1192 if ($Hour > 23) { 1193 $Hour -= 24; 1194 $perp_days += 1; 1195 &count_to_date($perp_days); 1196 $Year = $perp_year+1900; 1197 $Month = $perp_mon; 1198 if (length($Month) == 1) { $Month = "0".$Month; } 1199 $Month = $NumberToMonth{$Month}; 1200 $Day = $perp_day; 1201 } 1202 } 1203 if (length($Hour) == 1) { $Hour = "0".$Hour; } 1204 if (length($Day) == 1) { $Day = "0".$Day; } 1205 $Today="$Year $MonthToNumber{$Month} $Day $Hour $Minute $Second"; 1206 next if ($EOMDate && ($Today gt $EOMDate)); 1207 $EndDate=$Today if ($Today gt $EndDate); 1208 $FileEndDate=$Today if ($Today gt $FileEndDate); 1209 $FileStartDate=$Today if ($Today lt $FileStartDate); 1210 next unless (!$LogOnlyNew || ($Today gt $InitialEndDate)); 1211 ($Method,$FileName,$Protocol)=split(/\s/,$Request,3); 1212 $FileName=~ s/\%7[eE]/~/o; 1213 $FileName=~ s#//#/#go; 1214 $FileName=~ s#\.((s|p)*html*)/.*$#\.$1#o; 1215 unless ($IncludeQuery) { 1216 $FileName=~ s#\.(cgi)/.*$#\.$1#o; 1217 $FileName=~ s#\.(pl)/.*$#\.$1#o; 1218 } 1219 $FileName=~ s#^ *$#/#o; 1220 $FileName=~ s/#.+$//o; 1221 while ($FileName =~ /\/[^\/]*\/\.\.\//) { 1222 $FileName =~ s/\/[^\/]*\/\.\.\//\//; 1223 } 1224 $target = $FileName; 1225 unless ($IncludeQuery) { 1226 $FileName=~s#\?.*$##o; 1227 } 1228 $FileName=~ s/${DefaultPageName}$//oi; 1229 next if (($IncludeOnlyRefsTo) 1230 && !($FileName=~m#$IncludeOnlyRefsTo#o)); 1231 next if (($ExcludeRefsTo) 1232 && ($FileName=~m#$ExcludeRefsTo#o)); 1233 $FileName=~s#&#\&amp\;#go; 1234 $FileName=~s#<#\&lt\;#go; 1235 $FileName=~s#>#\&gt\;#go; 1236 $FileName=~s#"#\"#go; 1237 $TrimmedDomain = $Domain; 1238 $IOD_flag = 0; 1239 if (($IncludeOnlyDomain) && ($Domain=~m#$IncludeOnlyDomain#o)) { 1240 $IOD_flag = 1; 1241 } 1242 next if (($ExcludeDomain) && ($Domain=~m#$ExcludeDomain#o)); 1243 if (($Domain =~ /\d+\.\d+\.\d+\.\d+/) && !($NoResolve)) { 1244 $TrimmedDomain =~ s/(\d+\.\d+\.\d+)\.\d+/$1\.XXX/; 1245 if ($Resolved{$TrimmedDomain}) { 1246 $Domain = $Resolved{$TrimmedDomain}; 1247 } 1248 else { 1249 $ResolutionCount++; 1250 @bytes = split(/\./, $Domain); 1251 $packaddr = pack("C4",@bytes); 1252 $Resolved{$TrimmedDomain} = (gethostbyaddr($packaddr, 2))[0]; 1253 unless ($Resolved{$TrimmedDomain} =~ 1254 /^[a-zA-Z0-9\-\.]+\.([a-zA-Z]{2,3}|[0-9]{1,3})$/) { 1255 $Resolved{$TrimmedDomain} = ""; 1256 } 1257 $Resolved{$TrimmedDomain} = &TrimDomain($Resolved{$TrimmedDomain}); 1258 unless ($Resolved{$TrimmedDomain}) { 1259 $Resolved{$TrimmedDomain} = "unresolved"; 1260 } 1261 $Domain = $Resolved{$TrimmedDomain}; 1262 } 1263 $TrimmedDomain = "$Domain ($TrimmedDomain)"; 1264 } 1265 else { 1266 if ($Domain =~ /\d$/) { 1267 $TrimmedDomain =~ s/(\d+\.\d+\.\d+)\.\d+/$1\.XXX/; 1268 $Domain = "unresolved"; 1269 $TrimmedDomain = "$Domain ($TrimmedDomain)"; 1270 } 1271 else { 1272 $TrimmedDomain = $Domain; 1273 if ($PrintDomains eq "2") { 1274 $Domain = &TrimDomain($Domain); 1275 $TrimmedDomain = "$Domain ($TrimmedDomain)"; 1276 } 1277 } 1278 } 1279 if (($IncludeOnlyDomain) && ($Domain=~m#$IncludeOnlyDomain#o)) { 1280 $IOD_flag = 1; 1281 } 1282 next if (($IncludeOnlyDomain) && !($IOD_flag)); 1283 next if (($ExcludeDomain) && ($Domain=~m#$ExcludeDomain#o)); 1284 $PViewFilter = 0; 1285 if ((($Status>199)&&($Status<300))||($Status==304)) { 1286 unless ($DetailsFilter && ($FileName=~m#$DetailsFilter#oi)) { 1287 $PViewFilter = 1; 1288 } 1289 } 1290 $Domain=~tr/A-Z/a-z/; 1291 $TrimmedDomain=~tr/A-Z/a-z/; 1292 if ($PrintUserIDs && (length($authuser) > 1)) { 1293 $UserID = $authuser." (".$Domain.")"; 1294 } 1295 if (!($EOMDate) && $referer && ($RefsFile || $KeywordsFile)) { 1296 $target=~ s/\?.*$//o; 1297 $target=~ s/${DefaultPageName}$//oi; 1298 $target = &Simplify($target); 1299 $referer=~ s/\%7[eE]/~/o; 1300 $referer=~ s#^(http://[^/]+):80/#$1/#o; 1301 $refdomain = ""; 1302 $refquery = ""; 1303 if ($referer =~ m#srd\.yahoo\.com/goo/([^/]+)/\d+/#oi) { 1304 $refquery = $1; 1305 } 1306 elsif ($referer =~ m#srd\.yahoo\.com/[d|s]rst/\d+/([^/]+)/\d+/#oi) { 1307 $refquery = $1; 1308 } 1309 elsif ($referer =~ m#\?(.+)#o) { 1310 $refquery = $1; 1311 } 1312 $referer=~ s/#.*$//o; 1313 $referer=~ s/\?.*$//o; 1314 $referer=~ s/\*.*$//o; 1315 $referer=~ s/${DefaultPageName}$//oi; 1316 $referer = &Simplify($referer); 1317 unless (($refsexcludefrom && ($referer =~ m#$refsexcludefrom#oi)) 1318 || ($refsexcludeto && ($target =~ m#$refsexcludeto#oi))) { 1319 $keyform=$target.' '.$referer; 1320 $keyform=~ s#\&#\&amp\;#og; 1321 $keyform=~ s#<#\&lt\;#og; 1322 $keyform=~ s#>#\&gt\;#og; 1323 $keyform=~ s#"#\&quot\;#og; 1324 $referer=~ s#\&#\&amp\;#og; 1325 $referer=~ s#<#\&lt\;#og; 1326 $referer=~ s#>#\&gt\;#og; 1327 $referer=~ s#"#\&quot\;#og; 1328 $TargetCounter{$keyform}++; 1329 } 1330 else { 1331 $referer = ""; 1332 } 1333 if ($referer =~ m#^.+//([\w|\.|-]+)#o) { 1334 $refdomain = $1; 1335 } 1336 unless ($refdomain =~ /\d$/) { 1337 if ($refdomain =~ /([^.]*\.[^.]{3,})$/) { 1338 $refdomain{$1}++; 1339 } 1340 elsif ($refdomain =~ /([^.]*\.[^.]{1,3}\.[^.]*)$/) { 1341 $refdomain{$1}++; 1342 } 1343 elsif ($refdomain =~ /([^.]*\.[^.]*)$/) { 1344 $refdomain{$1}++; 1345 } 1346 } 1347 if ($referer && $refquery) { 1348 $refquery =~ tr/A-Z/a-z/; 1349 $refquery =~ s/\%25/\%/g; 1350 $refquery =~ s/%([0-7][a-fA-F0-9])/pack("C", hex($1))/eg; 1351 $refquery =~ s/%([a-fA-F8-9][a-fA-F0-9])/ /eg; 1352 $refquery =~ s/<([^>]|\n)*(>|$)/ /g; 1353 $refquery =~ s/"//gi; 1354 $refquery =~ s/\+/ /g; 1355 $refquery =~ s/\-/ /g; 1356 $refquery =~ s/\s+/ /g; 1357 $refquery =~ s/\s$//g; 1358 $refquery =~ s/\s&/&/g; 1359 $refquery =~ s/\s;/;/g; 1360 $refquery =~ s/xspc/ /g; 1361 if (($refdomain =~ /about\.com/i) && ($refquery =~ /terms=\s*([^&;]+)/i)) { 1362 $aboutcom{$1}++; 1363 $topkeywords{$1}++; 1364 } 1365 elsif ($refdomain =~ /altavista/i) { 1366 if (($refquery =~ /\&q=\s*([^&;]+)/i) 1367 || ($refquery =~ /^q=\s*([^&;]+)/i)) { 1368 $altavista{$1}++; 1369 $topkeywords{$1}++; 1370 } 1371 } 1372 elsif (($refdomain =~ /alltheweb/i) && ($refquery =~ /query=\s*([^&;]+)/i)) { 1373 $fast{$1}++; 1374 $topkeywords{$1}++; 1375 } 1376 elsif (($refdomain =~ /directhit/i) && ($refquery =~ /qry=\s*([^&;]+)/i)) { 1377 $directhit{$1}++; 1378 $topkeywords{$1}++; 1379 } 1380 elsif ($refdomain =~ /\.aol\./i) { 1381 if (($refquery =~ /search=\s*([^&;]+)/i) 1382 || ($refquery =~ /\&s=\s*([^&;]+)/i) 1383 || ($refquery =~ /^s=\s*([^&;]+)/i) 1384 || ($refquery =~ /\query=\s*([^&;]+)/i)) { 1385 $netfind{$1}++; 1386 $topkeywords{$1}++; 1387 } 1388 } 1389 elsif (($refdomain =~ /aj\.com/i) || ($refdomain =~ /ask\.com/i) 1390 || ($refdomain =~ /askjeeves/i)) { 1391 if (($refquery =~ /ask=\s*([^&;]+)/i) 1392 || ($refquery =~ / qry (.*) rnk /i)) { 1393 $askjeeves{$1}++; 1394 $topkeywords{$1}++; 1395 } 1396 } 1397 elsif (($refdomain =~ /cnet/i) || ($refdomain =~ /search\.com/i)) { 1398 if (($refquery =~ /query=\s*([^&;]+)/i) 1399 || ($refquery =~ /\&q=\s*([^&;]+)/i) 1400 || ($refquery =~ /^q=\s*([^&;]+)/i)) { 1401 $cnet{$1}++; 1402 $topkeywords{$1}++; 1403 } 1404 } 1405 elsif (($refdomain =~ /dogpile/i) && ($refquery =~ /q=\s*([^&;]+)/i)) { 1406 $dogpile{$1}++; 1407 $topkeywords{$1}++; 1408 } 1409 elsif (($refdomain =~ /euroferret/i) && ($refquery =~ /p=\s*([^&;]+)/i)) { 1410 $euroferret{$1}++; 1411 $topkeywords{$1}++; 1412 } 1413 elsif (($refdomain =~ /euroseek/i) && ($refquery =~ /query=\s*([^&;]+)/i)) { 1414 $euroseek{$1}++; 1415 $topkeywords{$1}++; 1416 } 1417 elsif (($refdomain =~ /magellan/i) || ($refdomain =~ /mckinley/i)) { 1418 if (($refquery =~ /search=\s*([^&;]+)/i) 1419 || ($refquery =~ /\&s=\s*([^&;]+)/i) 1420 || ($refquery =~ /^s=\s*([^&;]+)/i)) { 1421 $magellan{$1}++; 1422 $topkeywords{$1}++; 1423 } 1424 } 1425 elsif ($refdomain =~ /excite/i) { 1426 if (($refquery =~ /search=\s*([^&;]+)/i) 1427 || ($refquery =~ /\&s=\s*([^&;]+)/i) 1428 || ($refquery =~ /^s=\s*([^&;]+)/i)) { 1429 $excite{$1}++; 1430 $topkeywords{$1}++; 1431 } 1432 } 1433 elsif (($refdomain =~ /google/i) && ($refdomain !~ /yahoo/i)) { 1434 if (($refquery =~ /q=\s*([^&;]+)/i) 1435 || ($refquery =~ /query=\s*([^&;]+)/i)) { 1436 unless (($1 =~ /^cache:/) || (length($1) < 3)) { 1437 $google{$1}++; 1438 $topkeywords{$1}++; 1439 } 1440 } 1441 } 1442 elsif (($refdomain =~ /goto/i) && ($refquery =~ /keywords=\s*([^&;]+)/i)) { 1443 $goto{$1}++; 1444 $topkeywords{$1}++; 1445 } 1446 elsif (($refdomain =~ /hotbot/i) 1447 && (($refquery =~ /mt=\s*([^&;]+)/i) 1448 || ($refquery =~ /query=\s*([^&;]+)/i))) { 1449 $hotbot{$1}++; 1450 $topkeywords{$1}++; 1451 } 1452 elsif (($refdomain =~ /lycos/i) 1453 && (($refquery =~ /mt=\s*([^&;]+)/i) 1454 || ($refquery =~ /query=\s*([^&;]+)/i))) { 1455 $lycos{$1}++; 1456 $topkeywords{$1}++; 1457 } 1458 elsif (($refdomain =~ /infoseek/i) 1459 || ($refdomain =~ /^go\.com/i) || ($refdomain =~ /\.go\.com/i)) { 1460 $iseek = ""; 1461 if ($refquery =~ /qt=\s*([^&;]+)/i) { $iseek .= $1; } 1462 if ($refquery =~ /oq=\s*([^&;]+)/i) { $iseek .= " ".$1; } 1463 $iseek =~ s/^\s+//g; 1464 $iseek =~ s/\s+/ /g; 1465 if ($iseek) { 1466 $infoseek{$iseek}++; 1467 $topkeywords{$iseek}++; 1468 } 1469 } 1470 elsif (($refdomain =~ /looksmart/i) && ($refquery =~ /key=\s*([^&;]+)/i)) { 1471 $looksmart{$1}++; 1472 $topkeywords{$1}++; 1473 } 1474 elsif (($refdomain =~ /mamma/i) && ($refquery =~ /query=\s*([^&;]+)/i)) { 1475 $mamma{$1}++; 1476 $topkeywords{$1}++; 1477 } 1478 elsif (($refdomain =~ /metacrawler/i) || ($refdomain =~ /go2net/i)) { 1479 if ($refquery =~ /general=\s*([^&;]+)/i) { 1480 $metacrawler{$1}++; 1481 $topkeywords{$1}++; 1482 } 1483 } 1484 elsif ($refdomain =~ /msn/i) { 1485 if (($refquery =~ /mt=\s*([^&;]+)/i) 1486 || ($refquery =~ /\&q=\s*([^&;]+)/i) 1487 || ($refquery =~ /^q=\s*([^&;]+)/i)) { 1488 $msn{$1}++; 1489 $topkeywords{$1}++; 1490 } 1491 } 1492 elsif (($refdomain =~ /netscape/i) && ($refquery =~ /search=\s*([^&;]+)/i)) { 1493 $netscape{$1}++; 1494 $topkeywords{$1}++; 1495 } 1496 elsif (($refdomain =~ /northernlight/i) && ($refquery =~ /qr=\s*([^&;]+)/i)) { 1497 $northernlight{$1}++; 1498 $topkeywords{$1}++; 1499 } 1500 elsif (($refdomain =~ /planetsearch/i) && ($refquery =~ /text=\s*([^&;]+)/i)) { 1501 $planetsearch{$1}++; 1502 $topkeywords{$1}++; 1503 } 1504 elsif (($refdomain =~ /savvysearch/i) && ($refquery =~ /q=\s*([^&;]+)/i)) { 1505 $savvysearch{$1}++; 1506 $topkeywords{$1}++; 1507 } 1508 elsif (($refdomain =~ /snap/i) || ($refdomain =~ /nbci/i)) { 1509 if (($refquery =~ /keyword=\s*([^&;]+)/i) 1510 || ($refquery =~ /k=\s*([^&;]+)/i)) { 1511 $snap{$1}++; 1512 $topkeywords{$1}++; 1513 } 1514 } 1515 elsif ($refdomain =~ /webcrawler/i) { 1516 if (($refquery =~ /search=\s*([^&;]+)/i) 1517 || ($refquery =~ /searchtext=\s*([^&;]+)/i) 1518 || ($refquery =~ /text=\s*([^&;]+)/i)) { 1519 $webcrawler{$1}++; 1520 $topkeywords{$1}++; 1521 } 1522 } 1523 elsif (($refdomain =~ /yahoo/i) && ($refquery =~ /p=\s*([^&;]+)/i)) { 1524 $yahoo{$1}++; 1525 $topkeywords{$1}++; 1526 } 1527 elsif ($refdomain =~ /srd.yahoo/i) { 1528 $yahoo{$refquery}++; 1529 $topkeywords{$refquery}++; 1530 } 1531 elsif (($refquery =~ /general=\s*([^&;]+)/i) 1532 || ($refquery =~ /key=\s*([^&;]+)/i) 1533 || ($refquery =~ /keyword=\s*([^&;]+)/i) 1534 || ($refquery =~ /keywords=\s*([^&;]+)/i) 1535 || ($refquery =~ /mt=\s*([^&;]+)/i) 1536 || ($refquery =~ /\&q=\s*([^&;]+)/i) 1537 || ($refquery =~ /^q=\s*([^&;]+)/i) 1538 || ($refquery =~ /qry=\s*([^&;]+)/i) 1539 || ($refquery =~ /query=\s*([^&;]+)/i) 1540 || ($refquery =~ /search=\s*([^&;]+)/i) 1541 || ($refquery =~ /searchfor=\s*([^&;]+)/i) 1542 || ($refquery =~ /string=\s*([^&;]+)/i) 1543 || ($refquery =~ /term=\s*([^&;]+)/i) 1544 || ($refquery =~ /terms=\s*([^&;]+)/i) 1545 || ($refquery =~ /text=\s*([^&;]+)/i) 1546 || ($refquery =~ /topic=\s*([^&;]+)/i)) { 1547 unless ($1 =~ /^[\d\.]*$/) { 1548 unless (($1 =~ /^cache:/) || (length($1) < 3)) { 1549 $othersearch{$1}++; 1550 $topkeywords{$1}++; 1551 } 1552 } 1553 } 1554 } 1555 } 1556 if ($Agent) { 1557 $Agent =~ s/\&/\&amp\;/go; 1558 $Agent =~ s/"/\&quot\;/go; 1559 $Agent =~ s/</\&lt\;/go; 1560 $Agent =~ s/>/\&gt\;/go; 1561 ($mainagent,$other) = split(/ [ \n]/,$Agent,2); 1562 if (!($EOMDate) && $AgentsFile) { 1563 unless ($AgentsIgnore && ($FileName=~m#$AgentsIgnore#oi)) { 1564 $refscounter++; 1565 $agentcounter{$mainagent}++; 1566 } 1567 } 1568 } 1569 else { $mainagent = "-"; } 1570 if ($Bytes eq "-") { $Bytes = 0; } 1571 if ($Status<400) { $Bytes+=250; } 1572 else { $Bytes+=500; } 1573 $HourFilesCounter{$Hour}++; 1574 if ($BytesHour{$Hour}) { $BytesHour{$Hour}+=$Bytes; } 1575 else { $BytesHour{$Hour} = $Bytes; } 1576 &date_to_count(int($MonthToNumber{$Month}),$Day,($Year-1900)); 1577 $MonthlyFilesCounter{"$Month $Year"}++; 1578 if ($MonthlyBytesCounter{"$Month $Year"}) { $MonthlyBytesCounter{"$Month $Year"}+=$Bytes; } 1579 else { $MonthlyBytesCounter{"$Month $Year"} = $Bytes; } 1580 $Today="$Year $MonthToNumber{$Month} $Day"; 1581 $DayFilesCounter{$Today}++; 1582 if ($BytesDay{$Today}) { $BytesDay{$Today}+=$Bytes; } 1583 else { $BytesDay{$Today} = $Bytes; } 1584 if ($PViewFilter) { 1585 $MonthlyPViewsCounter{"$Month $Year"}++; 1586 $MonthlyTotalPViewsCounter++; 1587 $PViewsDay{$Today}++; 1588 $PViewsHour{$Hour}++; 1589 } 1590 $Today="$Today $Hour $Minute $Second"; 1591 $TopDomain=&GetTopDomain($Domain); 1592 if ($Month =~ /$CurrMonth/) { 1593 if ($PrintDomains) { 1594 $DomainsFilesCounter{$Domain}++; 1595 if ($DomainsBytesCounter{$Domain}) { $DomainsBytesCounter{$Domain}+=$Bytes; } 1596 else { $DomainsBytesCounter{$Domain} = $Bytes; } 1597 unless ($LastAccessDomain{$Domain}) { $LastAccessDomain{$Domain} = 0; } 1598 if ($LastAccessDomain{$Domain} lt $Today) { 1599 $LastAccessDomain{$Domain}=$Today; 1600 } 1601 } 1602 $TopDomainFilesCounter{$TopDomain}++; 1603 if ($TopDomainBytesCounter{$TopDomain}) { $TopDomainBytesCounter{$TopDomain}+=$Bytes; } 1604 else { $TopDomainBytesCounter{$TopDomain} = $Bytes; } 1605 if ($TopDomainAccess{$TopDomain} lt $Today) { 1606 $TopDomainAccess{$TopDomain} = $Today; 1607 } 1608 } 1609 $MonthlyTotalBytesCounter+=$Bytes; 1610 $MonthlyTotalHitsCounter++; 1611 if ($DayBreak-$perp_days < 36) { 1612 $DailyTotalBytesCounter+=$Bytes; 1613 $DailyTotalHitsCounter++; 1614 if ($PViewFilter) { 1615 $DailyTotalPViewsCounter++; 1616 } 1617 } 1618 $TodayBytes+=$Bytes; 1619 if ($PrintUserIDs && (length($authuser) > 1)) { 1620 if ($Month =~ /$CurrMonth/) { 1621 if ($PrintUserIDs) { 1622 $HitsUserIDCounter{$UserID}++; 1623 if ($BytesUserIDCounter{$UserID}) { $BytesUserIDCounter{$UserID}+=$Bytes; } 1624 else { $BytesUserIDCounter{$UserID} = $Bytes; } 1625 if ($LastAccessUserID{$UserID} lt $Today) { 1626 $LastAccessUserID{$UserID}=$Today; 1627 } 1628 } 1629 } 1630 } 1631 if ($Status==404) { 1632 if ($Month =~ /$CurrMonth/) { 1633 if ($Print404) { 1634 $fnfHitsFileCounter{$FileName}++; 1635 if ($fnfBytesFileCounter{$FileName}) { $fnfBytesFileCounter{$FileName}+=$Bytes; } 1636 else { $fnfBytesFileCounter{$FileName} = $Bytes; } 1637 if ($fnfLastAccessFile{$FileName} lt $Today) { 1638 $fnfLastAccessFile{$FileName}=$Today; 1639 } 1640 } 1641 } 1642 } 1643 unless ((($Status>199)&&($Status<300))||($Status==304)) { 1644 $ErrorName = $FileName; 1645 $FileName = $RespCodes{$Status}; 1646 } 1647 if ($Month =~ /$CurrMonth/) { 1648 if ($PrintFiles) { 1649 $HitsFileCounter{$FileName}++; 1650 if ($BytesFileCounter{$FileName}) { $BytesFileCounter{$FileName}+=$Bytes; } 1651 else { $BytesFileCounter{$FileName} = $Bytes; } 1652 if ($LastAccessFile{$FileName} lt $Today) { 1653 $LastAccessFile{$FileName}=$Today; 1654 } 1655 } 1656 } 1657 unless ($RefsFile) { $referer = "-"; } 1658 unless ($referer) { $referer = "-"; } 1659 unless ($AgentPlatform{$mainagent}) { 1660 if ($mainagent eq "-") { 1661 $AgentPlatform{$mainagent} = "Other Agent (Unknown Platform)"; 1662 } 1663 else { 1664 $agent = $mainagent; 1665 &Identify_Agent; 1666 &Identify_Platform; 1667 $AgentPlatform{$mainagent} = "$longagent"; 1668 if ($shortplatform) { $AgentPlatform{$mainagent} .= " ($shortplatform)"; } 1669 } 1670 } 1671 unless ((($Status>199)&&($Status<300))||($Status==304)) { 1672 if (($Status>299)&&($Status<400)) { $TodayHits++; } 1673 elsif ($Status==403) { $TodayBanned++; } 1674 else { $TodayErrors++; } 1675 unless ($Status==403) { 1676 unless ($DetailsFilter && ($ErrorName=~m#$DetailsFilter#oi)) { 1677 $TodayPagesErrors++; 1678 unless ($NoSessions) { 1679 $ErrorName =~ s/`//; 1680 print TEMPACCESSES "$Day`$Month`$perp_days`$Hour`$Minute`$Second`$TrimmedDomain`$FileName = $ErrorName`$referer`$AgentPlatform{$mainagent}\n"; 1681 } 1682 } 1683 } 1684 } 1685 else { 1686 $TodayHits++; 1687 if ($PViewFilter) { 1688 if ($OrgDomain && ($Domain !~ /nresolve/)) { 1689 if (($Domain !~ /\./) || ($Domain =~ /.*$OrgDomain/)) { 1690 $TodayLocal++; 1691 } 1692 } 1693 $TodayPagesHits++; 1694 $TodayDomains{$TopDomain}++; 1695 $TodayHosts{$TrimmedDomain}++; 1696 unless ($NoSessions) { 1697 print TEMPACCESSES "$Day`$Month`$perp_days`$Hour`$Minute`$Second`$TrimmedDomain`$FileName`$referer`$AgentPlatform{$mainagent}\n"; 1698 } 1699 } 1700 } 1701 $LogLineCount++; 1702 } 1703 close (TEMPACCESSES); 1704 close (LOGFILE); 1705 if ($IPLog) { 1706 if ($DBMType==2) { dbmclose (%Resolved); } 1707 else { untie %Resolved; } 1708 } 1709 unless (($logtype eq "combined") || ($logtype eq "microsoft")) { 1710 unless ($logtype eq "standard") { 1711 if ($Verbose) { print " Error: Empty log file or unrecognized log format\n"; } 1712 return; 1713 } 1714 $RefsFile = ""; 1715 $KeywordsFile = ""; 1716 $AgentsFile = ""; 1717 } 1718 if ($Verbose) { print " ",&commas($LogLineCount)," of ",&commas($LogLineTotal)," log entries processed\n"; } 1719 if ($Verbose) { print " ",&commas($ResolutionCount)," IP addresses resolved\n"; } 1720 } 1721 1722 sub GetTopDomain { 1723 local($domname) = $_[0]; 1724 ($TopDomain)=($domname=~m#\.(\w+)$#o); 1725 $TopDomain=~tr/A-Z/a-z/; 1726 $TopDomain=~s/\d//g; 1727 $TopDomain="xxx" if ($TopDomain eq ""); 1728 unless ($CountryCodes{$TopDomain}) { $TopDomain = "ooo"; } 1729 return $TopDomain; 1730 } 1731 1732 sub TrimDomain { 1733 local($trimresult) = $_[0]; 1734 $trimresult =~ s/[^\.]*\.(.*)/$1/; 1735 if ($PrintDomains eq "2") { 1736 if ($trimresult =~ /([^.]*\.[^.]{3,})$/) { 1737 $trimresult = $1; 1738 } 1739 elsif ($trimresult =~ /([^.]*\.[^.]{1,3}\.[^.]*)$/) { 1740 $trimresult = $1; 1741 } 1742 elsif ($trimresult =~ /([^.]*\.[^.]*)$/) { 1743 $trimresult = $1; 1744 } 1745 } 1746 return $trimresult; 1747 } 1748 1749 sub PrintReport { 1750 open (REPORT,">$FileDir/$ReportFile") || die " Error Opening File: $ReportFile\n"; 1751 print REPORT "<HTML><HEAD><TITLE>Overall Activity Report: $SystemName</TITLE></HEAD>"; 1752 print REPORT "<BODY $bodyspec>\n"; 1753 if ($headerfile) { &Header; } 1754 print REPORT "<H1 ALIGN=CENTER>Overall Activity Report: $SystemName</H1>\n"; 1755 unless ($EndDate eq "0000 00 00 00 00 00") { 1756 print REPORT "(Accesses Through "; 1757 ($Year,$Month,$Day,$Hour,$Minute,$Second)= 1758 split(/ /,$EndDate); 1759 print REPORT "$Hour:$Minute:$Second $Day "; 1760 print REPORT "$NumberToMonth{$Month} $Year)\n"; 1761 } 1762 if ($FullListFile && ($PrintFiles || $Print404 || $PrintUserIDs || $PrintDomains)) { 1763 close (REPORT); 1764 open (REPORT,">$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 1765 print REPORT "<HTML><HEAD><TITLE>Overall Activity Report (Full Lists): $SystemName</TITLE></HEAD>"; 1766 print REPORT "<BODY $bodyspec>\n"; 1767 if ($headerfile) { &Header; } 1768 print REPORT "<H1 ALIGN=CENTER>Overall Activity Report (Full Lists): $SystemName</H1>\n"; 1769 unless ($EndDate eq "0000 00 00 00 00 00") { 1770 print REPORT "(Accesses Through "; 1771 ($Year,$Month,$Day,$Hour,$Minute,$Second)= 1772 split(/ /,$EndDate); 1773 print REPORT "$Hour:$Minute:$Second $Day "; 1774 print REPORT "$NumberToMonth{$Month} $Year)\n"; 1775 } 1776 close (REPORT); 1777 open (REPORT,">>$FileDir/$ReportFile") || die " Error Opening File: $ReportFile\n"; 1778 } 1779 if (!($EOMDate) && ($DetailsFile || $RefsFile || $KeywordsFile || $AgentsFile)) { 1780 print REPORT "|"; 1781 if ($DetailsFile) { 1782 print REPORT " <A HREF=\"$DetailsFile\">$AccessDetailsReport</A> |"; 1783 } 1784 if ($RefsFile) { 1785 print REPORT " <A HREF=\"$RefsFile\">Referring URLs Report</A> |"; 1786 } 1787 if ($KeywordsFile) { 1788 print REPORT " <A HREF=\"$KeywordsFile\">Keywords Report</A> |"; 1789 } 1790 if ($AgentsFile) { 1791 print REPORT " <A HREF=\"$AgentsFile\">Agents/Platforms Report</A> |"; 1792 } 1793 print REPORT "\n"; 1794 } 1795 print REPORT "<A NAME=\"index\"><HR></A>\n"; 1796 print REPORT "<H2 ALIGN=CENTER>Index</H2>\n"; 1797 print REPORT "<CENTER><TABLE><TR>\n"; 1798 print REPORT "<TD VALIGN=TOP>\n"; 1799 print REPORT "Long-Term Statistics:<UL>\n"; 1800 print REPORT "<LI><A HREF=\"#monthly\">Monthly Statistics</A>\n"; 1801 print REPORT "<LI><A HREF=\"#daily\">Daily Statistics (Past Five Weeks)</A>\n"; 1802 print REPORT "<LI><A HREF=\"#dayofweek\">Day of Week Averages</A>\n"; 1803 print REPORT "<LI><A HREF=\"#houraverage\">Hourly Averages</A>\n"; 1804 print REPORT "<LI><A HREF=\"#records\">"Record Book"</A>\n"; 1805 print REPORT "</UL></TD><TD>    &nbsp"; 1806 print REPORT "</TD><TD VALIGN=TOP>\n"; 1807 print REPORT "Statistics for the Current Month ($CurrMonth $CurrYear):<UL>\n"; 1808 if ($PrintFiles) { 1809 if ($PrintTopNFiles) { 1810 print REPORT "<LI><A HREF=\"#topnfilesbyhits\">Top $PrintTopNFiles Files by Number of Hits</A>\n"; 1811 print REPORT "<LI><A HREF=\"#topnfilesbyvolume\">Top $PrintTopNFiles Files by Volume</A>\n"; 1812 } 1813 print REPORT "<LI><A HREF=\"$FullListFile"; 1814 print REPORT "#files\">Complete File Statistics</A>\n"; 1815 } 1816 if ($Print404) { 1817 if ($PrintTopNFiles) { 1818 print REPORT "<LI><A HREF=\"#topn404\">Top $PrintTopNFiles Most Frequently Requested 404 Files</A>\n"; 1819 } 1820 print REPORT "<LI><A HREF=\"$FullListFile"; 1821 print REPORT "#404\">Complete 404 File Not Found Statistics</A>\n"; 1822 } 1823 if ($PrintFiles) { 1824 if ($PrintTopNFiles) { 1825 print REPORT "<LI><A HREF=\"#topentrance\">Top $PrintTopNFiles "Entrance" Pages</A>\n"; 1826 } 1827 print REPORT "<LI><A HREF=\"$FullListFile"; 1828 print REPORT "#entrancepages\">Complete "Entrance" Page List</A>\n"; 1829 if ($PrintTopNFiles) { 1830 print REPORT "<LI><A HREF=\"#topexit\">Top $PrintTopNFiles "Exit" Pages</A>\n"; 1831 } 1832 print REPORT "<LI><A HREF=\"$FullListFile"; 1833 print REPORT "#exitpages\">Complete "Exit" Page List</A>\n"; 1834 } 1835 if ($PrintUserIDs) { 1836 print REPORT "<LI><A HREF=\"$FullListFile"; 1837 print REPORT "#userids\">User ID Statistics</A>\n"; 1838 } 1839 print REPORT "<LI><A HREF=\"#topleveldomain\">\"Top Level\" Domains</A>\n"; 1840 if ($PrintDomains) { 1841 if ($PrintTopNDomains) { 1842 print REPORT "<LI><A HREF=\"#topnsitesbyhits\">Top $PrintTopNDomains Domains by Number of Hits</A>\n"; 1843 print REPORT "<LI><A HREF=\"#topnsitesbyvolume\">Top $PrintTopNDomains Domains by Volume</A>\n"; 1844 } 1845 print REPORT "<LI><A HREF=\"$FullListFile"; 1846 print REPORT "#domains\">Complete Domain Statistics</A>\n"; 1847 } 1848 print REPORT "</UL></TD></TR></TABLE></CENTER>\n"; 1849 print REPORT "<CENTER><TABLE BORDER CELLPADDING=6><TR>\n"; 1850 print REPORT "<TD><BIG>Key Terms:</BIG>\n"; 1851 print REPORT "Hits: The total number of files requested from the server.\n"; 1852 print REPORT " Bytes: The amount of information transferred in filling those requests.\n"; 1853 print REPORT " Visits: The (approximate) number of actual individual visitors.\n"; 1854 print REPORT " PViews: The number of Web pages viewed by those visitors.\n"; 1855 print REPORT "The bar graphs below are based upon "; 1856 if ($GraphBase eq "hits") { print REPORT "number of hits"; } 1857 elsif ($GraphBase eq "visits") { print REPORT "number of visitors"; } 1858 elsif ($GraphBase eq "pviews") { print REPORT "page views"; } 1859 else { print REPORT "bytes transferred"; } 1860 print REPORT ".</TD>\n"; 1861 print REPORT "</TR></TABLE></CENTER>\n"; 1862 &PrintMonthlyReport; 1863 &PrintDailyReport; 1864 &PrintDayofWeekReport; 1865 &PrintHourlyReport; 1866 &PrintRecords; 1867 if ($PrintFiles) { 1868 if ($PrintTopNFiles) { 1869 &PrintTopNFilesByHitsReport; 1870 &PrintTopNFilesByVolumeReport; 1871 } 1872 &PrintFilesReport; 1873 } 1874 if ($Print404) { 1875 if ($PrintTopNFiles) { 1876 &PrintTopN404Report; 1877 } 1878 &Print404Report; 1879 } 1880 if ($PrintFiles) { 1881 if ($PrintTopNFiles) { 1882 &PrintTopNEntranceReport; 1883 } 1884 &PrintEntranceReport; 1885 if ($PrintTopNFiles) { 1886 &PrintTopNExitReport; 1887 } 1888 &PrintExitReport; 1889 } 1890 if ($PrintUserIDs) { 1891 &PrintUserIDsReport; 1892 } 1893 &PrintTopLevelDomainsReport; 1894 if ($PrintDomains) { 1895 if ($PrintTopNDomains) { 1896 &PrintTopNDomainsByHitsReport; 1897 &PrintTopNDomainsByVolumeReport; 1898 } 1899 &PrintReversedDomainsReport; 1900 } 1901 print REPORT "<HR>\n"; 1902 if (!($EOMDate) && ($DetailsFile || $RefsFile || $KeywordsFile || $AgentsFile)) { 1903 print REPORT "|"; 1904 if ($DetailsFile) { 1905 print REPORT " <A HREF=\"$DetailsFile\">$AccessDetailsReport</A> |"; 1906 } 1907 if ($RefsFile) { 1908 print REPORT " <A HREF=\"$RefsFile\">Referring URLs Report</A> |"; 1909 } 1910 if ($KeywordsFile) { 1911 print REPORT " <A HREF=\"$KeywordsFile\">Keywords Report</A> |"; 1912 } 1913 if ($AgentsFile) { 1914 print REPORT " <A HREF=\"$AgentsFile\">Agents/Platforms Report</A> |"; 1915 } 1916 print REPORT "\n"; 1917 } 1918 print REPORT ""; 1919 print REPORT "This report was generated with "; 1920 print REPORT "<A HREF="; 1921 print REPORT "\"http://awsd.com/scripts/weblog/\">"; 1922 print REPORT "WebLog $version</A>\n"; 1923 if ($footerfile) { &Footer; } 1924 print REPORT "</BODY></HTML>\n"; 1925 close (REPORT); 1926 if ($FullListFile && ($PrintFiles || $Print404 || $PrintUserIDs || $PrintDomains)) { 1927 open (REPORT,">>$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 1928 print REPORT "<HR>\n"; 1929 print REPORT "This report was generated with "; 1930 print REPORT "<A HREF="; 1931 print REPORT "\"http://awsd.com/scripts/weblog/\">"; 1932 print REPORT "WebLog $version</A>\n"; 1933 if ($footerfile) { &Footer; } 1934 print REPORT "</BODY></HTML>\n"; 1935 close (REPORT); 1936 } 1937 } 1938 1939 sub PrintMonthlyReport { 1940 print REPORT "<A NAME=\"monthly\"><HR></A><H2 ALIGN=CENTER>Monthly Statistics</H2>\n"; 1941 print REPORT "<PRE>\n"; 1942 print REPORT " Hits Bytes Visits PViews Month\n\n"; 1943 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$EndDate); 1944 &date_to_count(int($Month),$Day,($Year-1900)); 1945 $MonthlyCounter{$perp_mons}="$Month $Year"; 1946 $MonthCounter = $perp_mons; 1947 foreach $key (1..$perp_mons) { 1948 &count_to_mon($key); 1949 if (length($perp_mon) == 1) { $perp_mon = "0".$perp_mon; } 1950 $perp_year = $perp_year + 1900; 1951 $MonthlyCounter{$key}="$NumberToMonth{$perp_mon} $perp_year"; 1952 unless ($MonthlySessionsCounter{$MonthlyCounter{$key}}) { 1953 $MonthlySessionsCounter{$MonthlyCounter{$key}} = 0; 1954 } 1955 unless ($monthusersessions{$key}) { $monthusersessions{$key} = 0; } 1956 $MonthlySessionsCounter{$MonthlyCounter{$key}} += 1957 $monthusersessions{$key}; 1958 $MonthlyTotalVisitsCounter += $monthusersessions{$key}; 1959 } 1960 foreach $key (1..$perp_mons) { 1961 &count_to_mon($key); 1962 if (length($perp_mon) == 1) { $perp_mon = "0".$perp_mon; } 1963 $perp_year = $perp_year + 1900; 1964 $MonthlyCounter{$key}="$NumberToMonth{$perp_mon} $perp_year"; 1965 unless ($MonthlyBytesCounter{$MonthlyCounter{$key}}) { 1966 unless ($GoMonthly) { 1967 $MonthCounter--; 1968 next; 1969 } 1970 $MonthlyFilesCounter{$MonthlyCounter{$key}} = 0; 1971 $MonthlyBytesCounter{$MonthlyCounter{$key}} = 0; 1972 $MonthlySessionsCounter{$MonthlyCounter{$key}} = 0; 1973 $MonthlyPViewsCounter{$MonthlyCounter{$key}} = 0; 1974 } 1975 unless ($MonthlyPViewsCounter{$MonthlyCounter{$key}}) { 1976 $MonthlyPViewsCounter{$MonthlyCounter{$key}} = 0; 1977 } 1978 $GoMonthly = 1; 1979 if ($NoSessions) { $MonthlySessionsCounter{$MonthlyCounter{$key}} = "-"; } 1980 printf REPORT "%12s%20s%12s%12s%13s",&commas($MonthlyFilesCounter{$MonthlyCounter{$key}}), 1981 &commas($MonthlyBytesCounter{$MonthlyCounter{$key}}),&commas($MonthlySessionsCounter{$MonthlyCounter{$key}}),&commas($MonthlyPViewsCounter{$MonthlyCounter{$key}}), 1982 " $MonthlyCounter{$key} "; 1983 if ($GraphBase eq "hits") { 1984 if ($MonthlyTotalHitsCounter < 1.0) { $Percent=0; } 1985 else { 1986 $Percent=int((($MonthlyFilesCounter{$MonthlyCounter{$key}}/$MonthlyTotalHitsCounter)*($MonthCounter*10))+.5); 1987 } 1988 } 1989 elsif ($GraphBase eq "visits") { 1990 if ($MonthlyTotalVisitsCounter < 1.0) { $Percent=0; } 1991 else { 1992 $Percent=int((($MonthlySessionsCounter{$MonthlyCounter{$key}}/$MonthlyTotalVisitsCounter)*($MonthCounter*10))+.5); 1993 } 1994 } 1995 elsif ($GraphBase eq "pviews") { 1996 if ($MonthlyTotalPViewsCounter < 1.0) { $Percent=0; } 1997 else { 1998 $Percent=int((($MonthlyPViewsCounter{$MonthlyCounter{$key}}/$MonthlyTotalPViewsCounter)*($MonthCounter*10))+.5); 1999 } 2000 } 2001 else { 2002 if ($MonthlyTotalBytesCounter < 1.0) { $Percent=0; } 2003 else { 2004 $Percent=int((($MonthlyBytesCounter{$MonthlyCounter{$key}}/$MonthlyTotalBytesCounter)*($MonthCounter*10))+.5); 2005 } 2006 } 2007 &PrintBarGraph; 2008 } 2009 print REPORT "</PRE>\n[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2010 } 2011 2012 sub PrintBarGraph { 2013 if ($GraphURL) { 2014 unless ($Percent<8) { 2015 print REPORT "<IMG SRC=\"$GraphURL/bar1.gif\" HEIGHT=7 WIDTH=32 ALT=\"\">"; 2016 $Percent=$Percent-8; 2017 } 2018 unless ($Percent<6) { 2019 print REPORT "<IMG SRC=\"$GraphURL/bar2.gif\" HEIGHT=7 WIDTH=24 ALT=\"\">"; 2020 $Percent=$Percent-6; 2021 } 2022 unless ($Percent<4) { 2023 print REPORT "<IMG SRC=\"$GraphURL/bar3.gif\" HEIGHT=7 WIDTH=16 ALT=\"\">"; 2024 $Percent=$Percent-4; 2025 } 2026 unless ($Percent<2) { 2027 print REPORT "<IMG SRC=\"$GraphURL/bar4.gif\" HEIGHT=7 WIDTH=8 ALT=\"\">"; 2028 $Percent=$Percent-2; 2029 } 2030 unless ($Percent == 0) { 2031 print REPORT "<IMG SRC=\"$GraphURL/bar5.gif\" HEIGHT=7 WIDTH=",4*$Percent," ALT=\"\">"; 2032 } 2033 } 2034 print REPORT "\n"; 2035 } 2036 2037 sub PrintDailyReport { 2038 print REPORT "<A NAME=\"daily\"><HR></A><H2 ALIGN=CENTER>Daily Statistics (Past Five Weeks)</H2>\n"; 2039 print REPORT "<PRE>\n"; 2040 print REPORT " Hits Bytes Visits PViews Date\n\n"; 2041 $DayCount=36; 2042 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$EndDate); 2043 &date_to_count(int($Month),$Day,($Year-1900)); 2044 $FinalDay = $perp_days; 2045 $FinalDayOK = 0; 2046 if ($Hour>12) { $FinalDayOK = 1; } 2047 $DayCount = $DayCount-($DayBreak-$FinalDay); 2048 foreach $key (($DayBreak-35)..$FinalDay) { 2049 &count_to_date($key); 2050 $Year = $perp_year+1900; 2051 $Month = $perp_mon; 2052 if (length($Month) == 1) { $Month = "0".$Month; } 2053 $Day = $perp_day; 2054 if (length($Day) == 1) { $Day = "0".$Day; } 2055 $Today="$Year $Month $Day"; 2056 unless ($DomainsDay{$Today}) { $DomainsDay{$Today} = 0; } 2057 unless ($dayusersessions{$key}) { $dayusersessions{$key} = 0; } 2058 $DomainsDay{$Today} += $dayusersessions{$key}; 2059 $DailyTotalVisitsCounter += $dayusersessions{$key}; 2060 } 2061 foreach $key (($DayBreak-35)..$FinalDay) { 2062 &count_to_date($key); 2063 $Year = $perp_year+1900; 2064 $Month = $perp_mon; 2065 if (length($Month) == 1) { $Month = "0".$Month; } 2066 $Day = $perp_day; 2067 if (length($Day) == 1) { $Day = "0".$Day; } 2068 $Today="$Year $Month $Day"; 2069 unless ($BytesDay{$Today}) { 2070 unless ($GoDaily) { 2071 $DayCount--; 2072 next; 2073 } 2074 $DayFilesCounter{$Today} = 0; 2075 $BytesDay{$Today} = 0; 2076 $DomainsDay{$Today} = 0; 2077 $PViewsDay{$Today} = 0; 2078 } 2079 unless ($PViewsDay{$Today}) { 2080 $PViewsDay{$Today} = 0; 2081 } 2082 $GoDaily = 1; 2083 $WhichDay = ($key-(int($key/7)*7)); 2084 if ($WhichDay > 2) { $WhichDay -= 2; } 2085 else { $WhichDay += 5; } 2086 if ($WhichDay==1) { print REPORT "\n"; } 2087 $RecordDate = "$NumberToMonth{$Month} $Day, $Year"; 2088 if ($DayFilesCounter{$Today} > $RecordHits) { 2089 $RecordHits = $DayFilesCounter{$Today}; 2090 $RecordHitsDate = $RecordDate; 2091 } 2092 if ($BytesDay{$Today} > $RecordBytes) { 2093 $RecordBytes = $BytesDay{$Today}; 2094 $RecordBytesDate = $RecordDate; 2095 } 2096 if ($DomainsDay{$Today} > $RecordVisits) { 2097 $RecordVisits = $DomainsDay{$Today}; 2098 $RecordVisitsDate = $RecordDate; 2099 } 2100 if ($PViewsDay{$Today} > $RecordPViews) { 2101 $RecordPViews = $PViewsDay{$Today}; 2102 $RecordPViewsDate = $RecordDate; 2103 } 2104 if ($NoSessions) { $DomainsDay{$Today} = "-"; } 2105 unless (($key == $FinalDay) && ($FinalDayOK < 1)) { 2106 $WhichDayCounter{$WhichDay} ++; 2107 $WhichDayFiles{$WhichDay} += $DayFilesCounter{$Today}; 2108 $WhichDayBytes{$WhichDay} += $BytesDay{$Today}; 2109 $WhichDayPViews{$WhichDay} += $PViewsDay{$Today}; 2110 unless ($NoSessions) { $WhichDayDomains{$WhichDay} += $DomainsDay{$Today}; } 2111 } 2112 printf REPORT "%12s%20s%12s%12s%13s",&commas($DayFilesCounter{$Today}),&commas($BytesDay{$Today}),&commas($DomainsDay{$Today}),&commas($PViewsDay{$Today}), 2113 "$Day $NumberToMonth{$Month} "; 2114 if ($GraphBase eq "hits") { 2115 if ($DailyTotalHitsCounter < 1.0) { $Percent=0; } 2116 else { 2117 $Percent=int((($DayFilesCounter{$Today}/$DailyTotalHitsCounter)*($DayCount*10))+.5); 2118 } 2119 } 2120 elsif ($GraphBase eq "visits") { 2121 if ($DailyTotalVisitsCounter < 1.0) { $Percent=0; } 2122 else { 2123 $Percent=int((($DomainsDay{$Today}/$DailyTotalVisitsCounter)*($DayCount*10))+.5); 2124 } 2125 } 2126 elsif ($GraphBase eq "pviews") { 2127 if ($DailyTotalPViewsCounter < 1.0) { $Percent=0; } 2128 else { 2129 $Percent=int((($PViewsDay{$Today}/$DailyTotalPViewsCounter)*($DayCount*10))+.5); 2130 } 2131 } 2132 else { 2133 if ($DailyTotalBytesCounter < 1.0) { $Percent=0; } 2134 else { 2135 $Percent=int((($BytesDay{$Today}/$DailyTotalBytesCounter)*($DayCount*10))+.5); 2136 } 2137 } 2138 &PrintBarGraph; 2139 } 2140 print REPORT "</PRE>\n[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2141 } 2142 2143 sub PrintDayofWeekReport { 2144 print REPORT "<A NAME=\"dayofweek\"><HR></A><H2 ALIGN=CENTER>Day of Week Averages</H2>\n"; 2145 print REPORT "<PRE>\n"; 2146 print REPORT " Hits Bytes Visits PViews Day\n\n"; 2147 $DoWTotalHitsCounter=$DoWTotalBytesCounter=$DoWTotalVisitsCounter=$DoWTotalPViewsCounter=0; 2148 foreach $key (1..7) { 2149 unless ($WhichDayCounter{$key} < 1) { 2150 $DoWFilesCounter{$DoWCounter{$key}} = int(($WhichDayFiles{$key}/$WhichDayCounter{$key})+.5); 2151 $DoWBytesCounter{$DoWCounter{$key}} = int(($WhichDayBytes{$key}/$WhichDayCounter{$key})+.5); 2152 $DoWPViewsCounter{$DoWCounter{$key}} = int(($WhichDayPViews{$key}/$WhichDayCounter{$key})+.5); 2153 unless ($NoSessions) { $DoWSessionsCounter{$DoWCounter{$key}} = int(($WhichDayDomains{$key}/$WhichDayCounter{$key})+.5); } 2154 $DoWTotalHitsCounter += $DoWFilesCounter{$DoWCounter{$key}}; 2155 $DoWTotalBytesCounter += $DoWBytesCounter{$DoWCounter{$key}}; 2156 $DoWTotalPViewsCounter += $DoWPViewsCounter{$DoWCounter{$key}}; 2157 $DoWTotalVisitsCounter += $DoWSessionsCounter{$DoWCounter{$key}}; 2158 } 2159 } 2160 foreach $key (1..7) { 2161 if ($DoWBytesCounter{$DoWCounter{$key}} < 1) { 2162 $DoWFilesCounter{$DoWCounter{$key}} = 0; 2163 $DoWBytesCounter{$DoWCounter{$key}} = 0; 2164 $DoWSessionsCounter{$DoWCounter{$key}} = 0; 2165 $DoWPViewsCounter{$DoWCounter{$key}} = 0; 2166 } 2167 unless ($DoWPViewsCounter{$DoWCounter{$key}}) { 2168 $DoWPViewsCounter{$DoWCounter{$key}} = 0; 2169 } 2170 if ($NoSessions) { $DoWSessionsCounter{$DoWCounter{$key}} = "-"; } 2171 printf REPORT "%12s%20s%12s%12s%13s",&commas($DoWFilesCounter{$DoWCounter{$key}}), 2172 &commas($DoWBytesCounter{$DoWCounter{$key}}), 2173 &commas($DoWSessionsCounter{$DoWCounter{$key}}), 2174 &commas($DoWPViewsCounter{$DoWCounter{$key}})," $DoWCounter{$key} "; 2175 if ($GraphBase eq "hits") { 2176 if ($DoWTotalHitsCounter < 1.0) { $Percent=0; } 2177 else { 2178 $Percent=int((($DoWFilesCounter{$DoWCounter{$key}}/$DoWTotalHitsCounter)*70)+.5); 2179 } 2180 } 2181 elsif ($GraphBase eq "visits") { 2182 if ($DoWTotalVisitsCounter < 1.0) { $Percent=0; } 2183 else { 2184 $Percent=int((($DoWSessionsCounter{$DoWCounter{$key}}/$DoWTotalVisitsCounter)*70)+.5); 2185 } 2186 } 2187 elsif ($GraphBase eq "pviews") { 2188 if ($DoWTotalPViewsCounter < 1.0) { $Percent=0; } 2189 else { 2190 $Percent=int((($DoWPViewsCounter{$DoWCounter{$key}}/$DoWTotalPViewsCounter)*70)+.5); 2191 } 2192 } 2193 else { 2194 if ($DoWTotalBytesCounter < 1.0) { $Percent=0; } 2195 else { 2196 $Percent=int((($DoWBytesCounter{$DoWCounter{$key}}/$DoWTotalBytesCounter)*70)+.5); 2197 } 2198 } 2199 &PrintBarGraph; 2200 } 2201 print REPORT "</PRE>\n[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2202 } 2203 2204 sub PrintHourlyReport { 2205 print REPORT "<A NAME=\"houraverage\"><HR></A><H2 ALIGN=CENTER>Hourly Averages</H2>\n"; 2206 print REPORT "<PRE>\n"; 2207 print REPORT " Hits Bytes Visits PViews Hour\n\n"; 2208 if ($FileStartDate > $InitialEndDate) { 2209 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$FileStartDate); 2210 } 2211 else { 2212 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$InitialEndDate); 2213 } 2214 &date_to_count(int($Month),$Day,($Year-1900)); 2215 $StartDay = $perp_days; 2216 $StartHour = int($Hour); 2217 if ($Minute > 30) { $StartHour ++; } 2218 if ($StartHour > 23) { $StartHour = 0; $StartDay ++; } 2219 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$FileEndDate); 2220 &date_to_count(int($Month),$Day,($Year-1900)); 2221 $EndDay = $perp_days; 2222 $EndHour = int($Hour); 2223 if ($Minute < 30) { $EndHour --; } 2224 if ($EndHour < 0) { $EndHour = 23; $EndDay --; } 2225 foreach $day ($StartDay..$EndDay) { 2226 foreach $hour (00..23) { 2227 next if (($day == $StartDay) && ($hour < $StartHour)); 2228 next if (($day == $EndDay) && ($hour > $EndHour)); 2229 if (length($hour) == 1) { $hour="0".$hour; } 2230 $HourlyDayCounter{$hour} ++; 2231 } 2232 } 2233 foreach $key (00..23) { 2234 if (length($key) == 1) { $key="0".$key; } 2235 if ($NoSessions) { $VisitsHour{$key} = "-"; } 2236 else { $VisitsHour{$key} += $hourusersessions{$key}; } 2237 if ($HourlyDayCounter{$key} > 0) { 2238 $BytesHour{$key} = int(($BytesHour{$key}/$HourlyDayCounter{$key})+.5); 2239 $HourFilesCounter{$key} = int(($HourFilesCounter{$key}/$HourlyDayCounter{$key})+.5); 2240 $PViewsHour{$key} = int(($PViewsHour{$key}/$HourlyDayCounter{$key})+.5); 2241 unless ($NoSessions) { $VisitsHour{$key} = int(($VisitsHour{$key}/$HourlyDayCounter{$key})+.5); } 2242 } 2243 $HourlyTotalBytesCounter+=$BytesHour{$key}; 2244 $HourlyTotalHitsCounter+=$HourFilesCounter{$key}; 2245 $HourlyTotalPViewsCounter+=$PViewsHour{$key}; 2246 unless ($NoSessions) { $HourlyTotalVisitsCounter+=$VisitsHour{$key}; } 2247 } 2248 foreach $key (00..23) { 2249 if (length($key) == 1) { $key="0".$key; } 2250 unless ($HourFilesCounter{$key}) { $HourFilesCounter{$key} = 0; } 2251 unless ($BytesHour{$key}) { $BytesHour{$key} = 0; } 2252 unless ($VisitsHour{$key}) { $VisitsHour{$key} = 0; } 2253 unless ($PViewsHour{$key}) { $PViewsHour{$key} = 0; } 2254 printf REPORT "%12s%20s%12s%12s%13s",&commas($HourFilesCounter{$key}), 2255 &commas($BytesHour{$key}),&commas($VisitsHour{$key}), 2256 &commas($PViewsHour{$key})," $key "; 2257 if ($GraphBase eq "hits") { 2258 if ($HourlyTotalHitsCounter < 1.0) { $Percent=0; } 2259 else { 2260 $Percent=int((($HourFilesCounter{$key}/$HourlyTotalHitsCounter)*240)+.5); 2261 } 2262 } 2263 elsif ($GraphBase eq "visits") { 2264 if ($HourlyTotalVisitsCounter < 1.0) { $Percent=0; } 2265 else { 2266 $Percent=int((($VisitsHour{$key}/$HourlyTotalVisitsCounter)*240)+.5); 2267 } 2268 } 2269 elsif ($GraphBase eq "pviews") { 2270 if ($HourlyTotalPViewsCounter < 1.0) { $Percent=0; } 2271 else { 2272 $Percent=int((($PViewsHour{$key}/$HourlyTotalPViewsCounter)*240)+.5); 2273 } 2274 } 2275 else { 2276 if ($HourlyTotalBytesCounter < 1.0) { $Percent=0; } 2277 else { 2278 $Percent=int((($BytesHour{$key}/$HourlyTotalBytesCounter)*240)+.5); 2279 } 2280 } 2281 &PrintBarGraph; 2282 } 2283 print REPORT "</PRE>\n[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2284 } 2285 2286 sub PrintRecords { 2287 print REPORT "<A NAME=\"records\"><HR></A><H2 ALIGN=CENTER>"Record Book"</H2>\n"; 2288 print REPORT "Most Hits: "; 2289 print REPORT &commas($RecordHits)," ($RecordHitsDate)\n"; 2290 print REPORT "Most Bytes: "; 2291 print REPORT &commas($RecordBytes)," ($RecordBytesDate)\n"; 2292 unless ($NoSessions) { 2293 print REPORT "Most Visits: "; 2294 print REPORT &commas($RecordVisits)," ($RecordVisitsDate)\n"; 2295 } 2296 print REPORT "Most PViews: "; 2297 print REPORT &commas($RecordPViews)," ($RecordPViewsDate)\n"; 2298 print REPORT "[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2299 } 2300 2301 sub commas { 2302 local($_)=@_; 2303 1 while s/(.*\d)(\d\d\d)/$1,$2/; 2304 $_; 2305 } 2306 2307 sub PrintTopNFilesByHitsReport { 2308 print REPORT "<A NAME=\"topnfilesbyhits\"><HR></A><H2 ALIGN=CENTER>Top $PrintTopNFiles Files "; 2309 print REPORT "by Number of Hits ($CurrMonth $CurrYear)</H2>\n"; 2310 print REPORT "<PRE>\n"; 2311 print REPORT "Last Accessed Hits Bytes File\n\n"; 2312 $Counter=1; 2313 foreach $key (sort ByHitsFiles keys(%HitsFileCounter)) { 2314 last if ($Counter > $PrintTopNFiles); 2315 next if ($TopFileListFilter && ($key=~m#$TopFileListFilter#oi)); 2316 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$LastAccessFile{$key}); 2317 $Month=$NumberToMonth{$Month}; 2318 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2319 &commas($HitsFileCounter{$key}),&commas($BytesFileCounter{$key}),$key; 2320 $Counter++; 2321 } 2322 print REPORT "</PRE>\n"; 2323 if ($FullListFile) { 2324 print REPORT "[ <A HREF=\"$FullListFile"; 2325 print REPORT "#files\">Complete File Statistics</A> ]\n "; 2326 } 2327 print REPORT "[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2328 } 2329 2330 sub ByHitsFiles { 2331 $HitsFileCounter{$b}<=>$HitsFileCounter{$a}; 2332 } 2333 2334 sub PrintTopNFilesByVolumeReport { 2335 print REPORT "<A NAME=\"topnfilesbyvolume\"><HR></A><H2 ALIGN=CENTER>Top $PrintTopNFiles Files "; 2336 print REPORT "by Volume ($CurrMonth $CurrYear)</H2>\n"; 2337 print REPORT "<PRE>\n"; 2338 print REPORT "Last Accessed Hits Bytes File\n\n"; 2339 $Counter=1; 2340 foreach $key (sort ByVolumeFiles keys(%HitsFileCounter)) { 2341 last if ($Counter > $PrintTopNFiles); 2342 next if ($TopFileListFilter && ($key=~m#$TopFileListFilter#oi)); 2343 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$LastAccessFile{$key}); 2344 $Month=$NumberToMonth{$Month}; 2345 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2346 &commas($HitsFileCounter{$key}),&commas($BytesFileCounter{$key}),$key; 2347 $Counter++; 2348 } 2349 print REPORT "</PRE>\n"; 2350 if ($FullListFile) { 2351 print REPORT "[ <A HREF=\"$FullListFile"; 2352 print REPORT "#files\">Complete File Statistics</A> ]\n "; 2353 } 2354 print REPORT "[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2355 } 2356 2357 sub ByVolumeFiles { 2358 $BytesFileCounter{$b}<=>$BytesFileCounter{$a}; 2359 } 2360 2361 sub PrintFilesReport { 2362 if ($FullListFile && !($EOMDate)) { 2363 close (REPORT); 2364 open (REPORT,">>$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 2365 } 2366 print REPORT "<A NAME=\"files\"><HR></A><H2 ALIGN=CENTER>Complete File Statistics ($CurrMonth $CurrYear)</H2>\n"; 2367 print REPORT "<PRE>\n"; 2368 print REPORT "Last Accessed Hits Bytes File\n\n"; 2369 foreach $key (sort keys(%HitsFileCounter)) { 2370 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$LastAccessFile{$key}); 2371 $Month=$NumberToMonth{$Month}; 2372 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2373 &commas($HitsFileCounter{$key}),&commas($BytesFileCounter{$key}),$key; 2374 } 2375 print REPORT "</PRE>\n[ <A HREF=\""; 2376 if ($FullListFile && !($EOMDate)) { print REPORT "$ReportFile"; } 2377 print REPORT "#index\">Return to Index</A> ]\n\n"; 2378 if ($FullListFile && !($EOMDate)) { 2379 close (REPORT); 2380 open (REPORT,">>$FileDir/$ReportFile") || die " Error Opening File: $ReportFile\n"; 2381 } 2382 } 2383 2384 sub PrintTopN404Report { 2385 print REPORT "<A NAME=\"topn404\"><HR></A><H2 ALIGN=CENTER>Top $PrintTopNFiles Most Frequently Requested "; 2386 print REPORT "404 Files ($CurrMonth $CurrYear)</H2>\n"; 2387 print REPORT "<PRE>\n"; 2388 print REPORT "Last Accessed Hits Bytes File\n\n"; 2389 $Counter=1; 2390 foreach $key (sort ByfnfHitsFiles keys(%fnfHitsFileCounter)) { 2391 last if ($Counter > $PrintTopNFiles); 2392 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$fnfLastAccessFile{$key}); 2393 $Month=$NumberToMonth{$Month}; 2394 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2395 &commas($fnfHitsFileCounter{$key}),&commas($fnfBytesFileCounter{$key}),$key; 2396 $Counter++; 2397 } 2398 print REPORT "</PRE>\n"; 2399 if ($FullListFile) { 2400 print REPORT "[ <A HREF=\"$FullListFile"; 2401 print REPORT "#404\">Complete 404 File Not Found Statistics</A> ]\n "; 2402 } 2403 print REPORT "[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2404 } 2405 2406 sub ByfnfHitsFiles { 2407 $fnfHitsFileCounter{$b}<=>$fnfHitsFileCounter{$a}; 2408 } 2409 2410 sub Print404Report { 2411 if ($FullListFile && !($EOMDate)) { 2412 close (REPORT); 2413 open (REPORT,">>$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 2414 } 2415 print REPORT "<A NAME=\"404\"><HR></A><H2 ALIGN=CENTER>Complete 404 File Not Found Statistics ($CurrMonth $CurrYear)</H2>\n"; 2416 print REPORT "<PRE>\n"; 2417 print REPORT "Last Accessed Hits Bytes File\n\n"; 2418 foreach $key (sort keys(%fnfHitsFileCounter)) { 2419 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$fnfLastAccessFile{$key}); 2420 $Month=$NumberToMonth{$Month}; 2421 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2422 &commas($fnfHitsFileCounter{$key}),&commas($fnfBytesFileCounter{$key}),$key; 2423 } 2424 print REPORT "</PRE>\n[ <A HREF=\""; 2425 if ($FullListFile && !($EOMDate)) { print REPORT "$ReportFile"; } 2426 print REPORT "#index\">Return to Index</A> ]\n\n"; 2427 if ($FullListFile && !($EOMDate)) { 2428 close (REPORT); 2429 open (REPORT,">>$FileDir/$ReportFile") || die " Error Opening File: $ReportFile\n"; 2430 } 2431 } 2432 2433 sub PrintTopNEntranceReport { 2434 print REPORT "<A NAME=\"topentrance\"><HR></A><H2 ALIGN=CENTER>Top $PrintTopNFiles "; 2435 print REPORT ""Entrance" Pages ($CurrMonth $CurrYear)</H2>\n"; 2436 print REPORT "<PRE>\n"; 2437 print REPORT " Hits File\n\n"; 2438 $Counter=1; 2439 foreach $key (sort ByEntranceCount keys(%firstpages)) { 2440 last if ($Counter > $PrintTopNFiles); 2441 next if ($TopFileListFilter && ($key=~m#$TopFileListFilter#oi)); 2442 printf REPORT "%32s %-s\n",&commas($firstpages{$key}),$key; 2443 $Counter++; 2444 } 2445 print REPORT "</PRE>\n"; 2446 if ($FullListFile) { 2447 print REPORT "[ <A HREF=\"$FullListFile"; 2448 print REPORT "#entrancepages\">Complete File Statistics</A> ]\n "; 2449 } 2450 print REPORT "[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2451 } 2452 2453 sub ByEntranceCount { 2454 $firstpages{$b}<=>$firstpages{$a}; 2455 } 2456 2457 sub PrintEntranceReport { 2458 if ($FullListFile && !($EOMDate)) { 2459 close (REPORT); 2460 open (REPORT,">>$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 2461 } 2462 print REPORT "<A NAME=\"entrancepages\"><HR></A><H2 ALIGN=CENTER>Complete "Entrance" "; 2463 print REPORT "Page List ($CurrMonth $CurrYear)</H2>\n"; 2464 print REPORT "<PRE>\n"; 2465 print REPORT " Hits File\n\n"; 2466 foreach $key (sort keys(%firstpages)) { 2467 printf REPORT "%32s %-s\n",&commas($firstpages{$key}),$key; 2468 } 2469 print REPORT "</PRE>\n[ <A HREF=\""; 2470 if ($FullListFile && !($EOMDate)) { print REPORT "$ReportFile"; } 2471 print REPORT "#index\">Return to Index</A> ]\n\n"; 2472 if ($FullListFile && !($EOMDate)) { 2473 close (REPORT); 2474 open (REPORT,">>$FileDir/$ReportFile") || die " Error Opening File: $ReportFile\n"; 2475 } 2476 } 2477 2478 sub PrintTopNExitReport { 2479 print REPORT "<A NAME=\"topexit\"><HR></A><H2 ALIGN=CENTER>Top $PrintTopNFiles "; 2480 print REPORT ""Exit" Pages ($CurrMonth $CurrYear)</H2>\n"; 2481 print REPORT "<PRE>\n"; 2482 print REPORT " Hits File\n\n"; 2483 $Counter=1; 2484 foreach $key (sort ByExitCount keys(%lastpages)) { 2485 last if ($Counter > $PrintTopNFiles); 2486 next if ($TopFileListFilter && ($key=~m#$TopFileListFilter#oi)); 2487 printf REPORT "%32s %-s\n",&commas($lastpages{$key}),$key; 2488 $Counter++; 2489 } 2490 print REPORT "</PRE>\n"; 2491 if ($FullListFile) { 2492 print REPORT "[ <A HREF=\"$FullListFile"; 2493 print REPORT "#exitpages\">Complete File Statistics</A> ]\n "; 2494 } 2495 print REPORT "[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2496 } 2497 2498 sub ByExitCount { 2499 $lastpages{$b}<=>$lastpages{$a}; 2500 } 2501 2502 sub PrintExitReport { 2503 if ($FullListFile && !($EOMDate)) { 2504 close (REPORT); 2505 open (REPORT,">>$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 2506 } 2507 print REPORT "<A NAME=\"exitpages\"><HR></A><H2 ALIGN=CENTER>Complete "Exit" "; 2508 print REPORT "Page List ($CurrMonth $CurrYear)</H2>\n"; 2509 print REPORT "<PRE>\n"; 2510 print REPORT " Hits File\n\n"; 2511 foreach $key (sort keys(%lastpages)) { 2512 printf REPORT "%32s %-s\n",&commas($lastpages{$key}),$key; 2513 } 2514 print REPORT "</PRE>\n[ <A HREF=\""; 2515 if ($FullListFile && !($EOMDate)) { print REPORT "$ReportFile"; } 2516 print REPORT "#index\">Return to Index</A> ]\n\n"; 2517 if ($FullListFile && !($EOMDate)) { 2518 close (REPORT); 2519 open (REPORT,">>$FileDir/$ReportFile") || die " Error Opening File: $ReportFile\n"; 2520 } 2521 } 2522 2523 sub PrintUserIDsReport { 2524 if ($FullListFile && !($EOMDate)) { 2525 close (REPORT); 2526 open (REPORT,">>$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 2527 } 2528 print REPORT "<A NAME=\"userids\"><HR></A><H2 ALIGN=CENTER>User ID Statistics ($CurrMonth $CurrYear)</H2>\n"; 2529 print REPORT "<PRE>\n"; 2530 print REPORT "Last Accessed Hits Bytes User ID (Domain)\n\n"; 2531 foreach $key (sort keys(%HitsUserIDCounter)) { 2532 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$LastAccessUserID{$key}); 2533 $Month=$NumberToMonth{$Month}; 2534 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2535 &commas($HitsUserIDCounter{$key}),&commas($BytesUserIDCounter{$key}),$key; 2536 } 2537 print REPORT "</PRE>\n[ <A HREF=\""; 2538 if ($FullListFile && !($EOMDate)) { print REPORT "$ReportFile"; } 2539 print REPORT "#index\">Return to Index</A> ]\n\n"; 2540 if ($FullListFile && !($EOMDate)) { 2541 close (REPORT); 2542 open (REPORT,">>$FileDir/$ReportFile") || die " Error Opening File: $ReportFile\n"; 2543 } 2544 } 2545 2546 sub PrintTopLevelDomainsReport { 2547 print REPORT "<A NAME=\"topleveldomain\"><HR></A><H2 ALIGN=CENTER>\"Top Level\" Domains ($CurrMonth $CurrYear)</H2>\n"; 2548 print REPORT "<PRE>\n"; 2549 print REPORT "Last Accessed Hits Bytes \"Top Level\" Domain\n\n"; 2550 foreach $TopDomain (sort ByTopDomain keys(%TopDomainBytesCounter)) { 2551 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$TopDomainAccess{$TopDomain}); 2552 $Month=$NumberToMonth{$Month}; 2553 unless ($TopDomainFilesCounter{$TopDomain}<1) { 2554 printf REPORT "%-21s%11s%16s %-4s = %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2555 &commas($TopDomainFilesCounter{$TopDomain}), 2556 &commas($TopDomainBytesCounter{$TopDomain}),$TopDomain,$CountryCodes{$TopDomain}; 2557 } 2558 } 2559 print REPORT "</PRE>\n[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2560 } 2561 2562 sub ByTopDomain { 2563 $TopDomainBytesCounter{$b}<=>$TopDomainBytesCounter{$a}; 2564 } 2565 2566 sub PrintTopNDomainsByHitsReport { 2567 print REPORT "<A NAME=\"topnsitesbyhits\"><HR></A><H2 ALIGN=CENTER>Top $PrintTopNDomains Domains "; 2568 print REPORT "by Number of Hits ($CurrMonth $CurrYear)</H2>\n"; 2569 print REPORT "<PRE>\n"; 2570 print REPORT "Last Access Hits Bytes Domain\n\n"; 2571 $Counter=1; 2572 foreach $key (sort ByNDomains keys(%DomainsFilesCounter)) { 2573 last if ($Counter > $PrintTopNDomains); 2574 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$LastAccessDomain{$key}); 2575 $Month=$NumberToMonth{$Month}; 2576 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2577 &commas($DomainsFilesCounter{$key}),&commas($DomainsBytesCounter{$key}),$key; 2578 $Counter++; 2579 } 2580 print REPORT "</PRE>\n"; 2581 if ($FullListFile) { 2582 print REPORT "[ <A HREF=\"$FullListFile"; 2583 print REPORT "#domains\">Complete Domain Statistics</A> ]\n "; 2584 } 2585 print REPORT "[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2586 } 2587 2588 sub ByNDomains { 2589 $DomainsFilesCounter{$b}<=>$DomainsFilesCounter{$a}; 2590 } 2591 2592 sub PrintTopNDomainsByVolumeReport { 2593 print REPORT "<A NAME=\"topnsitesbyvolume\"><HR></A><H2 ALIGN=CENTER>Top $PrintTopNDomains Domains "; 2594 print REPORT "by Volume ($CurrMonth $CurrYear)</H2>\n"; 2595 print REPORT "<PRE>\n"; 2596 print REPORT "Last Access Hits Bytes Domain\n\n"; 2597 $Counter=1; 2598 foreach $key (sort ByVolumeDomains keys(%DomainsFilesCounter)) { 2599 last if ($Counter > $PrintTopNDomains); 2600 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$LastAccessDomain{$key}); 2601 $Month=$NumberToMonth{$Month}; 2602 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2603 &commas($DomainsFilesCounter{$key}),&commas($DomainsBytesCounter{$key}),$key; 2604 $Counter++; 2605 } 2606 print REPORT "</PRE>\n"; 2607 if ($FullListFile) { 2608 print REPORT "[ <A HREF=\"$FullListFile"; 2609 print REPORT "#domains\">Complete Domain Statistics</A> ]\n "; 2610 } 2611 print REPORT "[ <A HREF=\"#index\">Return to Index</A> ]\n\n"; 2612 } 2613 2614 sub ByVolumeDomains { 2615 $DomainsBytesCounter{$b}<=>$DomainsBytesCounter{$a}; 2616 } 2617 2618 sub PrintReversedDomainsReport { 2619 if ($FullListFile && !($EOMDate)) { 2620 close (REPORT); 2621 open (REPORT,">>$FileDir/$FullListFile") || die " Error Opening File: $FullListFile\n"; 2622 } 2623 print REPORT "<A NAME=\"domains\"><HR></A><H2 ALIGN=CENTER>Complete Domain Statistics ($CurrMonth $CurrYear)</H2>\n"; 2624 print REPORT "<PRE>\n"; 2625 print REPORT "Last Access Hits Bytes Domain\n\n"; 2626 foreach $key (sort ByReversedSubDomain keys(%DomainsFilesCounter)) { 2627 ($Year,$Month,$Day,$Hour,$Minute,$Second)=split(/ /,$LastAccessDomain{$key}); 2628 $Month=$NumberToMonth{$Month}; 2629 printf REPORT "%-21s%11s%16s %-s\n","$Hour:$Minute:$Second $Day $Month $Year", 2630 &commas($DomainsFilesCounter{$key}),&commas($DomainsBytesCounter{$key}),$key; 2631 } 2632 print REPORT "</PRE>\n[ <A HREF=\""; 2633 if ($FullListFile && !($EOMDate)) { print REPORT "$ReportFile"; } 2634 print REPORT "#index\">Return to Index</A> ]\n\n"; 2635 if ($FullListFile && !($EOMDate)) { 2636 close (REPORT); 2637 open (REPORT,">>$FileDir/$ReportFile") || die " Error Opening File: $ReportFile\n"; 2638 } 2639 } 2640 2641 sub ByReversedSubDomain { 2642 local(@adomains,@bdomains,$aisIP,$bisIP,$counter,$acounter,$bcounter,$result); 2643 $result=0; 2644 (@adomains)=split(/\./,$a); 2645 (@bdomains)=split(/\./,$b); 2646 $aisIP=$a=~m#^\d+\.\d+\.\d+\.\d+$#o; 2647 $bisIP=$b=~m#^\d+\.\d+\.\d+\.\d+$#o; 2648 if ($aisIP && (!$bisIP)) { 2649 $result=-1; 2650 } 2651 elsif ((!$aisIP) && $bisIP) { 2652 $result=1; 2653 } 2654 elsif ($aisIP && $bisIP) { 2655 $counter=0; 2656 while ((!$result) && ($counter < 4)) { 2657 $result=$adomains[$counter] <=> $bdomains[$counter]; 2658 $counter++; 2659 } 2660 } 2661 elsif ((!$aisIP) && (!$bisIP)) { 2662 $acounter=$#adomains; 2663 $bcounter=$#bdomains; 2664 while ((!$result) && ($acounter>=0) && ($bcounter>=0)) { 2665 $result=$adomains[$acounter] cmp $bdomains[$bcounter]; 2666 $acounter--; 2667 $bcounter--; 2668 } 2669 if (!$result) { 2670 $result=1 if ($acounter>=0); 2671 $result=-1 if ($bcounter>=0); 2672 } 2673 } 2674 $result; 2675 } 2676 2677 sub GetSessions { 2678 if (!$TodayHits) { $TodayHits=0; } 2679 if (!$TodayErrors) { $TodayErrors=0; } 2680 $TodaySpider=0; 2681 unless (($TodayPagesHits < 1 ) && ($TodayPagesErrors < 1 )) { 2682 open (TEMPACCESSES,"$FileDir/tempaccesses.txt"); 2683 if (!($EOMDate) && !($DetailsSummaryOnly) && $DetailsFile) { 2684 open (TEMPSESSIONS,">$FileDir/tempsessions.txt"); 2685 } 2686 while (<TEMPACCESSES>) { 2687 chomp; 2688 ($day,$month,$daycount,$hour,$minute,$second,$remote,$page,$referer,$Agent)=split("`",$_); 2689 if (($Agent eq "E-Mail Harvester") || ($Agent eq "Download Manager") 2690 || ($Agent eq "Link Checker") || ($Agent eq "Offline Browser") 2691 || ($Agent =~ m#Spider/Robot#oi)) { 2692 unless ($page =~ /^Code \d+/) { 2693 $TodaySpider++; 2694 } 2695 } 2696 $visit=(($hour*3600)+($minute*60)+$second); 2697 $duration=-.5; 2698 if ($prevvisit{"$remote $Agent"}) { 2699 $duration=($visit-$prevvisit{"$remote $Agent"}); 2700 } 2701 $prevvisit{"$remote $Agent"}=$visit; 2702 if (($duration < -.5) 2703 && (($daycount-$prevday{"$remote $Agent"})<2)) { 2704 $duration=$duration+86400; 2705 } 2706 elsif (($duration > -.5) 2707 && ($prevday{"$remote $Agent"} ne $daycount)) { 2708 $duration=-.5; 2709 } 2710 $prevday{"$remote $Agent"} = $daycount; 2711 if ($duration > 1800) { $duration=-.5; } 2712 if ($duration < .0) { 2713 $usersessions++; 2714 $sessionstime=$sessionstime+30; 2715 $dayusersessions{$daycount}++; 2716 &count_to_date($daycount); 2717 &date_to_count($perp_mon,$perp_day,$perp_year); 2718 $monthusersessions{$perp_mons}++; 2719 $hourusersessions{$hour}++; 2720 if ($previouspage{"$remote $Agent"}) { 2721 if ($lastpages{$previouspage{"$remote $Agent"}}) { 2722 $lastpages{$previouspage{"$remote $Agent"}}++; 2723 } 2724 else { 2725 $lastpages{$previouspage{"$remote $Agent"}} = 1; 2726 } 2727 } 2728 if ($firstpages{$page}) { 2729 $firstpages{$page}++; 2730 } 2731 else { 2732 $firstpages{$page} = 1; 2733 } 2734 } 2735 else { 2736 $sessionstime=$sessionstime+$duration; 2737 } 2738 $previouspage{"$remote $Agent"} = $page; 2739 $durminute=int($duration/60); 2740 $dursecond=$duration-($durminute*60); 2741 if (length($durminute) == 1) { $durminute="0".$durminute; } 2742 if (length($dursecond) == 1) { $dursecond="0".$dursecond; } 2743 unless ($AgentsFile) { $Agent = "-"; } 2744 if (!($EOMDate) && !($DetailsSummaryOnly) && $DetailsFile) { 2745 print TEMPSESSIONS "$day`$month`$hour`$minute`$second`$durminute`$dursecond`$remote`$Agent`$page`$referer\n"; 2746 } 2747 } 2748 close (TEMPACCESSES); 2749 foreach $key (keys(%previouspage)) { 2750 if ($lastpages{$previouspage{$key}}) { 2751 $lastpages{$previouspage{$key}}++; 2752 } 2753 else { 2754 $lastpages{$previouspage{$key}} = 1; 2755 } 2756 } 2757 if (!($EOMDate) && !($DetailsSummaryOnly) && $DetailsFile) { 2758 close (TEMPSESSIONS); 2759 } 2760 } 2761 } 2762 2763 sub PrintHostDetailsReport { 2764 if ($Verbose) { print " Generating Details Report\n"; } 2765 rename ("$FileDir/$DetailsFile","$FileDir/tempdetails.txt"); 2766 open (REPORT,">$FileDir/$DetailsFile") || die " Error Opening File: $DetailsFile\n"; 2767 print REPORT "<HTML><HEAD><TITLE>$AccessDetailsReport: $SystemName</TITLE></HEAD>"; 2768 print REPORT "<BODY $bodyspec>\n"; 2769 if ($headerfile) { &Header; } 2770 print REPORT "<H1 ALIGN=CENTER>Access "; 2771 if ($DetailsSummaryOnly) { print REPORT "Summary:"; } 2772 else { print REPORT "Details:"; } 2773 print REPORT " $SystemName</H1>\n"; 2774 unless ($EndDate eq "0000 00 00 00 00 00") { 2775 print REPORT "(Accesses Through "; 2776 ($Year,$Month,$Day,$Hour,$Minute,$Second)= 2777 split(/ /,$EndDate); 2778 print REPORT "$Hour:$Minute:$Second $Day "; 2779 print REPORT "$NumberToMonth{$Month} $Year)\n"; 2780 } 2781 print REPORT "| <A HREF=\"$ReportFile\">Overall Activity Report</A> |"; 2782 if ($RefsFile) { 2783 print REPORT " <A HREF=\"$RefsFile\">Referring URLs Report</A> |"; 2784 } 2785 if ($KeywordsFile) { 2786 print REPORT " <A HREF=\"$KeywordsFile\">Keywords Report</A> |"; 2787 } 2788 if ($AgentsFile) { 2789 print REPORT " <A HREF=\"$AgentsFile\">Agents/Platforms Report</A> |"; 2790 } 2791 print REPORT "<HR>\n"; 2792 unless ($DetailsSummaryOnly) { 2793 print REPORT "This report keeps track of "user sessions," "; 2794 print REPORT "showing the paths taken through the site by its visitors. "; 2795 print REPORT "It also provides an estimate of how many unique visitors "; 2796 print REPORT "the site has had and how long they've stayed. "; 2797 print REPORT "Please note, however, that precise tracking of "; 2798 print REPORT "the number of visitors is impossible; the information "; 2799 print REPORT "in this report is at best a reasonably close approximation "; 2800 print REPORT "based on the information in the server access log.\n"; 2801 print REPORT "<HR>\n"; 2802 } 2803 unless ((($TodayPagesHits < 1 ) && ($TodayPagesErrors < 1 )) || ($usersessions < 1)) { 2804 $TodayPagesTotal=$TodayPagesHits+$TodayPagesErrors; 2805 $TodayOutside=$TodayPagesTotal-$TodayLocal-$TodaySpider; 2806 $LocalPercent=int(($TodayLocal/$TodayPagesTotal)*1000+.5)/10; 2807 $OutsidePercent=int(($TodayOutside/$TodayPagesTotal)*1000+.5)/10; 2808 $SpiderPercent=int(($TodaySpider/$TodayPagesTotal)*1000+.5)/10; 2809 $TodayHosts=keys(%TodayHosts); 2810 $TodayKB=int(($TodayBytes/1024)+.5); 2811 $TodayMB=int(($TodayKB/1024)+.5); 2812 $AveragePages=sprintf("%3.1f",$TodayPagesHits/$usersessions); 2813 open (TEMPSESSIONS,"$FileDir/tempsessions.txt"); 2814 while (<TEMPSESSIONS>) { 2815 chomp; 2816 ($day,$month,$hour,$minute,$second,$durminute,$dursecond,$remote,$Agent,$page,$referer)=split("`",$_); 2817 $duration=$durminute.":".$dursecond; 2818 if ($duration =~ /-/) { 2819 $duration=" "; 2820 } 2821 $timestring=$day." ".$month." -- ".$hour.":".$minute.":".$second; 2822 if ($FirstHalf{"$Agent`$remote"}) { 2823 $hostlist{"$Agent`$remote"} .= $FirstHalf{"$Agent`$remote"}.$duration.$SecondHalf{"$Agent`$remote"}; 2824 } 2825 unless ($hostlist{"$Agent`$remote"}) { $hostlist{"$Agent`$remote"} = ""; } 2826 $FirstHalf{"$Agent`$remote"} = ""; 2827 if ($referer ne "-") { 2828 $FirstHalf{"$Agent`$remote"} = " ".$referer."\n"; 2829 } 2830 $FirstHalf{"$Agent`$remote"} .= " ".$timestring." -- "; 2831 $SecondHalf{"$Agent`$remote"} = " -- ".$page."\n"; 2832 } 2833 close (TEMPSESSIONS); 2834 $averagesession=$sessionstime/$usersessions; 2835 $averageminute=int(($averagesession/60)+.5); 2836 if ($averageminute == 0) { $averagesession="less than a minute"; } 2837 elsif ($averageminute == 1) { $averagesession="1 minute"; } 2838 else { $averagesession=$averageminute." minutes"; } 2839 } 2840 if ($DetailsSummaryOnly) { 2841 print REPORT "<H2 ALIGN=CENTER>Access Summary</H2>\n"; 2842 } 2843 else { 2844 print REPORT "<H2 ALIGN=CENTER>Detailed Access List</H2>\n"; 2845 } 2846 print REPORT "\n"; 2847 unless ($DetailsSummaryOnly) { 2848 print REPORT "<HR><BLOCKQUOTE>\n"; 2849 } 2850 print REPORT "Log Analyzed on $CurrDate"; 2851 unless ($FileEndDate eq "0000 00 00 00 00 00") { 2852 ($Year,$Month,$Day,$Hour,$Minute,$Second)= 2853 split(/ /,$FileStartDate); 2854 print REPORT " ($Hour:$Minute:$Second $Day "; 2855 print REPORT "$NumberToMonth{$Month} $Year to "; 2856 ($Year,$Month,$Day,$Hour,$Minute,$Second)= 2857 split(/ /,$FileEndDate); 2858 print REPORT "$Hour:$Minute:$Second $Day "; 2859 print REPORT "$NumberToMonth{$Month} $Year)"; 2860 } 2861 print REPORT ":\n"; 2862 if ($TodayPagesTotal < 1 ) { 2863 print REPORT " There were no accesses.\n"; 2864 unless ($DetailsSummaryOnly) { print REPORT "</BLOCKQUOTE>\n"; } 2865 } 2866 else { 2867 print REPORT " A total of ",&commas($TodayPagesTotal)," pages were requested "; 2868 print REPORT "(",&commas($TodayPagesHits)," successfully and ",&commas($TodayPagesErrors)," unsuccessfully) "; 2869 print REPORT "by ",&commas($TodayHosts)," unique hosts. "; 2870 print REPORT "Of those pages, "; 2871 if ($OrgDomain) { 2872 print REPORT "",&commas($TodayLocal)," "; 2873 printf REPORT ("(%.4g%%)",$LocalPercent); 2874 print REPORT " were "; 2875 if ($OrgName) { 2876 print REPORT "requested by $OrgName, "; 2877 } 2878 else { 2879 print REPORT "requested internally, "; 2880 } 2881 } 2882 print REPORT "",&commas($TodaySpider)," "; 2883 printf REPORT ("(%.4g%%)",$SpiderPercent); 2884 print REPORT " were requested by "spiders," "; 2885 print REPORT "and ",&commas($TodayOutside)," "; 2886 printf REPORT ("(%.4g%%)",$OutsidePercent); 2887 print REPORT " were requested by (presumably) human visitors. "; 2888 print REPORT "There were approximately "; 2889 print REPORT &commas($usersessions)," distinct visits; "; 2890 print REPORT "the typical visitor seems to have spent about "; 2891 print REPORT "$averagesession at the site and to have "; 2892 print REPORT "successfully viewed some $AveragePages pages. "; 2893 print REPORT "There were a total of "; 2894 print REPORT &commas($TodayHits)," hits and "; 2895 print REPORT "",&commas($TodayErrors)," errors "; 2896 print REPORT "related to $SystemName. "; 2897 if ($TodayBanned>0) { 2898 if ($TodayBanned<2) { 2899 print REPORT "A single additional access attempt was blocked. "; 2900 } 2901 else { 2902 print REPORT "An additional ",&commas($TodayBanned)," access attempts were blocked. "; 2903 } 2904 } 2905 print REPORT "Total bandwidth usage was approximately "; 2906 print REPORT &commas($TodayMB)," megabytes.\n"; 2907 unless ($DetailsSummaryOnly) { 2908 print REPORT "</BLOCKQUOTE>\n"; 2909 @hosts=sort byDomain (keys %hostlist); 2910 $prevTD="###"; 2911 print REPORT "<PRE>"; 2912 foreach $host (@hosts) { 2913 ($hostagent,$hostremote) = split("`",$host); 2914 $hostdomain = $hostremote; 2915 $hostdomain =~ s/([^\s]*).*/$1/; 2916 $TopDomain=&GetTopDomain($hostdomain); 2917 if ($TopDomain ne $prevTD) { 2918 $prevTD = $TopDomain; 2919 print REPORT "\n$CountryCodes{$TopDomain}:"; 2920 print REPORT "\n"; 2921 } 2922 print REPORT "\n$hostremote"; 2923 if ($AgentsFile) { 2924 print REPORT " - $hostagent"; 2925 } 2926 print REPORT "\n"; 2927 print REPORT "$hostlist{$host}"; 2928 print REPORT "$FirstHalf{$host}"; 2929 print REPORT " "; 2930 print REPORT "$SecondHalf{$host}"; 2931 } 2932 print REPORT "</PRE>\n"; 2933 } 2934 } 2935 $datestamp = 0; 2936 if (-e "$FileDir/tempdetails.txt") { 2937 open (FILE,"$FileDir/tempdetails.txt"); 2938 while (<FILE>) { 2939 if (m#DATESTAMP=\"(\d+)#o) { 2940 $datestamp = $1; 2941 $DetailsDays--; 2942 $DetailsSummaryDays--; 2943 } 2944 if ($datestamp > 0) { 2945 if ($DetailsDays > 0) { 2946 print REPORT $_; 2947 } 2948 elsif ($DetailsSummaryDays > 0) { 2949 if (m#</BLOCKQUOTE>#o) { 2950 $datestamp = 0; 2951 } 2952 else { 2953 unless (m#<HR>#o) { 2954 unless ($HRFlag) { 2955 $HRFlag = 1; 2956 print REPORT "<HR>\n"; 2957 } 2958 print REPORT $_; 2959 } 2960 } 2961 } 2962 } 2963 } 2964 close (FILE); 2965 unlink "$FileDir/tempdetails.txt"; 2966 } 2967 print REPORT "\n"; 2968 print REPORT "<HR>\n"; 2969 print REPORT "| <A HREF=\"$ReportFile\">Overall Activity Report</A> |"; 2970 if ($RefsFile) { 2971 print REPORT " <A HREF=\"$RefsFile\">Referring URLs Report</A> |"; 2972 } 2973 if ($KeywordsFile) { 2974 print REPORT " <A HREF=\"$KeywordsFile\">Keywords Report</A> |"; 2975 } 2976 if ($AgentsFile) { 2977 print REPORT " <A HREF=\"$AgentsFile\">Agents/Platforms Report</A> |"; 2978 } 2979 print REPORT "\n"; 2980 print REPORT ""; 2981 print REPORT "This report was generated with "; 2982 print REPORT "<A HREF="; 2983 print REPORT "\"http://awsd.com/scripts/weblog/\">"; 2984 print REPORT "WebLog $version</A>\n"; 2985 if ($footerfile) { &Footer; } 2986 print REPORT "</BODY></HTML>\n"; 2987 close (REPORT); 2988 } 2989 2990 sub byDomain { 2991 local($aHost) = ""; 2992 local($bHost) = ""; 2993 local($aIP) = ""; 2994 local($bIP) = ""; 2995 local(@aTemp,@bTemp)=(); 2996 $aHost = $a; 2997 $bHost = $b; 2998 $aHost =~ s/[^`]*`([^\s]*)(.*)/$1/; 2999 $aIP = $2; 3000 $bHost =~ s/[^`]*`([^\s]*)(.*)/$1/; 3001 $bIP = $2; 3002 if ($aHost =~ /[^0-9].*\.[^0-9].*/) { 3003 @aTemp=reverse split(/\./, $aHost); 3004 $aHost = ""; 3005 foreach (@aTemp) { $aHost .= $_ }; 3006 } 3007 else { $aHost="zzzzzzz".$aHost; } 3008 $aHost .= $aIP; 3009 if ($bHost =~ /[^0-9].*\.[^0-9].*/) { 3010 @bTemp=reverse split(/\./, $bHost); 3011 $bHost = ""; 3012 foreach (@bTemp) { $bHost .= $_ }; 3013 } 3014 else { $bHost="zzzzzzz".$bHost; } 3015 $bHost .= $bIP; 3016 return ($aHost cmp $bHost); 3017 } 3018 3019 sub PrintRefsReport { 3020 if ($Verbose) { print " Generating Referring URLs Report\n"; } 3021 if (-e "$FileDir/$RefsFile") { 3022 open (OLDLOG,"$FileDir/$RefsFile") || die " Error Opening File: $RefsFile\n"; 3023 while (<OLDLOG>) { 3024 chomp; 3025 if (m#listed pages, since (.*).#o) { $refsstartdate = $1; } 3026 if (m#^<DT>(.*)</DT>#o) { 3027 $target = &Simplify($1); 3028 next; 3029 } 3030 next if (! $target); 3031 if (m#^<DD><A HREF=\"([^"]*)\">.*</A> \(([\d,]+) reference#o) { 3032 $refurl = $1; 3033 $count = $2; 3034 $count =~ s/,//g; 3035 $url = &Simplify($refurl); 3036 if ($TargetCounter{"$target $url"}) { $TargetCounter{"$target $url"} += $count; } 3037 else { $TargetCounter{"$target $url"} = $count; } 3038 $refdomain = ""; 3039 if ($url =~ m#^.+//([\w|\.|-]+)#o) { 3040 $refdomain = $1; 3041 } 3042 unless ($refdomain =~ /\d$/) { 3043 if ($refdomain =~ /([^.]*\.[^.]{3,})$/) { 3044 $refdomain{$1} += $count; 3045 } 3046 elsif ($refdomain =~ /([^.]*\.[^.]{1,3}\.[^.]*)$/) { 3047 $refdomain{$1} += $count; 3048 } 3049 elsif ($refdomain =~ /([^.]*\.[^.]*)$/) { 3050 $refdomain{$1} += $count; 3051 } 3052 } 3053 } 3054 } 3055 close (OLDLOG); 3056 } 3057 foreach $key (keys (%TargetCounter)) { 3058 ($target,$referer)=split(/ /,$key,2); 3059 if ($TargetCounter{"$target/ $referer"} && $TargetCounter{"$target $referer"}) { 3060 $TargetCounter{$key}=-1; 3061 } 3062 } 3063 open (REPORT,">$FileDir/$RefsFile") || die " Error Opening File: $RefsFile\n"; 3064 print REPORT "<HTML><HEAD><TITLE>Referring URLs Report: $SystemName</TITLE></HEAD>"; 3065 print REPORT "<BODY $bodyspec>\n"; 3066 if ($headerfile) { &Header; } 3067 print REPORT "<H1 ALIGN=CENTER>Referring URLs: $SystemName</H1>\n"; 3068 unless ($EndDate eq "0000 00 00 00 00 00") { 3069 print REPORT "(Accesses Through "; 3070 ($Year,$Month,$Day,$Hour,$Minute,$Second)= 3071 split(/ /,$EndDate); 3072 print REPORT "$Hour:$Minute:$Second $Day "; 3073 print REPORT "$NumberToMonth{$Month} $Year)\n"; 3074 } 3075 print REPORT "| <A HREF=\"$ReportFile\">Overall Activity Report</A> |"; 3076 if ($DetailsFile) { 3077 print REPORT " <A HREF=\"$DetailsFile\">$AccessDetailsReport</A> |"; 3078 } 3079 if ($KeywordsFile) { 3080 print REPORT " <A HREF=\"$KeywordsFile\">Keywords Report</A> |"; 3081 } 3082 if ($AgentsFile) { 3083 print REPORT " <A HREF=\"$AgentsFile\">Agents/Platforms Report</A> |"; 3084 } 3085 print REPORT "<HR>\n"; 3086 unless ($refsstartdate) { 3087 ($Year,$Month,$Day,$Hour,$Minute,$Second) = split(/ /,$FileStartDate); 3088 $Day = int($Day); 3089 $refsstartdate = "$Day $NumberToMonth{$Month} $Year"; 3090 } 3091 print REPORT "This report logs the URLs reported by browsers as "; 3092 print REPORT "the "referers" directing them to the various "; 3093 print REPORT "listed pages, since $refsstartdate. (This "; 3094 print REPORT "information is far from perfect. Many browsers do not provide "; 3095 print REPORT "any information on the referring page, and even those that do "; 3096 print REPORT "can at times provide false or misleading data. The "; 3097 print REPORT "fact that a page is listed as the referer to a given "; 3098 print REPORT "page does not always mean that it "; 3099 print REPORT "actually contains a link to that page.) "; 3100 print REPORT "<HR>\n"; 3101 if ($TopNRefDoms) { 3102 print REPORT "<H2 ALIGN=CENTER>Top $TopNRefDoms "; 3103 print REPORT "Referring Domains</H2>\n"; 3104 print REPORT "<PRE>\n"; 3105 print REPORT " Referrals Domain\n\n"; 3106 $Counter=1; 3107 foreach $key (sort ByRefCount keys(%refdomain)) { 3108 last if ($Counter > $TopNRefDoms); 3109 printf REPORT "%32s%-s\n",&commas($refdomain{$key})," $key"; 3110 $Counter++; 3111 } 3112 print REPORT "</PRE><HR>\n\n"; 3113 } 3114 $RefsMinHits = 0; 3115 if ($RefsFilterLists) { 3116 $Counter=0; 3117 foreach $key (sort ByRefCount keys(%refdomain)) { 3118 $Counter++; 3119 $RefsMinHits += $refdomain{$key}; 3120 last if ($Counter > 24); 3121 } 3122 if ($Counter == 0) { $RefsMinHits = 0; } 3123 else { $RefsMinHits = (($RefsMinHits/$Counter)/250); } 3124 } 3125 print REPORT "<H2 ALIGN=CENTER>Web Pages and Referring URLs</H2>\n"; 3126 print REPORT "<DL>\n"; 3127 $LastWebPage = ''; 3128 foreach $key (sort bytargetthenhits keys(%TargetCounter)) { 3129 unless ($TargetCounter{$key} < $RefsMinHits) { 3130 ($target,$referer) = split(/ /,$key,2); 3131 if ("$target" ne "$LastWebPage") { 3132 print REPORT "<DT>$target</DT>\n"; 3133 } 3134 print REPORT "<DD><A HREF=\"$referer\">$referer</A> "; 3135 print REPORT "(",&commas($TargetCounter{$key})," reference"; 3136 if ($TargetCounter{$key} > 1) { print REPORT "s"; } 3137 print REPORT ")</DD>\n"; 3138 $LastWebPage = $target; 3139 } 3140 } 3141 print REPORT "</DL><HR>\n"; 3142 print REPORT "| <A HREF=\"$ReportFile\">Overall Activity Report</A> |"; 3143 if ($DetailsFile) { 3144 print REPORT " <A HREF=\"$DetailsFile\">$AccessDetailsReport</A> |"; 3145 } 3146 if ($KeywordsFile) { 3147 print REPORT " <A HREF=\"$KeywordsFile\">Keywords Report</A> |"; 3148 } 3149 if ($AgentsFile) { 3150 print REPORT " <A HREF=\"$AgentsFile\">Agents/Platforms Report</A> |"; 3151 } 3152 print REPORT "\n"; 3153 print REPORT ""; 3154 print REPORT "This report was generated with "; 3155 print REPORT "<A HREF="; 3156 print REPORT "\"http://awsd.com/scripts/weblog/\">"; 3157 print REPORT "WebLog $version</A>\n"; 3158 if ($footerfile) { &Footer; } 3159 print REPORT "</BODY></HTML>\n"; 3160 close (REPORT); 3161 } 3162 3163 sub Simplify { 3164 $url = $_[0]; 3165 $url =~ s/ //g; 3166 if ($url =~ m#(^.+)//([\w|\.|-]+)/(.+)#o) { 3167 $url_prefix = $1; 3168 $url_domain = $2; 3169 $url_path = $3; 3170 $url_domain = "\L$url_domain"; 3171 if ($RefsStripWWW) { 3172 $url_domain =~ s/^www\.//i; 3173 } 3174 return $url_prefix."//".$url_domain."/".$url_path; 3175 } 3176 elsif ($url =~ m#(^.+)//([\w|\.|-]+)#o) { 3177 $url_prefix = $1; 3178 $url_domain = $2; 3179 $url_domain = "\L$url_domain"; 3180 if ($RefsStripWWW) { 3181 $url_domain =~ s/^www\.//i; 3182 } 3183 return $url_prefix."//".$url_domain; 3184 } 3185 else { 3186 return $url; 3187 } 3188 } 3189 3190 sub ByRefCount { 3191 $refdomain{$b}<=>$refdomain{$a}; 3192 } 3193 3194 sub bytargetthenhits { 3195 ($targeta)=($a=~m#^(.+)\s#o); 3196 ($targetb)=($b=~m#^(.+)\s#o); 3197 $inequality=($targeta cmp $targetb); 3198 if ($inequality) { 3199 $inequality; 3200 } 3201 else { 3202 $TargetCounter{$b}<=>$TargetCounter{$a}; 3203 } 3204 } 3205 3206 sub PrintKeywordsReport { 3207 if ($Verbose) { print " Generating Keywords Report\n"; } 3208 if (-e "$FileDir/$KeywordsFile") { 3209 open (OLDLOG,"$FileDir/$KeywordsFile") || die " Error Opening File: $KeywordsFile\n"; 3210 while (<OLDLOG>) { 3211 chomp; 3212 if (m#and directories, since (.*).#o) { $keysstartdate = $1; } 3213 if (m#<A HREF=[^>]*>(.*)</A>#o) { 3214 $SearchEngine = $1; 3215 } 3216 elsif (m#(.*)#o) { 3217 $SearchEngine = $1; 3218 } 3219 if (m#\s*([\d,]+) (.*)$#o ) { 3220 $count = $1; 3221 $phrase = $2; 3222 $count =~ s/,//g; 3223 if ($SearchEngine eq "About:") { 3224 if ($aboutcom{$phrase}) { $aboutcom{$phrase} += $count; } 3225 else { $aboutcom{$phrase} = $count; } 3226 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3227 else { $topkeywords{$phrase} = $count; } 3228 } 3229 elsif ($SearchEngine eq "AltaVista:") { 3230 if ($altavista{$phrase}) { $altavista{$phrase} += $count; } 3231 else { $altavista{$phrase} = $count; } 3232 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3233 else { $topkeywords{$phrase} = $count; } 3234 } 3235 elsif ($SearchEngine eq "AOL Netfind:") { 3236 if ($netfind{$phrase}) { $netfind{$phrase} += $count; } 3237 else { $netfind{$phrase} = $count; } 3238 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3239 else { $topkeywords{$phrase} = $count; } 3240 } 3241 elsif ($SearchEngine eq "Ask Jeeves:") { 3242 if ($askjeeves{$phrase}) { $askjeeves{$phrase} += $count; } 3243 else { $askjeeves{$phrase} = $count; } 3244 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3245 else { $topkeywords{$phrase} = $count; } 3246 } 3247 elsif ($SearchEngine eq "c|net search.com:") { 3248 if ($cnet{$phrase}) { $cnet{$phrase} += $count; } 3249 else { $cnet{$phrase} = $count; } 3250 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3251 else { $topkeywords{$phrase} = $count; } 3252 } 3253 elsif ($SearchEngine eq "Direct Hit:") { 3254 if ($directhit{$phrase}) { $directhit{$phrase} += $count; } 3255 else { $directhit{$phrase} = $count; } 3256 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3257 else { $topkeywords{$phrase} = $count; } 3258 } 3259 elsif ($SearchEngine eq "Dogpile:") { 3260 if ($dogpile{$phrase}) { $dogpile{$phrase} += $count; } 3261 else { $dogpile{$phrase} = $count; } 3262 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3263 else { $topkeywords{$phrase} = $count; } 3264 } 3265 elsif ($SearchEngine eq "EuroFerret:") { 3266 if ($euroferret{$phrase}) { $euroferret{$phrase} += $count; } 3267 else { $euroferret{$phrase} = $count; } 3268 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3269 else { $topkeywords{$phrase} = $count; } 3270 } 3271 elsif ($SearchEngine eq "EuroSeek:") { 3272 if ($euroseek{$phrase}) { $euroseek{$phrase} += $count; } 3273 else { $euroseek{$phrase} = $count; } 3274 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3275 else { $topkeywords{$phrase} = $count; } 3276 } 3277 elsif ($SearchEngine =~ /Excite/) { 3278 if ($excite{$phrase}) { $excite{$phrase} += $count; } 3279 else { $excite{$phrase} = $count; } 3280 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3281 else { $topkeywords{$phrase} = $count; } 3282 } 3283 elsif ($SearchEngine =~ /FAST/) { 3284 if ($fast{$phrase}) { $fast{$phrase} += $count; } 3285 else { $fast{$phrase} = $count; } 3286 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3287 else { $topkeywords{$phrase} = $count; } 3288 } 3289 elsif ($SearchEngine eq "Google:") { 3290 if ($google{$phrase}) { $google{$phrase} += $count; } 3291 else { $google{$phrase} = $count; } 3292 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3293 else { $topkeywords{$phrase} = $count; } 3294 } 3295 elsif ($SearchEngine eq "GoTo:") { 3296 if ($goto{$phrase}) { $goto{$phrase} += $count; } 3297 else { $goto{$phrase} = $count; } 3298 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3299 else { $topkeywords{$phrase} = $count; } 3300 } 3301 elsif ($SearchEngine eq "HotBot:") { 3302 if ($hotbot{$phrase}) { $hotbot{$phrase} += $count; } 3303 else { $hotbot{$phrase} = $count; } 3304 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3305 else { $topkeywords{$phrase} = $count; } 3306 } 3307 elsif ($SearchEngine =~ /Infoseek/) { 3308 if ($infoseek{$phrase}) { $infoseek{$phrase} += $count; } 3309 else { $infoseek{$phrase} = $count; } 3310 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3311 else { $topkeywords{$phrase} = $count; } 3312 } 3313 elsif ($SearchEngine eq "LookSmart:") { 3314 if ($looksmart{$phrase}) { $looksmart{$phrase} += $count; } 3315 else { $looksmart{$phrase} = $count; } 3316 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3317 else { $topkeywords{$phrase} = $count; } 3318 } 3319 elsif ($SearchEngine eq "Lycos:") { 3320 if ($lycos{$phrase}) { $lycos{$phrase} += $count; } 3321 else { $lycos{$phrase} = $count; } 3322 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3323 else { $topkeywords{$phrase} = $count; } 3324 } 3325 elsif ($SearchEngine eq "Magellan:") { 3326 if ($magellan{$phrase}) { $magellan{$phrase} += $count; } 3327 else { $magellan{$phrase} = $count; } 3328 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3329 else { $topkeywords{$phrase} = $count; } 3330 } 3331 elsif ($SearchEngine eq "Mamma:") { 3332 if ($mamma{$phrase}) { $mamma{$phrase} += $count; } 3333 else { $mamma{$phrase} = $count; } 3334 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3335 else { $topkeywords{$phrase} = $count; } 3336 } 3337 elsif ($SearchEngine =~ /Metacrawler/i) { 3338 if ($metacrawler{$phrase}) { $metacrawler{$phrase} += $count; } 3339 else { $metacrawler{$phrase} = $count; } 3340 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3341 else { $topkeywords{$phrase} = $count; } 3342 } 3343 elsif ($SearchEngine =~ /MSN/) { 3344 if ($msn{$phrase}) { $msn{$phrase} += $count; } 3345 else { $msn{$phrase} = $count; } 3346 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3347 else { $topkeywords{$phrase} = $count; } 3348 } 3349 elsif ($SearchEngine eq "Netscape Search:") { 3350 if ($netscape{$phrase}) { $netscape{$phrase} += $count; } 3351 else { $netscape{$phrase} = $count; } 3352 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3353 else { $topkeywords{$phrase} = $count; } 3354 } 3355 elsif ($SearchEngine eq "Northern Light:") { 3356 if ($northernlight{$phrase}) { $northernlight{$phrase} += $count; } 3357 else { $northernlight{$phrase} = $count; } 3358 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3359 else { $topkeywords{$phrase} = $count; } 3360 } 3361 elsif ($SearchEngine eq "PlanetSearch:") { 3362 if ($planetsearch{$phrase}) { $planetsearch{$phrase} += $count; } 3363 else { $planetsearch{$phrase} = $count; } 3364 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3365 else { $topkeywords{$phrase} = $count; } 3366 } 3367 elsif ($SearchEngine eq "SavvySearch:") { 3368 if ($savvysearch{$phrase}) { $savvysearch{$phrase} += $count; } 3369 else { $savvysearch{$phrase} = $count; } 3370 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3371 else { $topkeywords{$phrase} = $count; } 3372 } 3373 elsif (($SearchEngine eq "Snap:") || ($SearchEngine eq "NBCi:")) { 3374 if ($snap{$phrase}) { $snap{$phrase} += $count; } 3375 else { $snap{$phrase} = $count; } 3376 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3377 else { $topkeywords{$phrase} = $count; } 3378 } 3379 elsif ($SearchEngine eq "WebCrawler:") { 3380 if ($webcrawler{$phrase}) { $webcrawler{$phrase} += $count; } 3381 else { $webcrawler{$phrase} = $count; } 3382 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3383 else { $topkeywords{$phrase} = $count; } 3384 } 3385 elsif ($SearchEngine eq "Yahoo:") { 3386 if ($yahoo{$phrase}) { $yahoo{$phrase} += $count; } 3387 else { $yahoo{$phrase} = $count; } 3388 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3389 else { $topkeywords{$phrase} = $count; } 3390 } 3391 elsif ($SearchEngine eq "Other Search Engines:") { 3392 if ($othersearch{$phrase}) { $othersearch{$phrase} += $count; } 3393 else { $othersearch{$phrase} = $count; } 3394 if ($topkeywords{$phrase}) { $topkeywords{$phrase} += $count; } 3395 else { $topkeywords{$phrase} = $count; } 3396 } 3397 } 3398 } 3399 close (OLDLOG); 3400 } 3401 open (REPORT,">$FileDir/$KeywordsFile") || die " Error Opening File: $KeywordsFile\n"; 3402 print REPORT "<HTML><HEAD><TITLE>Keywords Report: $SystemName</TITLE></HEAD>"; 3403 print REPORT "<BODY $bodyspec>\n"; 3404 if ($headerfile) { &Header; } 3405 print REPORT "<H1 ALIGN=CENTER>Referring Keywords: $SystemName</H1>\n"; 3406 unless ($EndDate eq "0000 00 00 00 00 00") { 3407 print REPORT "(Accesses Through "; 3408 ($Year,$Month,$Day,$Hour,$Minute,$Second)= 3409 split(/ /,$EndDate); 3410 print REPORT "$Hour:$Minute:$Second $Day "; 3411 print REPORT "$NumberToMonth{$Month} $Year)\n"; 3412 } 3413 print REPORT "| <A HREF=\"$ReportFile\">Overall Activity Report</A> |"; 3414 if ($DetailsFile) { 3415 print REPORT " <A HREF=\"$DetailsFile\">$AccessDetailsReport</A> |"; 3416 } 3417 if ($RefsFile) { 3418 print REPORT " <A HREF=\"$RefsFile\">Referring URLs Report</A> |"; 3419 } 3420 if ($AgentsFile) { 3421 print REPORT " <A HREF=\"$AgentsFile\">Agents/Platforms Report</A> |"; 3422 } 3423 print REPORT "<HR>\n"; 3424 unless ($keysstartdate) { 3425 ($Year,$Month,$Day,$Hour,$Minute,$Second) = split(/ /,$FileStartDate); 3426 $Day = int($Day); 3427 $keysstartdate = "$Day $NumberToMonth{$Month} $Year"; 3428 } 3429 print REPORT "This report logs the keywords used by visitors to find "; 3430 print REPORT "this site in the various Internet search engines and directories, "; 3431 print REPORT "since $keysstartdate. The major search engines "; 3432 print REPORT "are each listed individually.\n"; 3433 print REPORT "<HR>\n"; 3434 if ($TopNKeywords) { 3435 print REPORT "<H2 ALIGN=CENTER>Top $TopNKeywords Keywords</H2>\n"; 3436 print REPORT "<PRE>\n"; 3437 $Counter=1; 3438 foreach $key (sort ByKeyCount keys(%topkeywords)) { 3439 last if ($Counter > $TopNKeywords); 3440 printf REPORT "%32s%-s\n",&commas($topkeywords{$key})," $key"; 3441 $Counter++; 3442 } 3443 print REPORT "</PRE><HR>\n\n"; 3444 } 3445 $RefsMinHits = 0; 3446 if ($RefsFilterLists) { 3447 $Counter=0; 3448 foreach $key (sort ByKeyCount keys(%topkeywords)) { 3449 $Counter++; 3450 $RefsMinHits += $topkeywords{$key}; 3451 last if ($Counter > 24); 3452 } 3453 if ($Counter == 0) { $RefsMinHits = 0; } 3454 else { $RefsMinHits = (($RefsMinHits/$Counter)/25); } 3455 } 3456 print REPORT "<H2 ALIGN=CENTER>Referring Keywords</H2>\n"; 3457 if (%aboutcom) { &PrintSearchPhrases("About","www.about.com",%aboutcom); } 3458 if (%altavista) { &PrintSearchPhrases("AltaVista","www.altavista.com",%altavista); } 3459 if (%netfind) {&PrintSearchPhrases("AOL Netfind","search.aol.com",%netfind); } 3460 if (%askjeeves) {&PrintSearchPhrases("Ask Jeeves","www.askjeeves.com",%askjeeves); } 3461 if (%cnet) {&PrintSearchPhrases("c|net search.com","www.search.com",%cnet); } 3462 if (%directhit) {&PrintSearchPhrases("Direct Hit","www.directhit.com",%directhit); } 3463 if (%dogpile) {&PrintSearchPhrases("Dogpile","www.dogpile.com",%dogpile); } 3464 if (%euroferret) {&PrintSearchPhrases("EuroFerret","www.euroferret.com",%euroferret); } 3465 if (%euroseek) {&PrintSearchPhrases("EuroSeek","www.euroseek.net",%euroseek); } 3466 if (%excite) {&PrintSearchPhrases("Excite","www.excite.com",%excite); } 3467 if (%fast) {&PrintSearchPhrases("FAST (All the Web, All the Time)","www.alltheweb.com",%fast); } 3468 if (%hotbot) {&PrintSearchPhrases("HotBot","www.hotbot.com",%hotbot); } 3469 if (%google) {&PrintSearchPhrases("Google","www.google.com",%google); } 3470 if (%goto) {&PrintSearchPhrases("GoTo","www.goto.com",%goto); } 3471 if (%infoseek) {&PrintSearchPhrases("Infoseek (Go Network)","www.go.com",%infoseek); } 3472 if (%looksmart) {&PrintSearchPhrases("LookSmart","www.looksmart.com",%looksmart); } 3473 if (%lycos) {&PrintSearchPhrases("Lycos","www.lycos.com",%lycos); } 3474 if (%magellan) {&PrintSearchPhrases("Magellan","magellan.excite.com",%magellan); } 3475 if (%mamma) {&PrintSearchPhrases("Mamma","www.mamma.com",%mamma); } 3476 if (%metacrawler) {&PrintSearchPhrases("MetaCrawler (Go2Net)","www.go2net.com",%metacrawler); } 3477 if (%msn) {&PrintSearchPhrases("MSN (Microsoft Network)","search.msn.com",%msn); } 3478 if (%snap) {&PrintSearchPhrases("NBCi","www.nbci.com",%snap); } 3479 if (%netscape) {&PrintSearchPhrases("Netscape Search","search.netscape.com",%netscape); } 3480 if (%northernlight) {&PrintSearchPhrases("Northern Light","www.northernlight.com",%northernlight); } 3481 if (%planetsearch) {&PrintSearchPhrases("PlanetSearch","www.planetsearch.com",%planetsearch); } 3482 if (%savvysearch) {&PrintSearchPhrases("SavvySearch","www.savvysearch.com",%savvysearch); } 3483 if (%webcrawler) {&PrintSearchPhrases("WebCrawler","www.webcrawler.com",%webcrawler); } 3484 if (%yahoo) {&PrintSearchPhrases("Yahoo","www.yahoo.com",%yahoo); } 3485 if (%othersearch) {&PrintSearchPhrases("Other Search Engines","x",%othersearch); } 3486 print REPORT "<HR>\n"; 3487 &PrintKeywordsListKey; 3488 print REPORT "| <A HREF=\"$ReportFile\">Overall Activity Report</A> |"; 3489 if ($DetailsFile) { 3490 print REPORT " <A HREF=\"$DetailsFile\">$AccessDetailsReport</A> |"; 3491 } 3492 if ($RefsFile) { 3493 print REPORT " <A HREF=\"$RefsFile\">Referring URLs Report</A> |"; 3494 } 3495 if ($AgentsFile) { 3496 print REPORT " <A HREF=\"$AgentsFile\">Agents/Platforms Report</A> |"; 3497 } 3498 print REPORT "\n"; 3499 print REPORT ""; 3500 print REPORT "This report was generated with "; 3501 print REPORT "<A HREF="; 3502 print REPORT "\"http://awsd.com/scripts/weblog/\">"; 3503 print REPORT "WebLog $version</A>\n"; 3504 if ($footerfile) { &Footer; } 3505 print REPORT "</BODY></HTML>\n"; 3506 close (REPORT); 3507 } 3508 3509 sub ByKeyCount { 3510 $topkeywords{$b}<=>$topkeywords{$a}; 3511 } 3512 3513 sub PrintSearchPhrases { 3514 local ($title,$url,%keyphrases) = @_; 3515 foreach $phrase (sort PhrasesByHits keys(%keyphrases)) {