Results 1 to 1 of 1

Thread: How to handle Session Management - Bots and Spiders

  1. #1

    Default How to handle Session Management - Bots and Spiders

    First and foremost we suggest creating a robots.txt file in the web root of the domain to address two issues. First to control the rate at which the website is being crawled which can help prevent a bot/spider from creating a massive number of database connections at the same time. Second to prevent specific bots from crawling the website. We suggest the following defaults, however you might want to add or remove the user agents denied, and adjust the crawl rate but we suggest nothing lower than 3 seconds.

    Code:
    User-agent: *
    Crawl-delay: 10
    
    User-agent: Baiduspider
    Disallow: /
    
    User-agent: Sosospider
    Disallow: /
    Next we suggest setting your session timeout specifically lower for bots and spiders. These spiders and bots will crawl a page and when a session (ColdFusion) is created, it will persist during then entire page load. The page fully loaded allows the bot or spider to get the information from the webpage AND allows the session to expire quickly protecting ColdFusion from effects similar to a memory leak.

    Session Management code examples for the Applicaiton.CFM

    Application.CFC code instructions below, adjusting the timeout to your applications requirements:

    Replace within a cfscript block:
    Code:
    THIS.sessionTimeout = createTimeSpan(0,0,60,0);
    With:
    Code:
    <!--- This checks if a cookie is created, for bots this will return false and use the low session timeout --->
    if (StructKeyExists(cookie, "cfid")){
    THIS.sessionTimeout 	= createTimeSpan(0,0,60,0);
    } else {
    THIS.sessionTimeout 	= createTimeSpan(0,0,0,2);
    }
    Last edited by JonC; March 30th, 2012 at 12:18 PM.
    Jon Cavanaugh
    ColdFusion Systems Analysts
    Director of Business Development
    Linux Hosting | Christian Hosting | Railo Hosting

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •