com.norconex.collector.http.robot
Class RobotsTxt

java.lang.Object
  extended by com.norconex.collector.http.robot.RobotsTxt
All Implemented Interfaces:
Serializable

public class RobotsTxt
extends Object
implements Serializable

See Also:
Serialized Form

Field Summary
static float UNSPECIFIED_CRAWL_DELAY
           
 
Constructor Summary
RobotsTxt(IURLFilter[] filters)
           
RobotsTxt(IURLFilter[] filters, float crawlDelay)
           
 
Method Summary
 float getCrawlDelay()
           
 IURLFilter[] getFilters()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

UNSPECIFIED_CRAWL_DELAY

public static final float UNSPECIFIED_CRAWL_DELAY
See Also:
Constant Field Values
Constructor Detail

RobotsTxt

public RobotsTxt(IURLFilter[] filters)

RobotsTxt

public RobotsTxt(IURLFilter[] filters,
                 float crawlDelay)
Method Detail

getFilters

public IURLFilter[] getFilters()

getCrawlDelay

public float getCrawlDelay()


Copyright © 2009-2013 Norconex Inc.. All Rights Reserved.