<property> <name>http.agent.name</name> <value></value> <description>HTTP 'User-Agent' request header. MUST NOT be empty - please set this to a single word uniquely related to your organization. NOTE: You should also check other related properties: http.robots.agents http.agent.description http.agent.url http.agent.email http.agent.version and set their values appropriately. </description> </property> <property> <name>http.agent.description</name> <value></value> <description>Further description of our bot- this text is used in the User-Agent header. It appears in parenthesis after the agent name. </description> </property> <property> <name>http.agent.url</name> <value></value> <description>A URL to advertise in the User-Agent header. This will appear in parenthesis after the agent name. Custom dictates that this should be a URL of a page explaining the purpose and behavior of this crawler. </description> </property> <property> <name>http.agent.email</name> <value></value> <description>An email address to advertise in the HTTP 'From' request header and User-Agent header. A good practice is to mangle this address (e.g. 'info at example dot com') to avoid spamming. </description> </property>
P. Li, J. Nie, B. Wang, and J. He. Web Intelligence and Intelligent Agent Technology (WI-IAT), 2012 IEEE/WIC/ACM International Conferences on, 1, page 274-281. (December 2012)
C. Trattner, D. Helic, P. Singer, and M. Strohmaier. Proceedings of the 12th International Conference on Knowledge Management and Knowledge Technologies, page 14. ACM, (2012)
F. Gey. Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, page 222--231. New York, NY, USA, Springer-Verlag New York, Inc., (1994)
G. Dupret, and C. Liao. Proceedings of the third ACM international conference on Web search and data mining, page 181--190. New York, NY, USA, ACM, (2010)