What would be a good way to extract headlines, dates, and authors from news articles? It seems easy to write a scraper using xpath or similar to extract this information from a single site, but I'm not sure of a more scalable solution if you're extracting from say 10,000 sites.
This is the project page for SecondString, an open-source Java-based package of approximate string-matching techniques. This code was developed by researchers at Carnegie Mellon University from the Center for Automated Learning and Discovery, the Department of Statistics, and the Center for Computer and Communications Security.
SecondString is intended primarily for researchers in information integration and other scientists. It does or will include a range of string-matching methods from a variety of communities, including statistics, artificial intelligence, information retrieval, and databases. It also includes tools for systematically evaluating performance on test data. It is not designed for use on very large data sets.
TeSSI® (Terminology Supported Semantic Indexing) is a state-of-the-art tool that improves upon the existing search and retrieval tools by extracting the meaning out of medical free text and placing the resulting medical ‘concepts’ in the document...
M. Hearst. Proceedings of the 14th Conference on Computational Linguistics - Volume 2, Seite 539--545. Stroudsburg, PA, USA, Association for Computational Linguistics, (1992)
P. Kluegl, M. Atzmueller, und F. Puppe. Proc. LWA 2009, Knowledge Discovery and Machine Learning Track, Darmstadt, Germany, University of Darmstadt, (2009)
P. Kluegl, M. Toepfer, P. Beck, G. Fette, und F. Puppe. Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: System Demonstrations, Seite 29--33. Dublin, Ireland, Dublin City University and Association for Computational Linguistics, (August 2014)
Y. Chan, und D. Roth. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, Seite 551--560. Stroudsburg, PA, USA, Association for Computational Linguistics, (2011)