Distributed IR Testbed Definitions:

• trec123-100-sample300-callan99.v1a: (115 KB gzipped, 650 KB uncompressed):

  • Description: A 100 collection testbed created by sampling trec123-100-bysource-callan99.v2a.
  • Publication: In progress.

• trec123-100-bysource-callan99.v2a: (3 MB gzipped, 23 MB uncompressed):

  • Description: A 100 collection testbed created from TREC CDs 1, 2, and 3.
  • Publication: A.L. Powell, J.C. French, J. Callan, M. Connell, and C.L. Viles, "The impact of database selection on distributed searching." In Proceedings of the 23rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 00), Athens, 2000.

• trec123-100-bysource-callan99.v1b: (3 MB gzipped, 23 MB uncompressed):

  • Description: A 100 collection testbed created from TREC CDs 1, 2, and 3. 161 AP '90 documents from CD 3 were omitted inadvertently.
  • Publication: J. French, A. Powell, J. Callan, C. Viles, T. Emmitt, K. Prey, and Y. Mou, "Comparing the performance of database selection algorithms." In the Proceedings of the 22nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 99), Berkeley, CA, August 15-19, 1999.

• trecvlc1-921-bysource-callan99 (18 MB gzipped, 260 MB uncompressed):

  • Description: A 921 collection testbed created from the first TREC VLC corpus.
  • Authors: J. Callan and Y. Mou.
  • Publication: J. French, A. Powell, J. Callan, C. Viles, T. Emmitt, K. Prey, and Y. Mou, "Comparing the performance of database selection algorithms." In the Proceedings of the 22nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 99), Berkeley, CA, August 15-19, 1999.

• trec4-bysource-xu99: (1.5 MB gzipped, 11.5 MB uncompressed):

  • Description: A 100 collection testbed created from TREC-4 data.
  • Authors: J. Xu and W.B. Croft.
  • Publication: J. Xu and W.B. Croft, "Cluster-based language models for distributed retrieval." In Proceedings of the 22nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 99), 1999.

• trec4-kmeans-xu99: (2.2 MB gzipped, 11.9 MB uncompressed):

  • Description: A 100 collection testbed created from TREC-4 data, in which collections were created by clustering documents (i.e., documents in each collection are on roughly the same topic).
  • Authors: J. Xu and W.B. Croft.
  • Publication: J. Xu and W.B. Croft, "Cluster-based language models for distributed retrieval." In Proceedings of the 22nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 99), 1999.

• trec6-bysource-xu99: (1.4 MB gzipped, 10.5 MB uncompressed):

  • Description: A 100 collection testbed created from TREC-6 data.
  • Authors: J. Xu and W.B. Croft.
  • Publication: J. Xu and W.B. Croft, "Cluster-based language models for distributed retrieval." In Proceedings of the 22nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 99), 1999.

• trec6-kmeans-xu99: (2 MB gzipped, 11 MB uncompressed):

  • Description: A 100 collection testbed created from TREC-6 data, in which collections were created by clustering documents (i.e., documents in each collection are on roughly the same topic).
  • Authors: J. Xu and W.B. Croft.
  • Publication: J. Xu and W.B. Croft, "Cluster-based language models for distributed retrieval." In Proceedings of the 22nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 99), 1999.

• trec123-17-bysource-callan99.v1a: (?? MB gzipped, ?? MB uncompressed):

  • Description: A 17 collection testbed created from TREC CDs 1, 2, and 3.
  • Authors: J. Callan and Z. Lu.
  • Publication: J. P. Callan, Z. Lu and W. B. Croft. "Searching distributed collections with inference networks." In the Proceedings of the 18th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 95), Seattle, WA, 1995.

Public Comment Data:

• See the eRulemaking Testbed Web page.

Peer-to-Peer Testbed Definitions:

• trecwt10g-2500-bysource-v1.gz (digital libraries, 43 MB uncompressed, 4 MB compressed),
  trecwt10g-query-bydoc.v1.gz (queries, 99 MB uncompressed, 21 MB compressed):

  • Description: Data definitions for a peer-to-peer network of 2,500 small digital libraries, and a set of 1.6 million known-item queries that can be used with them. The data is derived from NIST's publicly available TREC WT10g dataset.
  • Authors: J. Lu and J. Callan.
  • Publication: J. Lu, and J. Callan. "Content-based retrieval in hybrid peer-to-peer networks." In Proceedings of the Twelfth International Conference on Information and Knowledge Management (CIKM'03). New Orleans: ACM. 2003.

• trecgov2-25000-bysource.v1.gz (digital libraries, 76 MB compressed, 528 MB compressed):

  • Description: Data definitions for a peer-to-peer network of 25,000 digital libraries of various sizes. The data is derived from NIST's publicly available TREC gov2 dataset.
  • Authors: J. Lu and J. Callan.
  • Publication: J. Lu. "Full-text federated search in peer-to-peer networks." PhD dissertation, Language Technologies Institute, Carnegie Mellon University. In preparation.

Novelty Data:

• trec123-ap-wsj.v1

  • Description: A one gigabyte dataset of AP News and Wall Street Journal data from TREC CDs 1, 2, and 3. Includes novelty assessments for 33 TREC profiles.
  • Authors: Y. Zhang and J. Callan.
  • Publication: Y. Zhang, J. Callan, and T. Minka. (2002.) "Novelty and redundancy detection in adaptive filtering." In Proceedings of the Twenty Fifth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 81-88). Tampere, Finland: ACM.

Table Search Datasets:

• TableArXiv

  • Description: A dataset created to support research on scientific table retrieval. It uses 341,573 tables extracted from physics e-prints on arXiv.org, 105 information needs and corresponding relevance judgements in TREC format.
  • Authors: K. Y. Gao and J. Callan.

Web Data:

• clueweb09

  • Description: A 25 terabyte dataset of about 1 billion web pages crawled in January and February, 2009. The crawl order was best-first search, using the OPIC metric. The crawl was started from about 25 million URLs that either i) had high OPIC values in a web graph produced from an earlier 200 million page crawl, or ii) were ranked highly by a commercial search engine for one of 16,000 sample queries in one of 10 languages. This dataset covers web content in English, Chinese, Spanish, Japanese, French, German, Arabic, Portuguese, Korean, and Italian.
  • Creators: J. Callan, M. Hoy, C. Yoo, and L. Zhao.

• clueweb12

  • Description: A 27 terabyte dataset of about 733 million web pages crawled in early 2012. The default heritrix crawl order was used. The crawl was started from about 2.8 million URLs. The seed set included 262 seeds that represented popular sites in English-speaking countries, 5,950 seeds related to travel sites, and URLs of ClueWeb09 pages that had high PageRank and low Waterloo spam scores. Non-English pages and pages on a pornography, malware, and phishing blacklist were removed.
  • Creators: J. Callan and D. Pane.