Search Engines:
11-442 / 11-642
CMU logo
Course Description: This lecture-oriented course studies the theory, design, and implementation of text-based search engines. The core components include statistical characteristics of text, representation of information needs and documents, several important retrieval models, and experimental evaluation. The course also covers common elements of commercial search engines, for example, integration of diverse search engines into a single search service (federated search, vertical search), personalized search results, diverse search results, and sponsored search. The software architecture components include design and implementation of large-scale, distributed search engines.

This is a full-semester lecture-oriented course worth 12 units.

Beginning in Fall 2018, the undergraduate section (11-442) will be 9 units. The difference between 11-442 and 11-642 will the amount of analysis, writing, and time required to complete homework assignments.
Learning Objectives: By the end of the course, students are expected to have developed the skills listed below.
  • Recall and discuss well-known search engine architectures, methods of representing text documents, methods of representing information needs, and methods of evaluating search effectiveness;
  • Implement well-known retrieval algorithms and test them on standard datasets; and
  • Apply information retrieval techniques discussed in class to solve problems faced by governments and companies.
Skills are assessed by the homework assignments; and by midterm and final exams.
Eligibility: This course is open to all students who meet the prerequisites.
Prerequisites: This course requires good programming skills and an understanding of computer architectures and operating systems (e.g., memory vs. disk trade-offs). A basic understanding of probability, statistics, and linear algebra is helpful. Thus students should have preparation comparable to the following CMU undergraduate courses.
  • 15-210, Parallel and Sequential Data Structures and Algorithms (required)
  • 15-213, Introduction to Computer Systems (required)
  • 15-451, Algorithm Design and Analysis (helpful)
  • 21-241, Matrices and Linear Transformations or 21-341, Linear Algebra (required)
  • 21-325, Probability (required)
  • 36-202, Statistics & Data Science Methods (helpful)
Time & Location: Tu/Th 1:30-2:50, DH A302
Instructor: Jamie Callan
Teaching Assistants:
Akshat Gaur (agaur@andrew) Autograding
Anusha Prakash (anushap@andrew) Reading summaries
Anuva Agarwal (anuvaa@andrew) Reading summaries
Danqi Huang (danqih@andrew)
Qinyu Hu (qinyuh@andrew)
Srividya Potharaju (spothara@andrew)
Weizi Wang (weiziw@andrew)
Zi Liang (zliang1@andrew)
Office hours:
Day Time Location TA
Monday 11:00-12:30
GHC 5417
GHC 5417
Tuesday 10:30-12:00 GHC 5417 Srividya
Thursday    4:30-6:00 GHC 5417 Zi
Friday    4:30-6:00 GHC 5417 Weizi
Course Materials: The textbook is Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schutze, Cambridge University Press. 2008. You may use the printed copy or the online copy, but note that the reading instructions refer to the printed copy.

There are additional selected readings, which will be available through the class web page (this page).

Online access to some materials (additional readings, lecture notes, datasets, etc) is restricted to the domain. CMU people can get access from outside (e.g., from home) using CMU's WebVPN Service.

A discussion forum is provided for students to ask questions, answer questions, and discuss class-related topics. You must register yourself to access the discussion forum. Please provide a CMU email address when you join the discussion (you can use other email addresses, too). We will periodically remove students that do not have CMU email addresses.
Homework: 5 assignments that give hands-on experience with techniques discussed in class. Homework must be done individually, and students may not share their work with other students. See the course Academic Integrity policy for more information.
Grading: Weekly reading summaries (10% total), 5 homework assignments (10% each, 50% total), midterm exam (20%), final exam (20%).
Grading Scale: Grades are assigned using a curve. Typically the median GPA is about 3.5.
Course policies: Academic Integrity, Attendance, Auditing, Laptops & mobile devices, Late homework, Pass/Fail, Recording & videotaping, Waitlist
(subject to revision):
Date Topic Readings
Jan 16, Course overview (pdf, mp4)
Jan 18, Introduction to search: Exact-match retrieval (pdf, mp4)
Reading summaries (pdf, mp4)
Ch 1, Ch 5.1
Jan 23, Introduction to search: Query processing (pdf, mp4)
Software development requirements (pdf)
HW1 out
Ch 2.4
Jan 25, Introduction to search: QryEval (pdf, mp4)  
Jan 30, Evaluating search effectiveness (pdf, mp4) Ch 8-8.5
Feb 1, Evaluating search effectiveness (pdf, mp4)  
Feb 6, Best-match retrieval: VSM, BM25 (pdf, mp4)
HW1 due, HW2 out
Ch 6.0, 6.2-6.4.2, 11.4
Feb 8, Best-match retrieval: Language models (pdf, mp4)
HW2 implementation notes (pdf)
Ch 12
Feb 13, Query structure: Information needs and queries (pdf, mp4)
(recorded lecture - no class)
Nguyen & Callan, 2011
Feb 15, Document representation (pdf) (mp4 I, mp4 II)
(recorded lecture - no class)
Ch 2-2.2
Feb 20, Query structure: Relevance and pseudo relevance feedback (pdf, mp4)
HW2 due, HW3 out
Ch 9
Feb 22, Index creation (pdf, mp4) Ch 4
Feb 27, Index creation (pdf, mp4) Ch 7
Mar 1, Large-scale indexes (pdf, mp4)
Mar 6, Document structure (pdf, mp4) Ch 10
Mar 8, Midterm Exam Sample Midterm 1, Sample Midterm 2
Mar 20, Ranked retrieval: Feature-based models (pdf, mp4 I, mp4 II)
HW3 due, HW4 out
Clarke Ch 11.7; Li, 2011
Mar 22, Authority metrics (pdf, mp4) Ch 21
Mar 27, Page quality, web spam (pdf, mp4)
(recorded lecture - no class)
Mar 29, Ranked retrieval: Neural models (pdf, mp4) Guo, et al, 2016  
Apr 3, Diversity (pdf, mp4)
HW4 due, HW5 out
Santos, Ch 1-3
Apr 5, Diversity (pdf, mp4) Santos, Ch 4-5
Apr 10, Diversity (pdf, mp4)
Neural networks revisited (pdf, mp4)
Apr 12, Search log analysis (pdf, mp4) Eickhoff et al, 2014
Apr 17, Search log analysis (pdf, mp4)
HW5 due
Bennett et al, 2012
Apr 24, Search log analysis (pdf, mp4)
Personalization (pdf, mp4)
Apr 26, Federated, aggregated, & vertical search (pdf, mp4) Si & Callan, 2003, Arguello & Diaz, 2013
May 1, Enterprise search  
May 3, Final exam Sample final
Accommodations for Students with Disabilities: If you have a disability and have an accommodations letter from the Disability Resources office, please discuss your accommodations and needs with the instructor as early in the semester as possible. The instructor will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, please contact them at
Advice From The Faculty: This course is a lot of work. Take care of yourself. Do your best to maintain a healthy lifestyle this semester by eating well, exercising, avoiding drugs and alcohol, getting enough sleep and taking some time to relax. This will help you achieve your goals and cope with stress.

If you find yourself struggling with the material or workload, please ask for help. All of us benefit from support during times of struggle. You are not alone. There are many helpful resources available on campus and an important part of the college experience is learning how to ask for help. Asking for support sooner rather than later is often helpful.

If you or anyone you know experiences any academic stress, difficult life events, or feelings like anxiety or depression, we strongly encourage you to seek support. Counseling and Psychological Services (CaPS) is here to help: call 412-268-2922 and visit their website at Consider reaching out to a friend, faculty or family member you trust for help getting connected to the support that can help.

Copyright 2018, Carnegie Mellon University.
Updated on April 26, 2018
Jamie Callan