Week 12 - Intelligent Information Retrieval - Muddiest Point

I do not have a muddiest point for this week.

Week 12 - Intelligent Information Retrieval - Reading Note

This week I read,

1. Susan Gauch, Mirco Speretta, Aravind Chandramouli and Alessandro Micarelli. User Profiles for Personalized Information Access. Chapter 2 in  Brusilovsky, P., Kobsa, A., Neidl, W. (eds.) (2007) The Adaptive Web: Methods and Strategies of Web Personalization. Lecture Notes in Computer Science, Vol. 4321. Springer-Verlag, Berlin Heidelberg New York.
2. Michael J. Pazzani and Daniel Billsus. Content-Based Recommendation Systems. Chapter 10 of  Brusilovsky, P., Kobsa, A., Neidl, W. (eds.) (2007) The Adaptive Web: Methods and Strategies of Web Personalization. Lecture Notes in Computer Science, Vol. 4321. Springer-Verlag, Berlin Heidelberg New York.
3. Ahn, J., Brusilovsky, P., He, D., Grady, J., and Li, Q.  (2008). Personalized web exploration with task models. In Proceeding of the 17th international Conference on World Wide Web (Beijing, China, April 21 - 25, 2008). WWW '08. ACM, New York, NY, 1-10. DOI= http://doi.acm.org/10.1145/1367497.1367499.
4. He, D., Brusiloviksy, P., Grady, J., Li, Q., and Ahn, J. (2007). How Up-to-date should it be? the Value of Instant Profiling and Adaptation in Information Filtering. In Proceedings of the IEEE/WIC/ACM international Conference on Web intelligence(November 02 - 05, 2007). Web  Intelligence. IEEE Computer Society, Washington, DC, 699-705. DOI= http://dx.doi.org/10.1109/WI.2007.135

From 1, user profiling is a way to narrow the scope of information avaliable based on some known information about the user. However, this requires the system to collect information about users, a potential privacy concern. Many people choose to opt out of these systems when given a chance.

From 2, content based recommendation systems are used to pair down the set of results to just the best to show a user. Think Netflix recommendations. Recommendations are usually driven from a user profile. 

Many techniques for making recommendations.

From 3, an alternate model for personalized search is task-based search. System attempts to gather information about the task, or information need, that the user is currently exploring over several searches. 

From 4, examines how often to refresh personalized results and user profiles. Real-time as a user is searching or just occasionally. Research shows that the real-time updates allow users to complete tasks faster. However, the speed comes at the cost of accuracy and user satisfaction.

Week 11 - Multilingual and Parallel Retrieval - Muddiest Point

I did not have a muddiest point this week.

Week 11 - Multilingual and Parallel Retrieval - Reading Note

This week I read:

1. Oard, D.W. and Diekema, A.R.  (1998)  Cross-Language Information Retrieval. Annual Review of Information Science and Technology, 33. 223-256.
2. Daqing He, Jianqiang Wang.  (2008)  Cross-Language Information Retrieval. In Ayse Goker, John Davies, Margaret Graham (eds) Information Retrieval, Wiley Publisher.
3. IES chapter 14 parallel information retrieval http://www.ir.uwaterloo.ca/book/14-parallel-information-retrieval.pdf

I was unable to obtain a copy of 1 online.
I was unable to obtain a copy of 2 online.

From 3, query processing can be parallelized.

• index partitioning - document vs. term partitioning - term partitioning allows for greater scalability by allowing the index to stay in memory - hybrid approach partitions by document and then by term
• replication - helps provide reliability and fault tolerance in addition to scaling benefits

MapReduce can be used to parallelize other tasks such as index building, link analysis, etc. MapReduce is able to handle massive collections of processors and huge data sets. Maps and Reduces can be sharded into smaller sets.

Week 10 - Web Information Retrieval - Muddiest Point

Why is HITS more susceptible to spamming than PageRank?

Week 10 - Web Information Retrieval - Reading Note

This week I read:
1. Christopher D. Manning, Prabhakar Raghavan, Hinrich Schutze, “Introduction to Information Retrieval”. Cambridge University Press. 2008. Chapters 19 and 21
2. J. Kleinberg. (1998) Authoritative sources in a hyperlinked environment. Proc. 9th ACM-SIAM Symposium on Discrete Algorithms, 1998. http://www.cs.cornell.edu/home/kleinber/auth.pdf
3. S. Brin, L. Page: (1998) The Anatomy of a Large-Scale Hypertextual Web Search Engine. WWW7 / Computer Networks 30(1-7): 107-117 (1998) http://dbpubs.stanford.edu:8090/pub/1998-8

• I like the term adversarial information retrieval.
• I did not realize that search engines used anchor text ... but it makes sense.
• It was interesting to read the paper from Google's founders but I imagine that Google has changed dramatically since those early days.

Week 9 - User Interaction and Visualization - Muddiest Point

Are there any any examples of mobile interfaces that provide document surrogates?

Week 9 - User Interaction and Visualization - Reading Note

This week I read:
1. Ricardo Baeza-Yates, Berthier Riberiro-Neto, “Modern Information Retrieval”, Addison Wesley, 1999. ISBN-10: 020139829X. chapter 10 (available at http://people.ischool.berkeley.edu/~hearst/irbook/)
2. Marti A. Hearst. Ch. 1: The Design Of Search User Interfaces. Search User Interfaces. (available at http://searchuserinterfaces.com/book/sui_ch1_design.html)
3. Marti A. Hearst. Ch. 11: Information Visualization For Text Analysis. Search User Interfaces. (available at http://searchuserinterfaces.com/book/sui_ch11_text_analysis_visualization.html)

According to Hearst, the simple user interfaces of search engines have not changed much since 1997 because search is a means to an end, but mentally challenging and so needs to be free of distraction, and it must make sense to all users. Web search engines must be simpler than historical search systems because the user base is so much broader. She goes on to enumerate many design aspects to consider when designing an interface. Hearst also covers Information Visualization from the perspective of text analysis (text mining).

Baeza-Yates & Riberiro-Neto, present an overview of the information retrieval interaction process and how the user interface must support all of those steps in the process. Many graphical examples are presented. It is surprising how the search interface has not drastically changed over the last 15 years.

Week 7 - Relevance Feedback and Query Expansion - Muddiest Point

My muddiest point for this week is: How are the original query vector, positive feedback, negative feedback back weights determined?

Week 7 - Relevance Feedback and Query Expansion - Reading Note

I read:

1. Christopher D. Manning, Prabhakar Raghavan, Hinrich Schutze, “Introduction to Information Retrieval”. Cambridge University Press. 2008. Chapter 9.
2. Xu, J. and Croft, W. B. (2000). Improving the effectiveness of information retrieval with local context analysis. ACM Trans. Inf. Syst.18, 1 (Jan. 2000), 79-112. (DOI= http://doi.acm.org/10.1145/333135.333138)
3. Wang, X., Fang, H., and Zhai, C. (2008). A study of methods for negative relevance feedback. In Proceedings of the 31st Annual international ACM SIGIR Conference on Research and Development in information Retrieval (Singapore, Singapore, July 20 - 24, 2008). SIGIR '08. ACM, New York, NY, 219-226. (DOI= http://doi.acm.org/10.1145/1390334.1390374)
4. Donna Harman, (1992). Relevance feedback revisited. Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval. Pages: 1 - 10. Copenhagen, Denmark. 1992. (http://doi.acm.org/10.1145/133160.133167)

From 1:
I have never seen a search engine with a relevance feedback system. I wonder if there are any practical examples? ... Oh, ok Excite used it primarily as a way to link people to more results like a certain result.

The weakness of the Rocchio algorithm is that it requires relevant documents to cluster into a single group. Any factor that causes the vector-space model to breakdown or cause relevant documents for form multiple clusters decrease the effectiveness of this relevance feedback algorithm.

Query reformulation, especially for technical queries is a way to make the user's query more likely to return relevant results.

From 3:
It is more difficult to use negative relevance feedback to improve results than positive relevance feedback. However, negative feedback works better in the language model than a vector-space model. Most helpful for very difficult queries with poor results.

From 4:
They found that query expansion should be used with relevance reweighting with most of the gains coming from query expansion.

Week 6 - Evaluation - Muddiest Point

Please cover the differences between DCG and NDCG again.

Week 5 - Matching Models: Probabilistic and Language Model - Reading Note

I read:
Christopher D. Manning, Prabhakar Raghavan, Hinrich Schutze, “Introduction to Information Retrieval”. Cambridge University Press. 2008. Chapters 11 & 12.

Djoerd Hiemstra and Arjen de Vries. (2000) Relating the New Language Models of Information Retrieval to the Traditional Retrieval Models. Technical Report, TR-CTIT-00-09, Centre for Telematics and Information Technology

From IIR Chapter 11 & 12, what did I just read? Maybe my brain is not working today but I only was able to form the highest level of understanding. Maybe the chapters were too dense with equations. I'm hoping that the article will make more sense.

From Hiemstra and de Vries (2000), "[Vector space] is based on the similarity between query and document, the [probabilistic model] is based on the probability of relevance, using the distribution of terms over relevant and non-relevant documents."

Week 4 - Matching Models: Ranked Boolean and Vector Space - Muddiest Point

My muddiest point for this week is cosine similarity.

Week 4 - Matching Models: Ranked Boolean and Vector Space - Reading Note

I read:
Christopher D. Manning, Prabhakar Raghavan, Hinrich Schutze, “Introduction to Information Retrieval”. Cambridge University Press. 2008. Sections 1.3 and 1.4 & Chapter 6.

From 1.3 & 1.4, boolean retrieval vs ranked retrieval: user preference generally relates to experience. Power users usually find complex boolean queries preferable to ranked retrieval queries.

From Chapter 6:
Approaches to determining result quality:

• Parametric and Zone indexes: basic ranking and metadata filtering
• Term Weighting - higher score the more times a term appears in a doc
• vector space scoring

Vector Space Modeling:

• represent documents and queries as vectors in a common vector space. This allows scoring, document classification, document clustering
• many specific schemes and methods

I think that I will need the lecture before I truly understand all of this material.

Question:
Why are vectors the best way to model this?  

Week 3 - Index Construction and Compression - Muddiest Point

I do not have muddiest point for this week.

Week 3 - Index Construction and Compression - Reading Note

I read:
Christopher D. Manning, Prabhakar Raghavan, Hinrich Schutze, “Introduction to Information Retrieval”. Cambridge University Press. 2008. Chapter 4-5.

From 4.2: Large collections where index too large for memory but index is on a single machine
    use blocked sort-based indexing (BSBI)
    Single-pass in-memory indexing (SPIMI) - better (from 4.3)

From 4.4: Distributed indexing
    term-partitioned index
    Map Reduce - (parser and inverter)

From 5.1: Must use lossless compression.

From 5.2: Methods for Dictionary Compression:
    store the whole dictionary as a string to avoid wasting space
    group terms into blocks and store pointer

From 5.3: Methods for Postings file compression
    variable encoding to store gaps between results
        variable byte codes
        γ codes

Question
Is it better to update an index to handle changes to documents or recreate on a schedule?

Week 2 - Document and Query Processing - Muddiest Point

This week my muddiest point is what are the factors for determining N in N-Gram tokenization. Is index size the primary factor?

Week 2 - Document and Query Processing - Reading Note

I read:
Christopher D. Manning, Prabhakar Raghavan, Hinrich Schutze, “Introduction to Information Retrieval”. Cambridge University Press. 2008. Section 1.2, Chapter 2, and Chapter 3.

The example process for creating an inverted index in Section 1.2 is a basic example but I think that it will prove to be a naive implementation.

In chapter 2, there are several mentions of doing tasks by hand; such as in machine learning tokenizing, filtering stop words, and synonym lists for token normalization. It is surprising to me that these systems require such careful hand-tuning but I suppose it makes sense given the crazy rules present in language.

From 3.1, Search trees > Hashing for Dictionaries.

From 3.2, Special Indexes for General Wildcard queries:
- permuterm index: rotations of each term (simple but space inefficient)
- k-Gram index: all k character long chunks of term (more complex and requires post-filtering of results)

From 3.3.3 - 3.3.4, isolated-term: k-Gram indexes used for spelling correction use the number of shared k-grams in the two terms.

From 3.3.5, Context-sensitive: use query frequency to correct.

Question
I'm curious how these techniques relate to real-world search engines. What of this is Google using?

Week 1 - Introduction and Course Overview - Muddiest Point

Thankfully, I understood the introductory lecture and do not have a muddiest point this week.

Week 1 - Introduction and Course Overview - Reading Note

Modern Information Retrieval
     - R. Baeza-Yates & B. Ribeiro-Neto

• Information Retrieval is built on Data Retrieval
• Information Retrieval is User Focused
• Data Retrieval is System Focused

• User Tasks
• Retrieval vs Browsing

Finding Out About
    - R. K. Belew
I found this reading interesting because the act of "finding out about" is such a basic function but I had never given it much thought. I think that step 3 assessing the answer is the most interesting step from a human perspective because I think that there are many factors at play to judge the answer.

This article also provided a nice overview of indexing and the challenges of selecting keywords to use for building an index.


Information Retrieval: Implementing and Evaluating Search Engines
    - S. Büttcher, C. L. A. Clarke & G. V. Cormack
This article introduced some practical considerations for Information Retrieval systems such as: result quality (relevance), challenges of indexing, updating documents, and system performance.I appreciated that this article was more recent than the others. I think Google has spoiled us and so we take for granted how effective web search currently is.


Question
Now that we've seen an over view of the components of a Information Retrieval system, which component has the most room for future study and improvement?