IR Evaluation: Search Engines

This document evaluates the performance of Google and Bing search engines for a specific query related to the price of a new Xbox One. It provides a list of relevant and irrelevant documents retrieved by each search engine.

COVER PAGE

Question 1
1a) Results of removing the stop words
 Document 1
Data science interdisciplinary field scientific methods, processes, algorithms systems extract
knowledge insights data various forms, structured unstructured.
 Document 2
Data mining process discovering patterns large data sets involving methods intersection
machine learning, statistics, database systems
 Document 3
Information systems study complementary networks hardware software people organizations
collect, filter, process, create, distribute data
(condition) S1  S2
Step 1a
SSES -> SS
Processes  Process
IES -> I
SS -> SS
Process-> process
S ->
Algorithms  Algorithm
Systems System
Insights  Insight
Forms  form
Patterns -> pattern
Sets  set
Methods  method
Statistics  statistic

Networks  network
Organizations  organization
Step 1b
(*v*) ED ->
Structured  structur
Unstructured->unstructur
(*v*) ING ->
Discovering  discover
Involving  involve
Learning  learn
(m=1 and *o) -> E
Knowledge  knowledg
Large  larg
Machine  machin
Hardware  hardwar
Software  softwar
Distribute  distribut
Step 1c
(*v*) Y -> I
Interdisciplinary -> interdisciplinari
Complementary->complementari
Study->studi

Step 2
(m>0) ATION -> ATE
Information->informate
(m>0) IZATION -> IZE
Organization->organize
(m>0) IVITI -> IVE
Activiti->active
Step 3
Step 4
(m>1) AL ->
(m>1) ATE ->
Informate->inform
(m>1 and (*S or *T)) ION ->
Intersection  Intersect
(m>1) IZE ->
(m>1) ANT ->
Step 5a
(m>1) E ->
Knowledge  knowledg
Large  larg
Machine  machin

Hardware  hardwar
Software  softwar
Distribute  distribut
(m=1 and not *o) E ->
Create->creat
People->peopl
Large->larg
Searche->search
Stemmed documents
 Document 1
Data scienc interdisciplinari field scientif method process algorithm system extract knowledg
insight data variou form structur unstructur
 Document 2
Data mine process discov pattern larg data set involv method intersect machin learn statist
databas system
 Document 3
Informat system studi complementari network hardwar softwar peopl organ collect filter
process creat distribut data
1b) Merged inverted list including within-document frequencies

Merged Sorted List with within document frequency
Term DocumentFrequency
algorithm 1 1
collect 3 1
complementari 3 1
creat 3 1
Data 1 2
Data 2 2
data 3 1
databas 2 1
discov 2 1
distribut 3 1
extract 1 1
field 1 1
filter 3 1
form 1 1
hardwar 3 1
informat 3 1
insigt 1 1
interdisciplinari 1 1
intersect 2 1
involv 2 1
knowledg 1 1
larg 2 1
learn 2 1
machin 2 1
method 1 1
method 2 1
mine 2 1
network 3 1
organ 3 1
pattern 2 1
peopl 3 1
process 1 1
process 2 1
process 3 1
Scienc 1 1
scientif 1 1
set 2 1
softwar 3 1
statist 2 1
structur 1 1
studi 3 1
system 1 1
system 2 1
system 3 1
unstructur 1 1
variou 1 1

1c) dictionary related Posting file

1d) testing the inverted index
Keywords: Data, system and algorithm
 Data
Inverted Index (Data)
 System
Inverted index (System)
 Algorithm

Based on the dictionary file, the first of the three words is algorithm which is contained only
once in document 1. The second word is data and is contained in all the three documents but it
first appears two times in document 1, two times in document 2 and once in document 3. The
last word is system and appears once in all the three documents.
Boolean model and vector model
a. Boolean Model queries
1) method Ʌ process Ʌ System
This query returns all documents
2) Database Ʌ Algorithm

This query returns Document 1 and 2
3) method V process V System

This query returns all documents
b. Vector model using cosine similarity
The similarity threshold is:
Q (Data, Data,System, Algorithm)
Document 1
D1= <2, 1, 1>
Q= <2, 1, 1>

σ ( D1 , Q) 2 x 2+1 x 1+ 1 x 1
√ 22 +12 +12 √ 22 +12 +12 = 6
√6 √6 = 0.99958
Document 2
D2= <2, 1, 0>
Q= <2, 1, 1>
σ ( D2 , Q) 2 x 1+1 x 1+ 0 x 1
√22 +12 +02 √22+12 +12 = 3
√5 √6 = 0.857232
Document 3
D= <1, 1, 0>
Q= <2, 1, 1>
σ ( D3 , Q)= 1 x 1+1 x 1+ 0 x 1
√ 12 +12 +02 √ 22+12 +12 = 2
√ 2 √ 6 = 0.686589
Based on the cosine similarity of the three documents to the query document 1 is retrieved first,
followed by document 2 and then document 3.
D1D2D3
The difference between vector space model and Boolean model is that vector space model is
more precise in showing the order in which the documents will be retrieved compared to
Boolean model.
Question 2 IR evaluation
Search engines
 Google
 Bing
Target
Target 5: Price of new xbox one
Queries
 Query 1= Price of new Xbox one
 Query 2= new Xbox one cost

Google Search engine
 Query 1 results
Relevant
D(1,2,3,4,5,9,10,11,12,14,16,19,20)
Irrelevant
D(6,7,8,13,15,17,18)
D1. Xbox One - EB Games Australia
D2. The best Xbox One S prices, bundles and sales in Australia (May 2018 ...
D3. The best Xbox One X prices, bundles and sales in Australia (May 2018 ...
D4. Xbox One Packages At JB Hi-Fi Stores + All The Newest Games Out ...
D5. The best price on Xbox One in Australia | finder.com.au
D6. Xbox | Official Site
D7. Xbox One S | Xbox
D8. Xbox | Target Australia
D9. Xbox One Console Deals, Games & Accessories | The Gamesmen
D10. Microsoft's Xbox One X price will start at $499 - The Verge
D11. Buy Xbox One S 500GB Console | Harvey Norman AU
D12. Buy Xbox One X 1TB Console | Harvey Norman AU
D13. xbox one console | Xbox | Gumtree Australia Free Local Classifieds
D14. Xbox One - GameStop
D15. Xbox One and Xbox One S Consoles, Games and Accessories ...
D16. Xbox One - Best Buy

D17. How Much Does an Xbox One Cost? - MakeUseOf
D18. Microsoft Xbox One X: Australian Pricing, Specs And Release Date ...
D19. Xbox One | Entertainment | BIG W
D20. Xbox One S 500GB Consoles | eBay
 Query 2 results
Relevant
D(1,2,4,5,6,7,9,11,13,14,16,17,18,19)
Irrelevant
D(3,8,10,12,15,20)
D1. Xbox One - EB Games Australia
D2. The best Xbox One S prices, bundles and sales in Australia (May 2018 ...
D3. The best Xbox One X prices, bundles and sales in Australia (May 2018 ...
D4. Xbox One Packages At JB Hi-Fi Stores + All The Newest Games Out ...
D5. The best price on Xbox One in Australia | finder.com.au
D6. Xbox | Official Site
D7. Xbox One S | Xbox
D8. Xbox | Target Australia
D9. Xbox One Console Deals, Games & Accessories | The Gamesmen
D10. Microsoft's Xbox One X price will start at $499 - The Verge
D11. Buy Xbox One S 500GB Console | Harvey Norman AU
D12. Buy Xbox One X 1TB Console | Harvey Norman AU
D13. xbox one console | Xbox | Gumtree Australia Free Local Classifieds
D14. Xbox One - GameStop
D15. Xbox One and Xbox One S Consoles, Games and Accessories ...
D16. Xbox One - Best Buy
D17. How Much Does an Xbox One Cost? - MakeUseOf
D18. Microsoft Xbox One X: Australian Pricing, Specs And Release Date ...

D19. Xbox One | Entertainment | BIG W
D20. Xbox One S 500GB Consoles | eBay

Bing Search Engine
 Query 1 Results
Relevant documents
D(1,2,3,7,8,10,14,16,17,18)
Irrelevant documents
D(4,5,6,9,11,12,13,15,19,20)

S1: Xbox One Packages At JB Hi-Fi Stores + All The Newest ...
S2: Xbox One - EB Games Australia
S3: Compare Xbox One Consoles | Xbox
S4: xbox one console | eBay
S5: Xbox One Games - Buy The Lastest Xbox Games Online ...
S6: Xbox One | Entertainment | BIG W
S7: xbox one - Compare The Best xbox one Prices In Australia ...
S8: Xbox One S 500GB Console | JB Hi-Fi
S9: Xbox One and Xbox One S Consoles, Games and Accessories ...
S10: xbox one console | eBay
S11: Xbox One S 500GB Console | JB Hi-Fi
S12: Xbox One X review: The pinnacle of console gaming ...
S13: Xbox One Best Price in Australia | Compare & Buy with ...
S14: Xbox One Console Deals, Games & Accessories | The Gamesmen
S15: Xbox One Games - Buy The Lastest Xbox Games Online ...
S16: Xbox One - Harvey Norman Australia
S17: New Xbox One Deal Drops The Price Of The System Nicely ...
S18: Xbox: Xbox One and Xbox One S Consoles, Games ...
S19: Xbox One - Best Buy
S20: Xbox One S 500GB Console | JB Hi-Fi

 Query 2 results
Relevant documents
D(1,3,4,6,10,13,14,15,18,19,20)
Irrelevant documents
D(2,7,8,9,11,12,16,17)
S1: Compare Xbox One Consoles | Xbox
S2: Xbox One S | Xbox
S3: Xbox One - EB Games Australia
S4: Compare Xbox One Consoles | Xbox
S5:Xbox One S | Xbox
S6: Xbox One Packages At JB Hi-Fi Stores + All The Newest ...
S7: Xbox One | Entertainment | BIG W
S8: Why does the Xbox One X cost $500? We asked Xbox chief ...
S9: Xbox One Games - Buy The Lastest Xbox Games Online ...
S10: xbox one | eBay
S11: Xbox One - Best Buy
S12: Why does the Xbox One X cost $500? We asked Xbox chief ...
S13: Xbox: Xbox One and Xbox One S Consoles, Games ...
S14: How Much Does an Xbox One Cost? - MakeUseOf
S15: Xbox One X review: The pinnacle of console gaming ...
S16: Xbox One Best Price in Australia | Compare & Buy with ...
S17: Xbox 360 | Entertainment | BIG W
S18: Xbox One Consoles, Games, Controllers - Walmart.com
S19:Xbox One - Harvey Norman Australia
S20: Xbox One and Xbox One S Consoles, Games and Accessories ...

Average

For the specified target of obtaining the price of new Xbox one Google performs better than yahoo
because it has a higher precision meaning the number of documents retrieved that are related to the
search query for yahoo search engine are more compared to those retrieved for Google.
Bibliography
classeval. (n.d.). Introduction to the precision-recall plot. [online] Available at:
https://classeval.wordpress.com/introduction/introduction-to-the-precision-recall-plot/
[Accessed 16 May. 2019].