Benutzer:Atlasowa/multilingual search

Wikipedia needs a multilingual search portal/feature.

Why?

Because most Wikipedia readers read more than 1 language Wikipedia (72% !) and
because Wikipedia readers want better Wikipedia search to find Wikipedia articles on the topics and
because Wikipedia readers shouldn't be forced to "google Wikipedia" (and thereby reveal their Wikipedia requests/search history) and
because Wikipedia readers can't "google Wikipedia" from Wikipedia apps/ Wikipedia Zero.

What do we need?

simultaneous multilingual Wikipedia search, especially bilingual (first language WP + language WP with broader coverage: zh/en, ar/fr, kz/ru, ...)
reader-optimized search portal, no clutter, to be linked at from language Wikipedia search/redlink pages
www.wikipedia.org Portal and www.m.wikipedia.org for mobile
www.wikipedia.org|ru|en permalinks for bilingual search, customizable (also for browser search plugin)
search results with snippets, with image ("Seitenbild", mw:Extension:PageImages), with short text description (mw:Extension:TextExtracts)
en:incremental search
future enhancement: Wiktionary search integration
future enhancement: image search [1]
future enhancement: search result location on OpenStreetMap

Readers of Wikipedia need better search

meta:Research:Wikipedia Readership Survey 2011/Results#b. Search is the most desired improvement to Wikipedia

Readers likely to use Wikipedia more if search was improved - by country. (meta:Research:Wikipedia Readership Survey 2011/Results)
Readers very likely to use Wikipedia more if search was improved - by language (meta:Research:Wikipedia Readership Survey 2011/Results)
Slightly over half of our readers (51%) said they specifically look for Wikipedia in search engine results like google (WMF Blog post)
Wikipedia access comparison across devices (C6. How do you access Wikipedia from the following devices? (note: different bases for desktop/laptop/mobile))
meta:Research:Characterizing Wikipedia Reader Behaviour
- Why are you reading this article today?
I am reading this article to … [Information Need]
Prior to visiting this article … [Prior Knowledge].
I am reading this article because … [Motivation] (microsurvey2015)

Wikipedia search survey powered by Qualtrics "What kind of feedback do you have about search on Wikipedia?" (June 2015) meta:Research:Measuring User Search Satisfaction by Oliver Keyes

Readers of Wikipedia are multilingual

European countries and requested WP language versions, 2010.
Countries in Northern Africa and the Middle East and requested WP language versions, 2010.
Q2a. Percent who read the listed language Wikipedia / Q2b. Percent who primarily read the listed language Wikipedia (n=4,930).
Q1a. Percent who contribute to the listed language Wikipedia / Q1b. Percent who primarily contribute to the listed language Wikipedia (n=4,930).
Q2a. Number of Wikipedia languages read (n=4,930) meta:Editor Survey 2011/Location & Language
Q1a. Number of Wikipedia languages edited (n=4,930)

Wikipedia full text search page

"incategory:" search
"deepcat:" search
"intitle:" search

MediaWiki-l: An "advanced search" page for CirrusSearch?

Interwiki search

mw:Search#Interwiki search, boxes on the side of the search results page where matching pages from sister projects are shown: so you can search a word on Wikipedia and get the link to an entry in Wiktionary.
The feature originally existed for all wikis around 2009 but was later disabled; as of now there isn't a timeline for re-enabling it by default.
T46420: Restore interwiki (sister projects) results in search queries (Closed, Resolved)
T87631: Interwiki search results should come after local ones in HTML
T87632: Enable interwiki search on Meta-Wiki (Open)
T96881: Don't show the "results from other projects" box if empty (Open)
restored on italian WP: example search
mailinglist January 2017: "secondary search results are now available over the API. This means that automated language detection (provided by TextCat <https://www.mediawiki.org/wiki/TextCat>) and query forwarding can now be used by API consumers."

Portals www.wikipedia.org and www.wikipedia.de

Wikipedia search portal at www.wikipedia.org, screenshot of 2021
Search portal at www.wikipedia.de, screenshot of 2015
ComScore trend data on WMF Sites, as of Jan 2010

http://www.wikipedia.org
meta:Talk:Www.wikipedia.org_template
T5665 Auto-detect interface language for anonymous users (2005!)
- tweet #wikipedia #what? wikipedia.DE: "Wozu Englisch, wenn man sich zwischen Obersorbisch und Saterfriesisch entscheiden kann?"

T26767 - Multilingual search on project portals (e.g. www.wikipedia.org)
T90835 - Investigate www.wikipedia.org traffic % ([2][3])
T98076 Does portal traffic come from zero? (No)
- Fwd: Traffic to the portal from Zero providers Oliver Keyes
- Reader use of the Wikipedia home page Oliver Keyes, 2015-05-01
- discovery: Portal A/B test announcements Oliver Keyes, Dec 21 2015 "initial test to identify if a more prominent search box would improve the rate at which users clicked through from the portal to our various projects (at the moment, only a third of users do so)."
T104984 - Add a search option "in any language" to the multi-lingual search on www.wikipedia.org
T71489 Expose mwgrep functionality on-wiki
- Cross-wiki search API?

Prominent place for readers, traffic to portal

Autosuggest search word (by article names)

search in all Wikipedia languages

no simultaneous multilingual Wikipedia search

cluttered search page

no search results page, directs to Wikipedia Special:Search

Mobile Wikipedia search

Wikipedia Android app search with descriptions
Wikipedia Beta search on Android 4.4.4 2015-02-09 (incremental search?)
WIKIPEDIA MOBILE READER TOPLINE 2011
Malaysia WP Zero landing page

meta:Research:Wikipedia_Mobile_User_Research#Searches for Wikipedia content start in Google Wikipedia Mobile User Research 2011: 20-22% want multilingual Wikipedia search

Wikimedia-l/Wikitech-l: Re: Making www.wikipedia.org mobile friendly 2012
WikimediaMobile: Re: Where to redirect m.wikipedia.org? 2013
mw:Search/Design
WikimediaMobile Mailinglist: Simplified search? It's possible! How good is our search? Here's some data! 2014

clean search interface, no clutter

autosuggest for search

Search results page with pictures

Search results with WP article lede sentences

no simultaneous multilingual Wikipedia search

Search tools by Wikipedians

Bong by Magnus Manske

Bong: Bing-like Wikimedia search interface. By Magnus Manske @tools.wmflabs
https://tools.wmflabs.org/magnus-toolserver/bong.php?doit=1&mode=pedia&q=lagunita
Bong image search (basic, small)

Clean search interface

Search results page with pictures

Shows WP article lede sentences

Supports several wikimedia projects (commons, wiktionary,...)

No search snippets

No simultaneous multilingual search, only english

No autosuggest for search

No incremental search

GlobalWPsearch by Aka

http://vs.aka-online.de/cgi-bin/globalwpsearch.pl
option: multi line search field (Please enter one article name per line.)
http://www.similarweb.com/website/vs.aka-online.de : 68.12% of total Vs.aka-online.de desktop traffic in the last 3 months came from referrals. Top Referring Sites:
- fr.wikipedia.org
- it.wikipedia.org
- hi.wikipedia.org
- en.wikipedia.org
- zh.wikipedia.org
- tl.wikipedia.org
- az.wikipedia.org
- de.wikipedia.org
- es.wikipedia.org
- mr.wikipedia.org
http://www.alexa.com/siteinfo/vs.aka-online.de : Which sites did people visit immediately before this site? 1. wikipedia.org 66.7%

Benutzer_Diskussion:Aka/GlobalWPSearch since 2005!

simultaneous multilingual search! (~50 Wikipedias)

prominently linked in several Wikipedia language versions:

IT it:MediaWiki:Noarticletext: "Cerca nell'enciclopedia se esiste un titolo simile. tutte"

FR fr:Nick_Helm: "Recherchez cet article dans d’autres langues, grâce à l’outil Global Wikipedia Article Search (en anglais)."

HI [4]

No search snippets, no article ledes

No images

No autosuggest for search

No incremental search

Wikimedia-Search by Kolossos

https://toolserver.org/~kolossos/tree/search.php?pro=wikipedia&langauswahl=en dead with toolserver
Wikipedia:Kurier/Archiv_2005/2: "Auf dem Toolserver befindet sich die von Kolossos entwickelte Suchmaschine Wikimedia-Search für alle Projekte und Sprachen der Wikimedia. Dieses Werkzeug verfügt über eine so genannte "Suggest"-Funktion, errät also welchen Artikel man sucht, dabei werden lange Artikel bevorzugt. Es sollen damit auch die Server in den USA entlastet werden, da Fehleingaben vermieden werden. Mit einem Mausklick lässt sich das Programm in die Sidebar des Browsers integrieren, und ist somit schnell aufrufbar.(Kolossos 22.12.)"

Autosuggest article names for search

Search in several WP languages

No simultaneous multilingual search

Offline, dead.

Wdsearch by Magnus Manske

Wdsearch script screenshot enwiki
Wdsearch screenshot frwiki 2014-04-24
Wdsearch at occitan Wikipedia

magnusmanske.de November 6, 2013
en:MediaWiki:Wdsearch.js
en:MediaWiki_talk:Wdsearch.js
Wikitech mailinglist: Re: searching across all languages, November 2014

simultaneous multilingual Wikipedia search !

search in several Wikimedia projects

No WP search snippets

No images in search results

cluttered search interface

hack for a Knowledge Engine by Magnus Manske

https://tools.wmflabs.org/magnustools/ke.html
MagnusManske 15. Feb. 2016: "My cheapo “knowledge engine” searches across Wikimedia projects (single language, apart from @wikidata)"

Wikipedia search at external sites

instapedia

http://web.archive.org/web/*/http://www.instapedia.com/
instapedia = super fast multilingual wikipedia search with autocomplete.
Instapedia - Mobile: Auto-complete style multi-language simultaneous Wikipedia search

simultaneous multilingual Wikipedia search!

Autocomplete search

Incremental search

Clean interface

Shows result with article lede

For mobile and non-mobile

No images in search results

Not integrated to WP, dump search?

Offline, dead.

WikiWand

en:WikiWand
www.wikiwand.com/
- www.wikiwand.com/en/Ebola_virus_epidemic_in_Liberia
Wikiwand app for iOS
- "Multi-language search (up to 3 languages simultaneously)"
- "Amazingly accurate search results with pictures"

Autocomplete search

Incremental search

For mobile and non-mobile

Clean interface

Search results with images

Simultaneous multilingual Wikipedia search (up to 3 languages)

External site, Wikipedia mirrors

buk.io

http://buk.io/ (by Minsu Kang)
Wikipedia in unseen UI ycombinator

Fast en:Incremental search: search as you type

For mobile and non-mobile

Clean interface

Shows article lede as search results

Search results with images

search in several Wikipedia languages (english, korean, german)

No simultaneous multilingual Wikipedia search

External site, Wikipedia mirrors

wiksearch.com

http://wiksearch.com
- "Type a topic above to see links to Wikipedia articles that match — English only, for now. As you type, the green bar below the input box will fill with matching Wikipedia links sorted by how popular they are."
- "This service uses the Datamuse API on a vocabulary of approximately 10 million Wikipedia titles and redirects." [5]
- "The titles come from the publicly available Wikiedpedia page view counts for the last 7 days."
- "Privacy: This website saves no data other than a count of the number of queries made to it. The API calls and the Wikipedia links use HTTPS. The API server does not log or remember your queries."
by Doug Beeferman - 16.06.2015 (http://www.dougb.com) [6][7]

Wikiwix

search in several Wikipedia languages

search in several Wikimedia projects (Wikipedia / Wikisource / Wiktionary / Wikiquote / Wikibooks / Wikispecies / Wikiversity / Wikinews / Commons)

No simultaneous multilingual Wikipedia search

External site, Wikipedia mirrors

Wikilinks / WikiWeb App

Wikilinks / WikiWeb App

WikiWeb App (wikilinks)
WikiWeb "developed by Baltimore-based design startup Friends of the Web, is an alternative iPad or iPhone reader that provides a visual and interactive way to explore all knowledge as described at Wikipedia." [8]
wikilinks3 @itunes: "Fast Wikipedia article search in all the languages of your choice at once."
video, example search for "Federer"

Autosuggest search

Clean interface

Wiktionary search

Simultaneous multilingual Wikipedia search

External site, Wikipedia mirrors

Google multilingual search

Google drives traffic to Wikipedia, but half of readers look for Wikipedia content WMF Blog Ayush Khanna on October 26, 2011
Who reads which Wikipedia? The WMF's surprising stats Signpost/2013-04-01

2012, Google Inc. United States Patent 8,224,836 - "Searching in multiple languages".
search results in multilingual format in google, (June 11th, 2012)
http://www.2lingual.com/about.html
en:Bilingual search engine
"wiki" is a top search term! https://www.google.nl/trends/explore#q=wikipedia%2C%20wiki&cmpt=q

Stats

http://searchdata.wmflabs.org/ SearchData Discovery Dashboards
- http://searchdata.wmflabs.org/metrics/

meta:Research:Measuring User Search Satisfaction

External Search API usage Oliver Keyes, 2015-05-20 ("Conclusions: (...) Drop in replacements for WP's interface potentially a big market now.")

Backlinks to Wikipedia

Backlinks TO Wikipedia for search [9], www.0pii.com
- 2013 wikilinks-corpus
- https://commoncrawl.org/ is the biggest open one
- http://80legs.com/giant-web-crawl.html is perhaps the biggest commercial one.

Search and Discovery (WMF)

mw:Discovery: "The Discovery department of Wikimedia Engineering is building the anonymous path of discovery to a trusted and relevant source of knowledge."
mw:Search/Old/Design ("old" since 30 April 2015‎)
wikimedia-search mailinglist started April 2015 - renamed September 2015:
- wikimedia-discovery mailinglist (wikimedia-discovery @gmane)

https://suggesty.wmflabs.org/suggest.html Completion suggester experiment
- http://en-suggesty.wmflabs.org/wiki/Special:Search (using page rank)

Discovery Dashboards:
- http://discovery.wmflabs.org/portal/ #clickthrough_breakdown
- http://discovery.wmflabs.org/external/ #traffic_summary #traffic_by_engine (it breaks down pageviews and shows how external search engines influence the traffic that hits wikis. Both a simple count of search-referred pageviews versus other pageviews, and a breakdown of how much traffic is coming from what specific search engines, is included.[10])
- http://discovery.wmflabs.org/metrics/

https://people.wikimedia.org/~jgirault/ (Top 10 based on navigator languages, Prototypes using React) [11]
- proxy: http://portal.wmflabs.org
- https://people.wikimedia.org/~jgirault/react-top10-fs/

Benutzer:Atlasowa/multilingual search

Inhaltsverzeichnis

Readers of Wikipedia need better search

Readers of Wikipedia are multilingual

Wikipedia full text search page

Interwiki search

Portals www.wikipedia.org and www.wikipedia.de

Mobile Wikipedia search