Overview: Extracting article text from HTML documents | My tech blog.
In the world of web scraping, text mining and article reading utilities (readability bookmarklet) there is an ever growing demand for utilities that are capable of distinguishing parts of a HTML document which represent an article apart from other common website building blocks like menus, headers, footers, ads etc.
The Endeca User Interface Design Pattern Library (UIDPL) describes principled ways to solve common user interface design problems related to search, faceted navigation, and discovery.
The library includes both specific UI design patterns as well as topics — groups of patterns related to significant aspects of search and discovery
High Scalability – High Scalability – Twitter’s Plan to Analyze 100 Billion Tweets
If Twitter is the “nervous system of the web” as some people think, then what is the brain that makes sense of all those signals (tweets) from the nervous system? That brain is the Twitter Analytics System
ElasticSearch – ElasticSearch Overview
Search Engines data model roots lies with schema free and document oriented databases, and as shown by the #nosql movement, this model proves to be very effective for building applications.
Elastic Search model is JSON, which slowly emerges as the de-facto standard for representing data these days. More over, with JSON, it is simple to provide semi-structured data with complex entities as well as being programming language natural with first level parsers.
Auto-Suggest From Popular Queries Using EdgeNGrams
A popular feature of most modern search applications is the auto-suggest or auto-complete feature where, as a user types their query into a text box, suggestions of popular queries are presented. As each additional character is typed in by the user the list of suggestions is refined. There are several different approaches in Solr to provide this functionality, but we will be looking at an approach that involves using EdgeNGrams as part of the analysis chain.
freshmeat.net: Project details for Lucene Image REtrieval
The LIRE (Lucene Image REtrieval) library provides a simple way to create a Lucene index of image features for content-based image retrieval (CBIR), which allows searching for similar images. The used features are taken from the MPEG-7 standard: ScalableColor, ColorLayout, and EdgeHistogram. Furthermore, methods for searching the index are provided.
SolrJS
A JQuery based ajaxian interface to the Solr search engine
Apache Solr has added many new features and performance improvements since the Search smarter with Apache Solr series was published. In this article, Solr and Lucene committer Grant Ingersoll details the improvements in Solr 1.3, including distributed search, easy database imports, integrated spell checking, new extension APIs, and much more.
PageRank is Google’s way of deciding a page’s importance. It matters because it is one of the factors that determines a page’s ranking in the search results. It isn’t the only factor that Google uses to rank pages, but it is an important one.
This article goes into details of how the pagerank is generated.
Searchme Visual Search
An entertaining new search engine. Time will show about the rest…
Librarian’s Ultimate Guide to Search Engines
Online research and the power of the web, have made accessing information only fingertips away from all of us, but the taxonomies and standards used for search will impact how people learn online and off for years to come. Below are some of the things librarians understand about search – and things that anyone doing online research can benefit from.
Sphinx – Free open-source SQL full-text search engine
Sphinx is a full-text search engine, distributed under GPL version 2. Commercial license is also available for embedded use.
Generally, it’s a standalone search engine, meant to provide fast, size-efficient and relevant fulltext search functions to other applications. Sphinx was specially designed to integrate well with SQL databases and scripting languages. Currently built-in data sources support fetching data either via direct connection to MySQL, or from an XML pipe.
Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.
Sitemap 0.90 is offered under the terms of the Attribution-ShareAlike Creative Commons License and has wide adoption, including support from Google, Yahoo!, and Microsoft.
Yotophoto
This search engine can find images that correspond to a specific color, entered trough a colorpicker or in hex.
Song Search by Tapping – Search for music by tapping the rhythm of the song’s melody.
This site lets you search for a song, by tapping the rhythm of its words (lyrics).
After you tap the rhythm, we’ll show you a list of matching songs. If we can’t identify your song, you can teach our system by entering the song name.
Who’s Who of the Major Search Engines and Directories
Do you feel that Google is hiding something from you? :o) try one of the top search engines reviewed in this article