The easiest way to do this is by simply adding the necessary files (jquery.js and jquery.quicksearch.js) to SharePoint, e.g. the Style Library, and then adding a Content Editor Web Part above the list you want to filter and edit the HTML content of the web part to add an input field and a call to the quicksearch method.
As an example, I’ve added a content editor web part with the following HTML:
'table#onetidDoclibViewTbl0 tbody tr'
to the quicksearch-function, which obviously reflects the table rows to filter on.
onetidDoclibViewTbl0 is SharePoint’s internal id of the list view, and will vary for different lists or if you have several list views on the page.
The quicksearch-method will search in the whole row, i.e. all the columns. Note that since this is just jQuery, the quicksearch will only search in the items displayed, so you need to display all items. If you want to do search in a similiar way without showing everything on the page, you should check out SPServices.
A better way to utilize this is to create a webpart with this code (could be deployed as a sandboxed solution) to be able to reuse this functionality more easily.
Screenshots of a list with a lot of documents:
Screenshot of the library filtering on “Pen”:
Screenshot of the library filtering on “2013”: