I’ve liked the Google custom search engine since I first played with it shortly after it came out. If you don’t know about Google CSE, it allows an individual or group to create a search form that will perform a full text search using the Google search engine but limited to sites which they choose. This search form, and the results page, can be embedded in any website. I think it is the obvious way to build a cross search across all the centres in an organization like the HE Academy (this was one of my first custom search engines). Better, for teaching and learning you can set up a reading list of recommended sites for a course and let students do a full google search that prioritizes those sites (for a sort of generic variation on this see Tony Hirst‘s Open Educational Resources search). Better still, let the students as a group decide which sites they want on their course reading list.
Building and editing a Google custom search using the interface on the Google site is by no means difficult, but over the past year or so some tools to make it even easier have come out. The Google Marker allows you to add sites to a CSE by pressing a button on your browser. Or you can make CSEs on the fly that search across the links in a web page (the example from Tony Hirst was based on links from this page from ZaidLearn). Scott Leslie enhanced this idea by putting the links into a wiki so that it was possible add sites.
That’s great, showing the potential for Google CSEs to fit in with other tools that are already being used. Now the tool that I use for marking interesting sites is del.icio.us, so what I really wanted was a CSE that searched over sites I have tagged in del.icio.us. I found a couple of examples of this. One wanted my delicious username and password. No thanks. The other, deligoo, looks good, but wanted me to install a Firefox plugin, which didn’t seem quite what I was after. Anyway, I couldn’t get it working (it wasn’t a good day).
I knew that you could specify the sites for a CSE to search in an XML file and thought it shouldn’t be too difficult to take a feed from del.icio.us and convert it into that XML file. There’s even a RESTy API thing from Google that promises to do that sort of thing. But I couldn’t get it working (it wasn’t a good day). So I thought why not take a tool like RSS2HTML to make a page with the links I wanted to search and cut’n’paste the code for an on-the-fly CSE which would search the links on that page. But I couldn’t get it working (it wasn’t a good day).
So then I thought I probably needed to start from scratch and find out what these XML files that define custom searches actually looked like and how they worked and perhaps I would start to get somewhere. The Google developer guide for this isn’t great, but with its help and by exporting the context and annotation files for some custom searches I had created through the Google web interface, I eventually worked out what I was doing. Along the way I found out that something somewhere was caching my files so that updates weren’t getting to Google. Even when I used the Google refresh and test page for CSEs the updates didn’t get through: I would make change to the XML file, try to test it and find it didn’t work, make some modification, test that and find that suddenly the CSE was working with the previous version. I have a bruised forehead now, but once I worked out where the problem was the day started getting better.
At the end of the day, I had a perl script that would read the RSS feed for pages bookmarked with a specified user/tag combination on del.icio.us to create the CSE XML description of hosted Google custom search engine that searches those sites tagged on del.icio.us. The del.icio.us user/tag combination that specifies the sites to be searched is hard coded into the script. It would be pretty trivial to change this so that they could given dynamically, but I think the caching might cause problems.