Comments on: del.icio.us driven Google custom search

By: Phil Barker

Phil Barker — Thu, 24 Jul 2008 11:57:26 +0000

The one that I created that I created using the extract links option, mentioned in my previous comment, worked once or twice and then stopped working. Looking at the advanced tab, it’s equivalent to using the CSE REST interface to make an annotation file, which I had problems with originally. When I look at the output of that interface, where I would expect a list of URLs based on my del.icio.us bookmarks, what I actually get is a set of URLs for Yahoos terms and conditions (Yahoo own del.icio.us). I have managed to fiddle with the options for what del.icio.us page I extract URLs from and get it working briefly, but pretty soon the Yahoo Ts&Cs come back. It’s almost like Yahoo doesn’t want to play nicely with Google.

By: jermallie

jermallie — Fri, 18 Jul 2008 19:55:31 +0000

hi Phil

we did use that way in scoofers, http://www.scoofers.com. we use del.icio.us, stumbleupon etc. and filled it now with app 5000 extract links times 10 on avarage links on 1 extraxted links site, so app. 50000 site have been labelled now, using social bookmark sites. But we are also looking at using the API’s

regards, Jeroen

By: Phil Barker

Phil Barker — Fri, 18 Jul 2008 14:00:09 +0000

Hi Jeroen, Short answer: because I hadn’t seen that option. :-} Yes, I think that would work if you weren’t generating the CSE on the fly (which, once I started doing I kind of found to be useful), also if you didn’t mind searching the links on that page that weren’t bookmarks (which I guess would not be a problem if you added del.icio.us to the list of sites not to search?). If I get the chance I’ll check it out–it would be better for people who don’t want to / can’t get involved with any scripting. Thanks.
edit: here’s one that works like that. [Later: well it did work]

By: jermallie

jermallie — Fri, 18 Jul 2008 00:12:43 +0000

why don’t you just u the ” extract links option” in the sites tab of the cse and use the url that displays 100 bookmarks and add the other url’s that diplays the next 100 etc.

regards,jeroen

By: Phil Barker

Phil Barker — Thu, 17 Jul 2008 09:02:43 +0000

I have a play with the API as a way round the limit on the number of entries in the RSS feed, but I don't want to have to ask for a password every time I update the search engine and i don't want to leave it lying around in a text file somewhere. Besides, I'ld be limited to my own tags. The best option seems to be the JSON feed, which will give up to 100 posts.

By: Phil Barker

Phil Barker — Thu, 10 Jul 2008 12:18:07 +0000

Thanks Tony, that’s very useful.
On the point of whether the search is limited to the exact page or the whole domain (there’s a third option: everything in directories below where the bookmarked page is) the Perl script has options to modify the links in the RSS feed for each of these–I think, I’ve only tested the third option, which is the one I’m using. So if I bookmark http://ocw.mit.edu/OcwWeb/Mechanical-Engineering/ I search everything in in that subdirectory and below it, which is all the OCW Mechanical-Engineering resources. It’s nice of MIT to arrange their courses like that.

By: Tony Hirst

Tony Hirst — Thu, 10 Jul 2008 11:49:08 +0000

One issue with taking the RSS feed from delicious is that if you have a lot of links tagged a particular way, they won’t all appear in the RSS feed… which is why you need a password and the delicious api…

I’m not sure if the Google CSE can be used to scrape links from a delicious page (eg set to display 100 links?)

There are also a couple of things to bear in mind when searching over links – do you want to search just the bookmarked pages, or are you happy to search over the whole domain that the bookmarkd page sits on? I explored several variations on this theme under the notion of ’search hubs’ ( http://blogs.open.ac.uk/Maths/ajh59/010686.html ). My demonstrator site for this (in desperate need of a rewrite) is searchfeedr ( http://blogs.open.ac.uk/Maths/ajh59/010000.html )

If you’re happy with only searching over a couple of dozen links at a time, this delisearch pipe should work: http://blogs.open.ac.uk/Maths/ajh59/009301.html

I thought I’d done a demo of how to use this pipe in a grazr widget that let you select a delicious user name and tag ( http://blogs.open.ac.uk/Maths/ajh59/010044.html ) but it looks as if i only ever did a cut’n'paste OPML generator ( http://blogs.open.ac.uk/Maths/ajh59/deli2opml2.html – the delicous search pipe code is at the bottom of the generated OPML)

tony

By: Martin Poulter

Martin Poulter — Thu, 10 Jul 2008 11:38:44 +0000

Thanks for slogging through all that adversity so we don’t have to, Phil.