My friends and I refer to that, in conjunction with also opening all the cupboards in the kitchen for the same reason, as “the search for Schrödinger’s Sandwich”. We are hoping that, while the cupboard/fridge is closed, the unobserved quantum wavefunction within will enter a state such that the molecules rearrange themselves into a tasty sandwich the next time it is opened.

It has not worked yet.

(We thought it did once, but it turned out someone else put the sandwich there while the searcher was elsewhere.)

Planet Venus supports multi-threaded polling. Our planets usually update in < 1 minute with 10 threads (and while they aren’t planet-debian-sized, they aren’t aren’t tiny)

For people who don’t run their own infrastructure, the 301 approach won’t help much.

The intent is actually for mod_virgule to keep only the most recent copy of a modified post and it works in most cases but some syndication formats make it almost impossible to distinguish a modified post from a new post. Any patch that improves this situation would helpful.

It is by design that we archive posts forever but a patch that provided a configuration option in config.xml to set a retention time limit would be fine.

Ikiwiki does not immediatly remove posts removed from feeds. I tend to see that as trying to close the barn door after the horse is out. (Or, it’s a probably minor modification to the code, that I never considered making before..) It will, however, update posts with no record of the old content. That seems a more common way to retract things anyway, based on the modified versions of posts that I sometimes see pile up after the original post in my rss2email mailbox.

http 301 support is on my todo list, but since any failure to access a feed shows up on the wiki page and can be fixed by anyone, and since many/most bloggers don’t know about using 301 when moving their blog anyway, it’s not been a priority. Nor has pinging, largely because I don’t feel pinging is a very scalable solution — I can’t expect to get all the blogs who I personally aggregate to ping my aggregator. Not until a lot of blog software supports rss 2.0′s clouds at least, which still seems to be an obscure and little-used thing.

The pragmatic solution to needing to poll more feeds than your desired poll interval would seem to be threading the polling. 5 threads should be able to handle 300 feeds in well under 10 minutes.

http://anarchogeek.com/articles/2008/7/23/beyond-rest-building-data-services-with-xmpp-pubsub