Garth's test of PyDS 2003/4


Render at upstream site

(also posted to pyds-dev)

This might seem a little controversial, but I'm thinking of making rendering as completely distinct from editing as possible.

I can run CGI scripts at my upstream site. Rather than render locally and FTP or XML-RPC upload every page, which can get quite bulky if posts are in multiple categories, I'd prefer to upload changed database entries and have the pages rendered upstream.

This plays well into the Funky Caching idea, where pages are rendered on the fly by a 404 handler. If you know a page is out of date, just delete it; if anyone wants it, it'll come back later.

Better yet, one could also run PyDS in some hypothetical CGI-based mode (my host doesn't permit persistent processes) to make posts from computers that can't contact the PyDS server over the network.

This ties in nicely with my database access refactoring idea, and with my (as yet undisclosed) idea for adding support for rendering via ZPT. It sounds pretty safe to create the schema database, add a tool to synchronize database entries based on those schema, add a way for tools to customise rendering based on that (which'd work either locally or remotely)... and all without disrupting the core too much. Cool.

posted at 16:51:44    #    comment []    trackback []

More thinking on database access in PyDS

After waking up at 3am to kill a mosquito, I couldn't get to sleep, and for some reason ended up thinking more about how to work around some of Metakit's limitations in PyDS.

Top of the list was a module used to mediate all access to databases. That module would cache storage references so that multiple tool modules could access other tools' databases safely. [Opening a database twice just doesn't work -- a new record or property defined by one instance can't be accessed via the other, and I suspect data corruption wouldn't be far away if you made changes on both.]

The module would also maintain a schema database, of known and fixed layout, that stored information on all the defined properties of the other databases. That'd work around the "need at least one valid property name to find out the other properties" problem.

The final realisation of the night, just before I drifted off, was that a nice way to let tools define attributes on other tools' databases was to take a leaf out of XML namespaces.

Inside Metakit, the properties would be (say) .id, .title etc for the WeblogTool, and .radio__id for the Radio ID property defined by the RadioMigrationTool.

As accessed by the rest of PyDS, it'd be .id, .title, and So long as we kept tool names from clashing with property names, I think we'd be safe, and that's something the schema database could greatly help with.

Having safely given tools access to other tools' data, all we need is a few callbacks in various tools (particularly the WeblogTool, but also others) so that other tools can hook the edit form, RSS generation, HTML generation, and other bits. That's later, though. :)

posted at 08:24:00    #    comment []    trackback []

Metakit Weirdness

If you perform a getas with no properties defined, the view's structure will be empty. Define even one [valid] property in the view, and you'll get all the properties when you call structure . If the property wasn't valid, of course, you'll end up defining it. Oops.
>>> import metakit
>>> db ='',0)
>>> vw = db.getas('posts')
>>> len(vw)
>>> vw.structure()
>>> vw = db.getas('posts[id:S]')
>>> len(vw)
>>> vw.structure()
[Property('S', 'id'), 
 Property('S', 'title'), 
 Property('S', 'link'), 
 Property('S', 'source'),
 Property('S', 'sourceurl'), 
 Property('S', 'text'), 
 Property('S', 'rendered'), 
 Property('I', 'onhome'), 
 Property('I', 'structured'), 
 Property('S', 'pubdate'), 
 Property('F', 'pubtime'), 
 Property('V', 'categories')]

I'm thinking PyDS might need some kind of lightweight schema manager to mediate between tools and their databases. My main driver for this is so that tools can safely access each others' data. Specifically, so that I can write a RadioMigrationTool. It'll want to add categories, add posts, and maintian its own preferences so you can do things like suck the data down again if necessary.

Another concern is adding properties. If I want to keep track of Radio's post ID, so I can keep the permalinks intact, how would I do that? There's a lot to ponder, here.

posted at 17:49:20    #    comment []    trackback []

Packet scanning and manipulation via Python

Fresh from the PyPI feed (RSS): Scapy 0.9.12beta, a Python-interfaced packet scanning and manipulation package. Most impressive to me is that the command line interface is the Python interpreter, making some very powerful transformations quite simple...
posted at 09:12:00    #    comment []    trackback []

a nice metaclass example

a nice metaclass example (post to python mailing list)
posted at 21:50:24    #    comment []    trackback []


Who couldn't love this NTK summary of the game Go?
... BBC offer baffling introduction to game of GO, a game apparently played with an infinite number of stones, on a chequered board, until both players are so bored they give up.
posted at 09:12:00    #    comment []    trackback []

Spam-Free Send Mail Link

I'm thoroughly impressed with the HiveWare Enkoder, which loosely encrypts your email address with JavaScript so address-harvesting robots can't feed your details to spammers.

It's too late for me, of course. My addresses are already out there in such force I'm drowning in spam.

posted at 08:57:04    #    comment []    trackback []


I'm intrigued by Joe Gregorio's RESTlog APIs, in which posting, comments, trackbacks and pingbacks are all handled by posting RSS fragments to a given location.
posted at 16:12:16    #    comment []    trackback []

A shame to speak her name?

Before I read Raid Qusti's article, I had no idea just how crazy things are in Saudi Arabia with respect to the rights of women.
How can we Saudis talk about women negotiating cultural barriers when the biggest cultural barrier of all stands solid as a concrete wall, unchanged through the passage of time: Uttering a womanís name in public?

I knew they were property. I knew they were locked up, and that being seen in public without their owner (father or husband) could result in their death. I had no idea it was so bad that anyone knowing their name was considered shameful. Far out.

posted at 19:14:40    #    comment []    trackback []

Cicadas? After Easter?

The weather just keeps getting weirder and weirder.
posted at 18:03:12    #    comment []    trackback []

JavaScript MD5 and authentication

I'm thoroughly impressed with JavaScript MD5, which I picked up via Simon Willison's blog. We no longer have any excuse sending passwords in the clear.
posted at 10:26:40    #    comment []    trackback []

Shortcuts and Substitutions

It occurs to me that shortcuts in reST postings should be handled via reST substitutions. The one position where that breaks down is shortcuts designed to expose HTML, like my RSS or em shortcuts (RSS, —), which should really be using .. raw:: html...
posted at 17:26:56    #    comment []    trackback []

Anonymous hyperlink test

Even with the official 0.2 version of docutils, trying this:


gives you this:

D:\Python22\Lib\site-packages\PyDS\WeblogTool.py599editOldEntryrendered = self._getDummyRequest().renderText(text, structured=structured)
D:\Python22\Lib\site-packages\PyDS\Tool.py503renderTexttmpl = PyDS.StructuredText.renderText(tmpl)
D:\Python22\Lib\site-packages\PyDS\StructuredText.py69visit_targetself.targetids[node['name']] = node['id']
D:\Python22\Lib\site-packages\docutils\nodes.py332__getitem__return self.attributes[key]

It works fine in docutils/tools/

posted at 17:00:16    #    comment []    trackback []

Rendering Tricks

Thanks to a quick shortcut, this inline now works:


posted at 16:45:20    #    comment []    trackback []

This is my primary blog for the time being...

... due to my box at home being quite, quite sick with a video card problem.
posted at 15:47:44    #    comment []    trackback []
April 2003
  1 2 3 4 5 6
7 8 910111213

Garth is testing PyDS rather more extensively than at first he thought he would.

XML-Image Letterimage

© 2003-2004, Garth T Kidd