argv0.net

Simple AJAX with MochiKit and JSON, Part I

Fri, 02 Sep 2005 05:57:06 GMT

I've just posted part one of a series of tutorials on using the MochiKit JavaScript framework to write AJAX web applications. Enjoy!

MochiKit Rules

Sat, 30 Jul 2005 05:29:08 GMT

There's much buzz about MochiKit - the insanely cool Javascript framework by Pythonista/uber-hacker/all-around nice guy Bob Ippolito - hundreds of del.icio.us links, blogosphere buzz, and silent awe from authors of existing Javascript frameworks as they ponder something objectively superior to just about of all the crapulent, buggy stuff currently out there

MochiKit holds a special place in my heart, though, as I just completed a flashy web app for my job, with all the AJAX bells and whistles managers expect ever since Google (with Google Maps) went and made our jobs much harder by upping expectations on how real web apps should behave. Most of this wouldn't be possible without MochiKit, especially MochiKit.DOM, which brings Nevow "stan" syntax to Javascript.

MochiKit also brings the Twisted idioms we've come to know and love, with Deferreds and callLater. Sometimes I forget if I'm editing client or server code. MochiKit encourages the use of JSON, and since Javascript dictionaries look just like Python dictionaries, writing server handlers is usually just a matter of returning a string representation of a Python dictionary, right from the database.

So, if you do any web development, it's imperative you check out MochiKit. Consider this a case study - MochiKit was a key enabling factor for pulling this project off on time.

Simple AJAX With OpenRICO

Fri, 01 Jul 2005 06:07:28 GMT

New JavaScript frameworks seem to be coming fast and furious these days, and it can be hard to weed out the good stuff from the garbage. Yesterday I came across OpenRICO, an open source version of what was formerly a proprietary framework at Sabre. The OpenRICO page has a bunch of cool demos (I especially dig the lazy-loading AJAX table views for large datasets).

So far, I've only done AJAX stuff with Nevow and LivePage, which is about as easy as it gets. Easy has its downsides though - I found myself writing lots of code in handlers on the server that could (and should) be executed on the client, like keyboard event handlers. Since the all client code for LivePage is either auto-generated or complicated enough to discourage manual edits, I decided it was time to bite the bullet and start writing client-side code that "manually" invokes server methods. Here I present a simple example on how to use OpenRICO and Nevow. While this example is specific to Nevow, it could be easily adapted to any framework or language.

Most of these JavaScript frameworks are just convenience wrappers around the various implementations of XmlHttpRequest, along with helper functions for manipulating the DOM when the results come back. For simple jobs, I could just use the Nevow implementation of these wrappers, but the cooler OpenRICO stuff depends on the OpenRICO implementation, so I use that here, even though this is a trivial example.

On the client, we have to mark up the area on the page we're going to to replace, then register a handler to make the request. We do this in a function that gets called when the page finishes loading, via the onLoad event of the body element.


<html>
<head>
<script src="/static/prototype.js">
<script src="/static/rico.js">
<title>Content Replacement Demo</title>
<script type=text/javascript>

function bodyOnLoad() {
    /* Register the div named 'targetDiv' as something that will be replaced
       by an AJAX call to the server */
    ajaxEngine.registerAjaxElement('targetDiv');

    /* Register our event handler and associate it with a method on the
       server */
    ajaxEngine.registerRequest( 'replaceMe', 'doReplace' );
    }

function replaceMe(thing) {
    /* Our event handler */
    ajaxEngine.sendRequest( 'replaceMe');
    }
</script>
<body onload="javascript:bodyOnLoad()">
<a onclick="javascript:replaceMe(this)">click me</a>
<div id="targetDiv">
<p>this text will be replaced</p>
</div>
</body>
</html>

Simple enough, and fairly self-explanatory. The registerRequest method maps a local event handler, replaceMe, to a method on the server, doReplace, which we'll implement now with a nice, reusable AJAXResponse class.

from nevow import rend, loaders, inevow, tags as T, static

class AJAXResponse(rend.Page):
    
    docFactory = loaders.xmlfile("ajaxresponse.xml")
    
    def __init__(self, targetId, replaceWith):
        self.targetId = targetId
        self.replaceWith = replaceWith
        
    def beforeRender(self, ctx):
        inevow.IRequest(ctx).setHeader(
            'Content-Type', 
            'text/xml; charset=UTF-8'
            )

    def data_innerHTML(self, ctx, data):
        return self.replaceWith

    def data_targetId(self, ctx, data):
        return self.targetId

This class takes two arguments, the id of the div to replace, and the content to replace it with. The XML format expected by OpenRICO is in the "ajaxresponse.xml" template, viewable here. We can use this class from within a child_ method of another Nevow page to handle AJAX requests (entire file here):

    def child_doReplace(self, ctx):
        return AJAXResponse(targetId="targetDiv",
                            replaceWith=T.p["ok, you're replaced"])

And that's it. I'm finding that once you get the hang of it, developing web apps in this style is easier than trying to give the illusion of continuity to a bunch of stateless pages, with all the long query strings, session state, and boilerplate HTML that comes along with it.

Simple AJAX With Nevow LivePage

Tue, 28 Jun 2005 04:44:32 GMT

I spent the weekend without an net connection and with a few free hours without much to do, so I played around with Nevow's LivePage, which enables one to write so-called "AJAX" (Asyncronous Javascript And XmlHttpRequest) web applications. Like most else with Nevow, it's ridiculously easy to do. I wrote a simple app that lets the user enter an RSS url and get a 'slideshow' of items from that feed (using the FeedParser module to parse the RSS). It's nothing special at all, but it serves as decent boilerplate for an AJAX page, and it was short enough that I figured I'd post it here as an example.

class AjaxRSS(livepage.LivePage):

    addSlash = True

    @livepage.handler(byId('input').value)
    def selectFeed(client, feedUrl):
        def updateClient(parsedFeed):
            if parsedFeed['entries']:
                entry = parsedFeed['entries'][0]
                parsedFeed['entries'] = parsedFeed['entries'][1:]
                client.set("link", T.a(href=entry['link'])[entry['title']])
                client.set("content", T.p()[loaders.htmlstr(entry['summary'])])
                reactor.callLater(5, updateClient, parsedFeed)
        d = getPage(feedUrl)
        d.addCallback(parse).addCallback(updateClient)
        
    docFactory = loaders.stan(
        T.html[
            T.head[livepage.glue],
            T.body[
                T.p["Enter an RSS url"],
                T.div(id="main")[
                    T.form(id="form", onsubmit=selectFeed)[
                        T.input(id="input", width="20"),
                        T.input(type="submit", value="submit")]],
                    T.div(id="post")[
                        T.a(id="link"), T.br,
                        T.p(id="content")]]]
        )

Live demo here. Try http://flickr.com/services/feeds/photos_public.gne?tags=cats&format=rss_200 as a url, for a slideshow of people's cat pictures on Flickr

The magic here is the @livepage.handler(byId('input').value) decorator, which basically says "pass me the value of the node with the id of 'input'". Within the handler, we call the 'set' method on the client, which allows us to change the contents of the users HTML page (another useful client method is sendScript, which allows you to send a chunk of Javascript for immediate evaluation to the user's browser.

Here's the source, and a .tac file that you can launch with twistd -noy ajaxrss.tac

Site Update

Mon, 20 Jun 2005 17:15:54 GMT

I just imported a bunch of old, mostly python-related content from my PyCS blog into here - I'm pretty sure it won't appear new to various aggregators like PlanetPython, but in case it does, consider this a preemptive apology.

Housekeeping

Thu, 16 Jun 2005 15:31:15 GMT

Apologies to anyone who was reading this site with an aggregator for the last 24 hours or so. I changed to code to make all the links absolute (so syncing the HTML content to community sites like PyCS works), but neglected to fix the RSS feeds, so they got an extra http://argv0.net at the beginning of all the links.

Speaking of RSS, you can get feeds of only certain tags by subscribing to links like http://argv0.net/tags/python/RSS/full. You can use multiple tags as well, as in http://argv0.net/tags/python+politics+catblogging/RSS/full. Nifty, eh?

Update: The comments are fixed now, too.

I'm Back

Wed, 15 Jun 2005 18:07:30 GMT

I have a bad habit of abandoning blogs, this one being no exception. I have an excuse, though- the workload from my real job has really ramped up lately, killing most of my free time. I also installed Tiger on my Powerbook and lost my PyDS installation, and there was no way I was going to install that beast again.

So, I finally found a few minutes to kick my stalled blogging project, Minnow, into a good enough state where I can use it for my own blog. The silver lining in all the "real" work is that it's all been in Python, PEAK specifically, so once work slows down, I'll be able to do the rewrite of Minnow I started at PyCon, based on PEAK and some other cool stuff I learned about there.

I haven't finished moving over the content from the old blog, so there may be broken links and various wierdnesses - feel free to report them to me. The blog software is bound to have bugs in it as well, so bear with me - I'll get this place in order soon.

Update: BTW, Anyone accessing this blog at the pycs.net URL should switch over to http://www.argv0.net, as I'll only be syncing posts to PyCS for a few days

More PyCon

Thu, 24 Mar 2005 04:05:36 GMT

Lots of great talks today, most notably Donovan Preston's talk on Nevow, and Bob Ippolito's pair of talks on pyobjc. I missed the PyPy and Type Inference talks due to a hair-on-fire work emergency, so hopefully they got that one on tape.

Ted Leung is keeping notes of the presentations, taken by various Pythonistas using SubEthaEdit, a collaborative e ditor that lets multiple users take notes over a Rendezvous network. I'd never used it in that fashion until today, and was very impressed by how powerful it was

Finally, if you're in the DC area and like beer, check out The Brickskeller (warning: horrible web design ahead). They have over 1000 different bottled beers from all over the world. I had some local DC ales, which were really good, and a few funky Belgian ones. Definitely a great spot, but probably not amenable to the 20+ dinner groups that conferences like this seem to spawn.

PyCon

Wed, 23 Mar 2005 05:16:00 GMT

I got to the convention center today around 2, after walking about 4 miles in the wrong direction from my hotel. While I was on the plane, the Twisted/Divmod folks released Twisted 2.0 and Nevow 0.4, which is very cool, of course. I'll probably use Nevow 0.4 in Minnow (which I'll put up for download soon, I promise).

I spent some time rounding out the XML-RPC APIs for Minnow (a rant on the state of those APIs later), and hacking together a couple quick CRUD screens for web-based weblog administration.

Anyways, it was great meeting/seeing everyone - what a great community! All the sprint rooms are packed with all sorts of different people, old and young, hacking together, Quite a sight to behold- I don't think any other language has such a vibrant and active community..

PyCon Bound!

Tue, 22 Mar 2005 05:58:40 GMT

I'm all packed, ready to hit the sack and make my way to DC in the morning. I should be over by the convention center around noon, so look out for me :)

Minnow

Wed, 09 Mar 2005 04:41:52 GMT

Naming software is one of the hardest parts of my job. I think it's hard for everyone (at least in Python-land, judging from the amount of Python software named PySomethingOrOther). So I'm naming my yet-to-be-written blogging package Minnow, after my cat. Now we can move on.

I'm really thankful for all the feedback I've received regarding features. It will certainly help guide me as I design this thing. I'd love to hear more ideas.

Here is a rough list of features/design goals I've adopted so far

REST-based HTTP api: A tag search should be possible using something like http://myblogaddress.org/tags/tag1+tag2 (Shawn Wheatley)
The other ReST: Minnow should parse ReStructuredText, including easy highlighting of Python code snippets.
Multiple posting interfaces: Minnow should accept posts from editors like MarsEdit, from the commandline, the web, or email.
Easy deployment: I like the flexibility of the Roundup issue tracker in this regard. Robust commandline tools for creating and managing blog "instances".
Standard features: RSS, tags, search, archives, trackbacks, comments, etc.
Nifty features: Maybe a built-in moblog, integration with del.icio.us, who knows.

There's still some open ends. How much effort, if any, should be spent on different storage backends? I think I'm willing to accept whatever SQLObject offers, which is most open-source RDBMS's. I could add another thin layer above SQLObject and support things like a textfile backend, but that can always be done later.

How important is the ability to generate static HTML, or can the whole thing be dynamic?

What about presentation? Right now, I'm prototyping using Kid, which I'm really liking so far. I suppose as long as theres a nice API for the data model, replacing the templating system will be trivial for those who want something else. For the first version, users will have to be comfortable with seeing some exposed wires, as there probably won't be any higher-level 'skinning' facilities than the page templates themselves.

I realize that the number of dependencies is quite large (Twisted, SQLite, PySQLite, SQLObject, Kid... whew.. that is a lot), but it's not completely unreasonable, and a lot of people have these things inst alled already.

Obviously, there's still a bit to figure out. I need to take a look at some of the current blogging systems to get an idea of the state of the art, and kick around this prototype some more. In the next few days, I'll set up a Wiki as a place to brainstorm. Again, your suggestions, advice, flames, etc. are encouraged.

What Does Your Code Taste Like?

Sun, 06 Feb 2005 06:09:20 GMT

Martin Fowler introduced the concept of "code smells" in his book Refactoring, which I think is a clever way of explaining vague aesthetic notions in an intuitive, sensory way. I'm currently reading Structure and Interpretation of Computer Programs, the following two passages of which made me reflect on how I relate to code on a sensory level, a level more intangible and personal than one of classes, functions, filenames, and line numbers.

Every computer program is a model, hatched in the mind, of a real or mental process. These processes, arising from human experience and thought, are huge in number, intricate in detail, and at any time only pa rtially understood. They are modeled to our permanent satisfaction rarely by our computer programs. Thus even though our programs are carefully handcrafted discrete collections of symbols, mosaics of interlock ing functions, they continually evolve: we change them as our perception of the model deepens, enlarges, generalizes until the model ultimately attains a metastable place within still another model with which we struggle. The source of the exhilaration associated with computer programming is the continual unfolding within the mind and on the computer of mechanisms expressed as programs and the explosion of percepti on they generate. If art interprets our dreams, the computer executes them in the guise of programs!

...

Master software engineers have the ability to organize programs so that they can be reasonably sure that the resulting processes will perform the tasks intended. They can visualize the behavior of their systems in advance. They know how to structure programs so that unanticipated problems do not lead to catastrophic consequences, and when problems do arise, they can debug their programs. Well-designed computational systems, like well-designed automobiles or nuclear reactors, are designed in a modular manner, so that the parts can be constructed, replaced, and debugged separately.

After some reflection, I realized that below the surface, I have a mostly auditory model of program execution. For example, a script that processes logfiles has a simple, repetitive rhythm - one that probably varies with the nature of the process and it's input data. Algorithmic code I "hear" similarly, except the rhythm is more complex and accentuated, and varies according to the nature of the process - Dijkstra sounds different than a binary search.

Visualization is the most common way of expressing program execution as sensory data, like these call graphs of recursion and generator expressions in Python code:

But visualization is, in my opinion, a relatively high-level abstraction, and not the most personal, at least not for me. They say smell is the most emotionally evocative and memory-stimulating of the sens es, and after a few weeks on vacation, I find myself forgetting what family members and close friends look like. Not completely, but the mental picture isn't as sharp, even after only a short time away.

I mentioned the sound of my code to my cube-neighbor Greg, half-expecting him to tell me to lay off the LSD. He didn't think I was crazy, and related his internal model to me, one that was more visual - "textures" was the main abstraction he seemed to have.

I think a large part of the learning curve involved with learning new codebases, even after reading the API docs, knowing the module layout, maybe even reading some of the source, is the lack of a low level sensory model of how that code runs. I know I have one of these for Twisted (and asynchronous programming in general), and it differs from that of threaded code, but I'm afraid I lack the language to describ e it, because its components are so integral to all the abstractions I build, including language.

Expressible or not, I think exploring and understanding one's own low-level models of how their programs behave is valuable. Does anyone have other interesting models? Can any chefs out there taste their code? Or should I lay off the LSD :) ?

Python Module and Package Patterns

Mon, 31 Jan 2005 01:51:12 GMT

I was reading Ryan Tomayko's recent article about the success of Ruby on Rails, and he raises a couple of excellent questions about package and module layout and design that have been bothering me lately:

This may sound silly but I waste a ton of time trying to organize my project's source files.

How should database scripts, templates, other web resources, scripts, documentation, etc. be organized?

How should my packages be laid out? Do I create separate sub-packages for model, view, and controller? How about just separate modules? Or should I organize by object, keeping the model, view, and controll er implementations in a single module? I've tried all of these approaches and found that each has pros and cons but that consistency is always king.

Another aspect of Rails that I appreciated is that it seems to promote<97>and possibly even require<97>a strict organizational structure of source files. This structure can be generated for you by helper sc ripts.

This is the kind of stuff I think is underrated. It significantly lowers the barrier to entry for new programmers to get moving with the framework and lowers the amount of time it takes for experienced prog rammers to create new projects.

Not silly at all, actually. I spent a lot of time doing this too. It's trickier than it may seem, and getting it right can greatly ease the barriers to entry for would-be developers and extenders of one's work. For many people, including myself, this is still somewhat of an afterthought, nowhere near the typical "design" tasks on the priority list.

I don't think much about package layout up-front. I'll typically start with a single main package, with subpackages organized by functional category ('protocols', 'scripts', 'utilities', core', etc). Event ually, it becomes apparent that one or more of these packages are unnecessary, and they'll get moved back to the top level. As the API approaches completion, typically new packages get created that weren't en visioned at the beginning. Finally, a top-level facade module or two that offer convenience APIs and define the main API of the package.

I've also noticed a few other distinct package layout patterns in popular Python software:

The Single Python File

RSS.py and SOAP.py come to mind here. These modules are usually protocol implementations with few dependencies, and are well-suited for this format.

Psychologically, adding one of these to a project doesn't feel as 'heavyweight' a dependency as adding a distutils-based tarball does - which is not really true - but it's one of the peculiarities of a mod ule's marriage to the filesystem. I was never really comfortable with the Smalltalk approach, where the code I'm extending is a monolithic image, that includes the IDE and all manner of other cruft. When I'm working with someone elses code, a part of me wants to know what file a given class is in.

The Messy Flat Namespace

These are common. The Python standard library is one, but an excusable one given the nature of it's evolution. Small teams and single developers are more able to design and maintain a sensible module hier archy, yet the MFN layout is common among packages that should know better (including many of my earlier efforts).

Script Package

Twisted and PEAK use this one, where the logic of command-line scripts is kept in modules in a 'scripts' package, instanti ated by a single frontend script (`twistd` and `peak`, respectively).

The script module invoked is usually responsible for returning an object that conforms to a some sort of 'Command' or 'Application' interface.

Facade Module

__init__.py is one of these - an abstraction point for the functionality of the modules in a package. Frameworks like PEAK and Zope have extended package conventions, usual ly including an 'api' and 'interfaces' module.

Advanced versions of the Facade module, as seen in PEAK, support funky things like lazy loading and module inheritance.

These patterns address module layout, but none address what to do with the other project files, static templates, images, and data that should probably live 'closer' to the code than /usr/share or /var/ whatever. It would be nice to have a distutils binary format that does something like py2app and py2exe do - put everything in a single executable for deployment. There are many cases where one doesn't want t o tell a customer to install Python, or where one doesn't trust a customer not to bollocks his copy of Python. And again, the Single File is nice.

Finally, I'll agree with Ryan that this stuff is indeed underrated. One major difference between my personal projects and work projects is the amount of time spent on ease-of-use bells and whistles like stub code generation in the latter.

pyunparse 0.2: A Python AST Printer/Unparser

Thu, 27 Jan 2005 21:22:24 GMT

Here's pyunparse 0.2. Mostly cosmetic improvements, but I'm not so ashamed of the code, and it should be easier to grok and modify. Please download it if you are using the old version.

Going to PyCon!

Thu, 27 Jan 2005 05:01:04 GMT

My itinerary for PyCon just hit my inbox. This will be my first one, and the lineup of talks this year has me really looking forward to going. I can also provide several anonymous Pythonistas with the beers I owe them. :>

pyunparse: From AST to Python Source

Tue, 25 Jan 2005 22:45:36 GMT

I just got done hacking up a proof-of-concept Python "unparser" that converts Python Abstract Syntax Trees to concrete Python source code. Pyunparse is intended to be used as a debugging aid for apps that use the compiler.ast classes.

Check out the pyunparse page for more information, or to download the script.

py2dot: Visualizing Python Syntax Trees with Graphviz

Fri, 21 Jan 2005 06:39:12 GMT

py2dot is a useful script I wrote that generates Graphviz ".dot" files from Python source code. The output images can be used to teach parsing concepts, or as cubicle art:

Check out the py2dot page for more details or click here to download.

py2dot requires Graphviz and Python 2.4

Bytecodehacks Are Back!

Mon, 20 Dec 2004 22:37:00 GMT

Bytecodehacks was a great package for doing evil things with python bytecode - inline functions, semantic macros, design by contract - all sorts of good stuff. Unfortunately, it stopped working after Python 2.0.

Enter Christian Tismer of Stackless fame, who has re-released a bytecodehacks package that works with newer Pythons.

It *nearly* works out of the box with Python 2.4 too, with one minor change. Apparently the SET_LINENO instruction was removed in 2.4, but a couple of modules in bytecodehacks depend on it. I fixed this by adding a stub SET_LINENO class in ops.py:

class SET_LINENO:
      pass

That seems to fix it for me. Thanks, Christian!