Speno's Pythonic Avocado 2004/6

Tacoma Narrows: Ethernet Bridge Update

I've made some progess on turning my iBook into an ethernet bridge, but nothing is working yet. I've built my patched kernel which is supposed to allow ethernet spoofing. It's not working, however, and I haven't had time to debug it. I posted on the libnet mailing list about it and the response was "total silence." I'll look for help elsewhere soon.

I did find an open-source solution for bridging in userland: tcpreplay. Assuming you can etherspoof, this syntax should do the thing:

% sudo tcpreplay -b -S 0 -i en0 -j en1

Also, the libdnet provides a nice python interface for sending raw ethernet frames (and more!). Check it out:

import dnet

def bin2mac(b):
    """Turn binary mac address into nicer format."""
    h = b.encode('hex_codec')
    return ':'.join([h[i:i+2] for i in range(0,len(h),2)])

intf = 'en0'
packet = 'your raw ethernet frame here'

datalink = dnet.eth('en0')
mac = bin2mac(datalink.get())
print '%s has address %s' % (intf, mac)
datalink.send(packet)

Take care.

A. Avocados all around!

Ah! The humble avocado. My favorite fruit. Given the name of this blog, it's time I said something about them.

The haas variety of avocado are the most common. With so much fat and fiber, they are low-carb friendly , delicious, and very good for you . Alas, they're expensive around here. The best regular price I've seen is $1.50 per fruit. A few times a year, the local Whole Foods has them on sale for $1, but you can only use so many before they turn brown and mushy. I buy them firm, and ripen them in individual brown paper bags, checking on them after a few days. There's something very satisfying about cutting open a perfect avocado...

Danger! Trader Joe's is now selling bags of frozen, pitted avocado halves. These things are the most vile, disgusting tasting avocados I have ever had the misfortune of attempting to eat. You have been warned. Stick with their pre-made guacamole, which is actually a pretty good base for making your own guac by adding ripe jersey tomatoes and fresh cilantro.

Allez cuisine!

Starting an Ethernet bridge in Python

I'm trying to write an ethernet bridge in Python on MacOS X. The idea is simple enough: take an ethernet frame from one interface and send it out a different interface skipping over those frames that are destined for the bridge itself (my iBook). It's probably not as hard as it sounds. At least I hope it won't be.

Getting packets off of an interface is easy using pcapy. Inspecting ethernet frames so that we don't forward ones that are addressed to the brige device is easy using Impacket. Alternativley, we can use a capture filter with pcapy to avoid getting our own frames. That part's already done, leaving just the hard parts to solve. However, there are a few problems to overcome first.

The major problem is that MacOS X will re-write the source of any ethernet frame such that it appears to come from the interface it was written to. Obviously, this will totally break our bridge. I have filed a bug report with Apple about this. In the meantime, you can build your own patched kernel to fix this problem.

Once I get my new kernel running, I'll need a way to write already formed ethernet frames to the network. The Python socket module doesn't support this on anything but Linux, where the AF_PACKET address family is implemented. I'll have to find an alternative such as using libnet, extending Python's own socketmodule.c or making a simple rawsocket module using Pyrex. Ideas and code are welcome, of course!

Take care.

Baby Steps

On a recent Saturday morning, I was called upon to extract data out of my family's web site so that a new site could be constructed using fancy web standards of which I know nothing. The old html files were in good order so it was easy to get the data we needed from them.

The pages make up an online photo album with captions. As the captions and images were in the same order, I was able to write this data scraping class fairly quickly to get all the unique bits out of the pages:

import sgmllib
from cgi import escape

class MSParser(sgmllib.SGMLParser):
    def __init__(self):
        self.dates = {}
        self.datelist = []
        self.curr_date = ''
        self.in_caption = False
        self.in_image = False
        self.in_datebox = False
        self.data = []
        sgmllib.SGMLParser.__init__(self)

    def unknown_starttag(self, tag, attrs):
        """We want anything in captions, including other tags."""
        if self.in_caption:
            s = '<%s ' % tag
            for a, v in attrs:
                s = '%s %s=%r' % (s, a, v)
            self.data.append(s)
            self.data.append('>')

    def start_p(self, attrs):
        """Ignore any p tags in captions"""
        if self.in_caption:
            pass

    def end_p(self):
        """Ignore any p tags in captions"""
        if self.in_caption:
            pass

    def start_td(self, attrs):
        for attr, value in attrs:
            if value == 'DateBox':
                self.in_datebox = True
            elif value == 'CaptionBox':
                self.in_caption = True

    def start_img(self, attrs):
        for attr, value in attrs:
            if attr == 'src':
                img = value
            elif attr == 'width':
                width = value
            elif attr == 'height':
                height = value
        img_data = (img, width, height)
        self.dates[self.curr_date]['images'].append(img_data)

    def end_td(self):
        if self.in_caption:
            caption = ''.join(self.data)
            self.dates[self.curr_date]['captions'].append(caption)
            self.data = [] 
            self.in_caption = False

    def unknown_endtag(self, tag):
        if self.in_caption:
            self.data.append('</%s>' % tag)

    def handle_data(self, text):
        if self.in_caption:
            self.data.append(escape(text, quote=True))
        if self.in_datebox:
            date = date.strip()
            if date: 
                self.curr_date = date
                self.datelist.append(date)
                self.dates[date] = {}
                self.dates[date]['captions'] = []
                self.dates[date]['images'] = []
                self.in_datebox = False

    def error(self, message):
        pass

After that, much more work went into automatically generating the new set of pages based on the dates, captions and image links I had obtained using that MSParser class. I think the resulting pages are fantastic looking. I may be a tad biased, however.

Now we have to think about a new way of keeping that site up to date. It doesn't make any sense to write new pages by hand when you can add new content to a database and have a program generate the resulting pages automatically. Baby steps. Literally.

The snake of good Omen

I've had good luck with all the (two) contract Python programmmers that we've hired in the past year. In both cases, they were the only people with any Python experience available immediately, and in both cases we hired them after one short interview. This says good things about both of them, but I also think it may say something good about Python.<wink>

And how is it for me going from a solo programmer to a team leader? Wonderful and difficult. It's wonderful to share ideas and solve problems with another programmer. Our products are better as a result. Also, we're way more productive as a team then when I'm working by myself by several factors.

It's difficult because I can sometimes fall behind dealing with all of the other issues I'm responsible for, while the other programmer only has to worry about one project. I'm trying to work well, and not fast, but I often have days where I feel lucky if I make any progress at all on Project Albatross.