Rudy's personal scratchpad

At last, up and running again

I've had a hard disk crash and lost everything. Lucky for me I had a data backup of a couple of days old. But restoring data is the easy part, I had to reinstall everything and that takes time.

In the mean time I started another hobby: genealogy. I must admit that this one has bitten me good.

I spend most of my free time researching my ancestors and their descendants.

I've been lucky so far that I could build upon the work from some other people and I already have about 2600 people in my family tree.

I'm thinking about starting a second PyDS only for my family tree. That one would then be in dutch of course.

I'm running 2 instances of PyDS as a Windows service now.

I wanted to use PyDS for some of my other hobbies too and I didn't want to mix with this instance. I prefer to use this one only for computer related things. I guess I'll move the jokes too.

There were 2 problems I had to take care of: the location of the data files on my laptop and the serverport.

The serverport could easily be configured in %PYDS_HOME%etcPyDSConfig.py but I found it to be easier adding one line to pandora-start.py.

PyDS first looks for the PYDS_HOME environment variable to find the location of its homedir. If it isn't found, it looks for HOME. On Windows, if that failes also it uses:

home = os.environ['HOMEDRIVE'] + os.environ['HOMEPATH'] 

The problem here is that you can only specify one location. The solution was to set the PYDS_HOME environment variable in pandora-start.py as you can see below:

import sys
import getopt
import os

os.environ['PYDS_HOME'] = r"G:\PyDS\Pandora"

import PyDS.ConfigLoader

_PyDS.serverport = 4335

import PyDS.Context
import PyDS.Translation

print _("Loading desktop server ...")

import PyDS.Server

(opts, args) = getopt.getopt(sys.argv[1:], 'fved')

for o in opts:
    if o[0] == '-f':
        _PyDS.daemon = 0
    elif o[0] == '-v':
        _PyDS.verbose = 1
    elif o[0] == '-e':
        _PyDS.echoLog = 1
    elif o[0] == '-d':
        import libwadpy

PyDS.Server.start_server(daemon=_PyDS.daemon)

Memory leak in PyDS?

The following picture clearly shows there is a memory leak in PyDS. Given the frequency at which the memory increases it must be in the aggregator section. All of my RSS feeds update every 60 minutes.

I'll try to find the time to debug as proposed by Jeremy Bowers:

If you want to debug this, I'd recommend whipping up a little tool that when poked, runs gc.collect then dumps the output of gc.get_objects out to the HTML. You'll have a lot to wade through, but odds are, if there is a memory leak, you'll see tons upon tons of one particular item that stands out, especially if you've run this for a while.

How far can you go in customizing PyDS?

Just some things I'd have to find out:

• Can you configure different "categories" of Pictures? Or, how would I separate the pictures from my last trip to Italy and those of my daughter's birthday party?
• Is it possible to have different style-sheets per Category. Can I have comments and backtracks enabled for one and disabled for another category?
• Can I have different upstream settings per category? The normal text ones to PyCS and the photos via FTP to my iSP webspace? Or should I try to have two instances of PyDS running at different ports?
• Can I easily extend PyDS to fetch my email from my POP server?

Need to solve the proxy problem (not only for PyDS)

First of all I'll explain why I have a problem with proxies. When I'm working at home I could use the ISP's proxyserver but I don't need to. When I'm working at the office I MUST use a proxyserver, but which one depends on the location. Fortunatley my company provides an autoproxy configuration script. When I'm working on a customer's LAN it all depends. Some do not allow Internet access, other don't need proxy servers, some have a fixed proxyserver and only a few use autoconfig scripts.

The problem I'm facing now is that I don't have a solution for all situations.

I've been looking at how urllib.py determines which proxy server it should use for which protocol, and I can't say I'm happy about it. The getproxies function first looks if there are environment variables of the form protocol_proxy (e.g. http_proxy) and uses those if found. If not, it looks in the Windows registry (HKCUSoftwareMicrosoftWindowsCurrentVersionInternet Settings) for ProxyEnable and ProxyServer.

Because I'm running PyDS as a service (as LocalSystem) (thanks to Georg) now, I not even sure which user is used in the registry (certainly not myself). I could run the service under my own accoutn and have that problem solved, but then I shouldn't forget to go and change the password for the service each time I renew mine. The environment variables are problematic too because Localsystem only sees changes at the next reboot. Changing to my useraccount would mean that I would need to restart the service every time the proxy settings change.

So, suppose I change the sevice account for PyDS into mine, I would still have a problem because in some of the situations I'd have to use a autoproxy configscript in which case I'd have to clear the ProxyEnable registry setting. I'll have to think about this one.

I change networks at least 3 times a day when working and I almost never reboot my laptop. This brings me to the next challenge: changing the proxy settings automagically. Now, that shouldn't be too hard. I can create a configuration table containing the correct proxy settings for each network I'm on ( combination of IP-range and domainname). I could write a PyDS-tool that checks the network every minute (shoudn't be that hard) and then changes the proxy settings according to the network I'm on at that moment.

PyDS running as a Windows service

Yep, Georg did read my yesterday posting. He even added comment which I didn't see until now looking at my cloud.

This adds new atsks on my todo list: - How can I get notified when someone comments my posts? - What is the best way to put links to other websites in my Desktop Log - How do I create separate sections (Python, music, jokes, ...)

Internet is too big, not enough time to read it all

This morning, I started reading the stuff the aggregator collected from the dozen blogs I configured, and now at 19.00 I still haven't finished. I even didn't find the time to finish reading the PyDS User Guide and the reStructuredText manual.

Since I will be sitting/laying in the couch for the next couple of weeks (broken ankles), I'll have plenty of time to explore and adapt PyDS the way I like. I guess Georg Bauer must have seen my remark of yesterday about the Command Prompt window, because this morning he explained on the PyDS-dev mailinglist that the tsctl-win32.py script could also be used to start PyDS as a Windows service. That is the first item on my todo list.

Another thing I'll have to find out is how PyDS fetches its information. Once I'm back at work I'll have the problem of being behind a firewall and I'll have to use a proxy server then.

I really start to like PyDS

When Pythons Attack

pyXLWriter 0.2.0a3

This is my first post on my new weblog

I discovered the Python Desktop Server a while ago, and finally found the time to set it up. Had I known it would be this simple, I hadn't waited that long.

Not only do you get a cool blogtool, you also get free blogspace in three languages: englisg, german or japanese.