Peter's Blog 2004/7

Exim recipe to forward users in local domain to another smtp server

Problem: an email server has a couple of local user accounts defined but the remaining users are handled by another server. How to route email to them?

Edit exim.conf. The last rule of the routing configuration should have the term

    cannot_route_message = Unknown user

in it to bin messages that cannot be delivered. Comment that line out and add the following clause as the last routing term:

  #
  # pcw: route users in the local domain but not on this server to gateway
  #
  send_to_gateway:
    driver = manualroute
    domains = +local_domains
    transport = remote_smtp
    route_list = * gateway.mycompany.com
    cannot_route_message = Unknown user

Wrapping text around Photos

The following can be used in PyDS to wrap text around pictures:

This makes me happy.

Posting Work Blog entrys on Public Blog.

At work I am using Drupal as a CMS for blogging, organising project related information etc. I feel it is more suited to a collaborative role than PyDS which is essentially a private tool for publishing to a community server. The community servers available do not appear to be strong CMS systems.

Drupal is very nicely written and even I, a PHP n00b, have managed to write a Drupal module that enables me to email a posting from the work server to my home PyDS installation where it can be posted to my personal/public blog. The email->PyDS gateway that I set up for photo blogging does the posting of the email to the blog.

Main stubling block encountered was when random pieces of the email message received were missing. This turned out to be because the message was stored in the database with \r line delimiters rather than \n. When the message was transmitted this looked to the email stuff like VERY long lines so it proceeded to screw up. I'm not sure where these came from, whether the web browser uses these in text fields or what. This script turns them to \n's which everybody loves (except dos/mac zealots).

Here is my Drupal 'sendhome.module' which does the posting.

<?php
/*
 * Add option to send postings to home.
 * Inspired by the emailpage module but simplified.
 * No admin options, edit this module.
 */

function sendhome_help($section = 'admin/help#sendhome') {
  $output = "";

  switch ($section) {
  case 'admin/system/modules#description':
    $output = t("Send page home");
    break;
  }
  return $output;
}

function sendhome_link($type, $node=0, $main) {
  global $user;

  $links = array();

  /*
   * If logged in user is moi, give me the option of sending this home.
   */
  if( $user->name == 'my_drupal_user_name') {
    if (($type == "system")) {
    // URL, page title, func called for page content, weight, 1 = don't disp menu
      menu("sendhome", t("Send Page Home"), "sendhome_page", 1, MENU_HIDE);
    }

    $links[] = l(t("Send Page Home"), "sendhome/confirm/$node->nid", array("title" => t("Send Page Home.")));
  }
  return $links;
}

function sendhome_page() {
  /*
   * Get function parameters.
   */
  $op = $_POST[ "op"];
  $edit = $_POST[ "edit"];

  if( empty($op)) {
    /*
     * In initial link nid is in the url
     */
    if( arg(1) == 'confirm') {
      $nid = arg(2);
    }
  } else {
    /*
     * From the send button the nid is in a hidden field.
     */
    $nid = $edit[ "nid"];
  }

  /*
   * Search db for nodes.
   */
  $nodes = db_query("SELECT nid, title, teaser, created FROM {node} WHERE nid = %d", $nid);

  drupal_set_breadcrumb(array(l(t('Home'), NULL), l(t('View'), "node/view/$nid")));

  while ($node = db_fetch_object($nodes)) {
    /*
    ** Load the specified node:
    */
    $item = node_load(array('nid' => $node->nid));

    /*
     * Add the title and the body to the output under construction.
     */
    $output .= $item->title;
    $output .= "\n";
    /* Pass body through filters to create html */
    $output .= check_output( str_replace( '<!--break-->', '', $item->body));
  }
  /*
  * Maybe it's my fault for using windows but \r line delimiters lead to VERY
  * long lines that don't make it through email without random bits missing.
  */
  $output = str_replace( "\r", "\n", $output);
  if ($op != t("Send")) {
    /*
     * Display the posting that is to be sent and a send button.
     */
    $output = "<div class=\"node\">$output</div>";
    $output .= form( form_submit(t("Send")) . form_hidden("nid", $nid));
    print theme( "page", $output, "Post Message Home?");
  }
  else {
    /*
     * Set email parameters.
     */
    $strTo = "me@home.com";
    $strFrom = "me@work.com";
    $strMagicSubjectLine = "post this for me";

    /*
     * Send operation so send the post and show a 'Done' page.
     */
    mail( $strTo, $strMagicSubjectLine, $output, "From: $strFrom\r\n", "-f$strFrom");
    print theme( "page", '', "Posted Message");
  }
}

?>

PyDS Upstreaming Woes

Python Desktop Server has been repeatedly upstreamed two files this evening, wiki/Index.html and wiki/index.html and has failed to upstream a long blog post I made earlier this evening. Earlier it raised a few internal errors while trying to upload postings that had already been deleted:

Background thread exception exceptions.ValueError: Invalid Posting id:    
P393Exception instance __dict__: {'args': ('Invalid Posting id: P93',)}

/usr/lib/python2.3/site-packages/PyDS/Tool.py[1300] in thunk
/usr/lib/python2.3/site-packages/PyDS/MirrorTool.py[77] in _initthread
/usr/lib/python2.3/site-packages/PyDS/MirrorTool.py[128] in _addItem
/usr/lib/python2.3/site-packages/PyDS/WeblogTool.py[1165] in getRSSItem
/usr/lib/python2.3/site-packages/PyDS/WeblogTool.py[889] in getPost

The way upstreaming works has a feel of something beyond my control where I have to cross my fingers and hope and wait patiently and keep refreshing the cloud until my changes appear.

How to do Photo Blogging

As promised, how to do photo blogging from mobile phone. Actually this should support submitting plain text blog posts from any email source as I want to use the same mechanism to post blog entries from work that are for public display, bypassing the firewall. This recipe works with UK O2 MMS messages.

The first line of text in the email message becomes the title of the post, the following lines become the body.

Requirements:

• Exim (or any MTA that supports forward files)

• Python Desktop Server (or any server that supports MetaWeblogAPI or Blogger API, although you have to hack the image upload yourself, depending on the server).

• Python (or you can rewrite it all in some other language)

Exim has to be set up to allow .forward files to process email. The .forward file should look like this:

# Exim filter
logfile ~/.forward.log

if $header_subject is "Multimedia message"
then
    pipe "/home/me/MailBot.py"
    seen finish
endif

This will direct emails with this subject line (i.e. UK O2 MMS messages) to the python script that does the hard work. This script is as follows and has to be made executable by the exim process:

I had to make a small hack in Python Desktop Server. Posted articles default to either raw html or strict structured text but the image link requires structured text + cheetah. I had to edit the file MetaWebWeblogAPI.py to change the two lines that said:

to

ToDo:

• Try to get text to wrap round image

• Save up to pay my mobile bill

Photo blogging

I've set up an exim forward script to allow me to submit photo blog articles from my mobile. I'm typing this post into my new mobile.

The pc on the left is the gentoo server and on the right is the windows box. Fascinating huh? The windows box is clad in it's shiny new case.

I'll post the scripts later. I don't really want to type them into the phone.

typeGenerator

I furled it but I've also gotta blogplug this site which is great for generating custom wallpaper.

New Nokia 6610i

My phone contract was up for renewal after 12 months so I looked into my options. I ended up changing tariff to one that will cost me £16 a month instead of £30, I got a new nokia 6610i for £19.99 and I keep my year old 6610 as a spare. Over the next year I save £150 and I have a new phone. How do they make money on mobiles? New phone has a crappy camera but that's better than no camera.

I am a nokia loyalist: I had a Sony-Ericson T68i for a while but it had very poor reception and was very slow to use. I gave it to someone else and, embarassingly, it stopped working due to a generic problem that they moan about on the online forums.

Other savings:

• £9 for a 16" fan

• One-for-all 4 for £10

• 4 Lith-ion rechargable AAA batteries for £3

At dixons 1/2 price closing down sale.

One-for-all works fine with Sky+ and means just one remote by the bed. Ergonomically it is awful.

Keyboard Shortcuts

Furl recommendations always seems to yield something interesting. This time it's Microsofts list of keyboard shortcuts. New ones to me that will be useful IF I can remember to use them are:

• Windows Logo+BREAK (Display the System Properties dialog box)

• Windows Logo+D (Display the desktop)

• Windows Logo+M (Minimize all of the windows)

• Windows Logo+SHIFT+M (Restore the minimized windows)

• Windows Logo+E (Open My Computer)

• Windows Logo+R (Open the Run dialog box)

I always wondered what the windows key was for. I thought it was like scroll lock. Hope they work on Win2k. UPDATE: They do.

WBloggar

Jul 22 16:53:54 ad-pc fetchmail[1492]: awakened at Thu Jul 22 16:53:54 2004
Jul 22 16:54:00 ad-pc fetchmail[1492]: sleeping at Thu Jul 22 16:54:00 2004
Jul 22 16:54:30 ad-pc fetchmail[1492]: awakened at Thu Jul 22 16:54:30 2004
Jul 22 16:54:36 ad-pc fetchmail[1492]: sleeping at Thu Jul 22 16:54:36 2004
Jul 22 16:55:06 ad-pc fetchmail[1492]: awakened at Thu Jul 22 16:55:06 2004
Jul 22 16:55:13 ad-pc fetchmail[1492]: sleeping at Thu Jul 22 16:55:13 2004

#
# Convert unicode to good old ascii
#
def DeUnicode( strString, bStripLF = 0):
    strOut = strString.encode( "iso8859_15", "backslashreplace")
    strOut = strOut.replace( '\\u2022', '*')
    strOut = strOut.replace( '\r', '')

    return strOut

Google Rankings

My name (Peter Wilkinson) is quite common and does not appear in the first 5 google pages. However, my wife's name is less common and by subtly mentioning her at the bottom of this page I have managed to get her name (Bisi Wilkinson) to bring my blog up at number 10. Petty or what?

Life with Gentoo

Two Gentoo servers, one at work, one at home. The one at home seems fine, perfectly stable. The one at work has a problem where once every few days a few deamons will drop dead: syslog-ng, cupsd, apache, atd, cron etc. It is not always the same pattern of daemons and I'm having trouble working out the problem.

More often than not the first sign is that syslog-ng stops logging to /var/logs/messages. It may be tied up with cron running logrotate and the htdig indexer. Logrotate is set up with the standard config script for syslog-ng and I've checked the Gentoo CVS to make sure that it is up to date and I've run it from the command line with no ill effects.

My suspicion is that it is tied up with htdig indexer for a few reasons:

• I'm not running it at home and home pc is ok.

• When it runs it can tie the pc up heavily for a few seconds, long enough for IMAP clients to complain about loss of connection.

• CRON keeps sending me error messages like this:

  /usr/bin/rundig: line 53: 11495 Aborted                 $BINDIR/htnotify $opts
  

Because it happens so infrequently I am waiting for the next occurence to do another post mortem.

I'd say Gentoo was hard work.

I think I did an emerge -u world last night but the at daemon hasn't sent me any mail. It seems to have happened because emerge --pretend -u world gives a different list compared to before.

The list of updated files includes vixie-cron and syslog-ng so it is worth doing.

Remote X Windows

I have managed to get remote X windows working via SSH. From a Windows 2000 desktop I can run X applications on my server at home. The steps involved are approximately as follows (i.e. I frigged around a bit but I think this will work):

• Install Cygwin. In particular, install the openssh and X11 base packages.

• Edit the sshd_config file on the remote server and make sure this is set:

  X11Forwarding yes
  

  and restart sshd. This is important for avoiding the need for xauth

• start X on the remote computer.

• on the remote computer run:

  xhost +
  

  This turns off strict access control. I am not entirely sure this command is needed when using ssh X port forwarding but I don't think it will hurt. Both local and remote systems are behind firewalls with an encrypted ssh link between then and I am not paranoid enough to worry about extra security.

• on the local pc run the cygwin command startxwin.bat. This should bring up a window with a X desktop in it.

• from a terminal window on the X desktop, run an ssh command of the form:

  ssh -X -l <username> <host address>
  

  the -X tells ssh to do magic X port forwarding ¹.

• again in the terminal window, do this to check things are ok:

  export | grep DISPLAY
  

  you should see that the DISPLAY variable has been set to something like 127.0.0.1:10.0. Setting this up is the job of the sshd server. If it is not there then the X port forwarding is broken.

• run an X program, e.g.:

  xterm &
  

You should get a new term on your desktop running from the remote computer. I got as far as running mozilla across this but it was too slow to be of any use (12kbytes/sec upload speed achieved by Bit Torrent yesterday).

Cygwin defaults to twm which is a little primitive. I tried WindowMaker but that crashed. fvwm2 works, looks a little prettier than twm but is not much more sophisticated.

It's cool, but an awful lot can be done with just a straight ssh console.

[1]	the howto on the Cygwin site says '-Y' which is rejected by ssh.

Bit Torrent

Using Bit Torrent to download the lastest Knoppix CD. It's coming down at about 60KB/s which is as high as I ever see. It's uploading at 8k/s so I'm doing someone a favour.

It is intersting in that it is a python application. It normally uses wxPython for it's gui and it installs only the following files:

bittorrent.ico
btdownloadgui.exe
datetime.pyd
library.zip
LICENSE.txt
python23.dll
redirdonate.html
select.pyd
uninstall.exe
w9xpopen.exe
wxc.pyd
wxmsw24h.dll
zlib.pyd
_socket.pyd
_sre.pyd
_ssl.pyd
_winreg.pyd

Interesting, just the dll's that are necessary. It does make it big though, this lot adds up to 10M, mostly unused library code.

The source includes a headless version that runs a treat on the Gentoo box so I have no qualms about leaving it running to upload to my peers. There is a Gentoo ebuild for bitTorrent but that has a whole lot of dependencies, including gtk and wxPython, which it wanted to build from source.

Dynamic Web Content

I've been looking around the options for dynamic web content recently for various projects. One of them is to be hosted under IIS, the other I can do with as I please. The IIS based one required a CMS so that many people can add content. The original request was for something that used ASP, VBScript and Access databases and was not .NET. I searched around and found nothing suitable. Widening the search I came across Drupal which seemed to fit the bill except that it was PHP and MySQL.

At about the same time I was considering this I came across the PHP EasyWindows Installer which installs PHP as a CGI engine which is cool enough, I don't think I have to worry about high volume traffic.

So I installed it and MySQL on Windows XP and it runs a treat. The pages are server up quite swiftly and the package looks quite powerful. It looks nicely designed: a clear segragation between content (all stored in the database) layout (a single, nicely laid out template file that builds all the pages on the site, easily edited in HTML-Kit) and formatting (.CSS).

I'm still exploring it but it is looking suitable for my needs and the person I am doing it for has agreed to install PHP and MySQL on his windows box. I'm not reluctant to learn PHP although I find that after python all the $s in the code look like noise: not as bad as perl (from which it is distantly related) but nearly so. The drupal source that I have looked through is quite clean. There is not a great deal of documentation but that is par for the course. I might not even need to write any PHP, I've found modules to do most of the things I want.

One nice thing about Drupal that Python Desktop Server is lacking: a button to preview messages before submitting them.

For my other project I am looking into a way to present email archives on a web site. I've looked through various options, none of which appealed:

HyperLink

Crude presentation, no Gentoo ebuild.

MHonarch

Gentoo package failed to emerge

Macho

Nice presentation but an embarrasing name to search for on the net and also written in lisp (((ugh))). I don't really want to install yet another language and I certainly don't want to learn lisp.

Mailman

GNU list manager thing which includes a modern version of PiperMail which was what I wanted (python email->web code). I installed it but was boggled by the complexity so I uninstalled it quick. Life is too short. There are various other list managers I could hack on but it's a matter of finding the small part for presenting the archives amongst the rest of it. Plus I need to be sure they can handle multiple addresses, CC's, attachments, html mail etc.

Mod_Python

So I gave mod_python a whirl. It installed easily enough but I soon ran into a problem whereby if I generated too much HTML (two or three pages) then I'd get a segmentation error and no response from Apache. It's probably another Gentoo version mismatch problem but I didn't want to get involved so I ditched mod_python.

I finally decided upon python CGI. This was easy to set up and gives me total control. No learning curve beyond the cgi module and I can use the IMAP library to access my email.