I've been getting a lot of attention for my moblog even though it's not such a big technical feat really. All you need is a script to convert email into MT entires. We're going to make the code available so that hopefully more people can experiment with moblogging. It's being made available under a standard GPL v.2 license. Please let me know what you think. We hope to continue to improve it. The file is available at: http://joi.ito.com/archives/src/mail2entry-2002-12-25.tgz

Here are some notes about installation.

Warning

This is a first attempt -- there are some things missing, but someone w/appropriate UNIX experience should be able to succeed in installation and configuration. I may update the instructions as I get feedback.

Prerequisites

The ability to read English and follow simple instructions.

A host connected to the Internet which can receive email [1] and:

Python >= 2.2.2
MT
enabled w/ the XML-RPC interface
a blog (determine your blog id)
a blog user, the associated password [2]

Please note that this code was designed to run on the host which the blog itself is running on. Other configurations may be possible, but I don't recommend them and don't plan on supporting them at the moment.

Thanks for all the work on this Sen

Please send feedback to me for the moment. We may designate someone to be in charge of the code later. Also, since it uses XML-RPC, it should be easy to get it to work with other blog platforms.

PS The GPL license on this code overrides the CC license on this site.

21 Comments

Of course Radio and Blogger both have mail to post features...

Hey, if you want to mail to your weblog and you use Radio, no need to wait. It's had the feature since the product was released on 1/11/02. All you need is a mail account for it to use and then turn it on and you're ready to go. It works. No problem.
Dave's right of course. I haven't tried sending images to Radio or blogger, but we've done a few things to cause the attached images to be saved into a directory and included in the entry as an embedded image. We'll be tuning it more and more for mobile image moblogging though...

btw, I have written similiar tool for Userland's Manila, see sample working at nipernaadi.kolhoos.ee with 690 pictures from around Estonia, taken by radio reporters this summer. Although the site is in Estonian language I wrote English description for the tool and I think there are couple of things worth stealing: for example automagically created icons, macro to display 10 random icons (with links to original images), option to "flip" pages where images + messages are stored etc.

Some problems right now with the code. The manual pipe doesn't work now.

aissata# cat /home/karl/essai | python mail2entry.py > toto
mail2entry.py:8: SyntaxWarning: import * only allowed at module level
def main():
Traceback (most recent call last):
File "mail2entry.py", line 50, in ?
main()
File "mail2entry.py", line 43, in main
return result
UnboundLocalError: local variable 'result' referenced before assignment

toto has the right format with a jpeg file in it.

Content-Type: image/jpeg
Content-Disposition: attachment; filename="essai.jpg"
Content-Transfer-Encoding: base64


Any ideas of the problem

First problem has been solved. But it seems that the program react very differently depending on the mailer. I had for example a message about UTF-8.

My problems seems now to be related to xmlrpc.cgi

Mobicarnet - first french Moblog

The pictures are saved as now.jpg and the secondary name but the post doesn't work yet. I'm trying to work out why.

Traceback (most recent call last):
File "/my_full_path/web/mobicarnet/mail2entry.py", line 34, in main
result = postentry.post(content)
File "postentry.py", line 38, in post
content, publish)
File "/usr/local/lib/python2.2/xmlrpclib.py", line 821, in __call__
return self.__send(self.__name, args)
File "/usr/local/lib/python2.2/xmlrpclib.py", line 975, in __request
verbose=self.__verbose
File "/usr/local/lib/python2.2/xmlrpclib.py", line 840, in request
self.send_content(h, request_body)
File "/usr/local/lib/python2.2/xmlrpclib.py", line 876, in send_content
connection.endheaders()
File "/usr/local/lib/python2.2/httplib.py", line 695, in endheaders
self._send_output()
File "/usr/local/lib/python2.2/httplib.py", line 581, in _send_output
self.send(msg)
File "/usr/local/lib/python2.2/httplib.py", line 548, in send
self.connect()
File "/usr/local/lib/python2.2/httplib.py", line 941, in connect
sock.connect((self.host, self.port))
error: (61, 'Connection refused')

Looking at the error you posted, I guess that some network connection is failing to establish -- note:

error: (61, 'Connection refused')

which is the last line of your posting.

Some number of lines above, the xmlrpc library is referenced, so one might guess that the problem lies in trying to establish a connection w/ the XMLRPC portion of MT.

If this guess is correct, there could be a number of different explanations for the current situation. Perhaps it would be worth testing whether XMLRPC is working for your MT configuration [1].

HTH

[1] It looks like XMLRPC capability is supposed to be enabled for this program to work -- at least the entry text lists:

MT
enabled w/ the XML-RPC interface

as one of the requirements.

it is working. I will complete your INSTALL manual to explain a few things

Thanks

* Some testing and examples

- Email that doesn't work at all.
in the mail headers

Mime-Version: 1.0
Content-Type: multipart/mixed;boundary="==IMail_v5.0=="

And after in the body

--==IMail_v5.0==
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Here Text

--==IMail_v5.0==
Content-Type: application/octet-stream; name="chalet.jpg"
Content-Transfer-Encoding: base64

here the coded information


For this email, the image is not saved and the post doesn't happen at all.

the tracelog says

Traceback (most recent call last):
File "mail2entry.py", line 28, in main
content, imagefilename = parsemsg.parse(sys.stdin)
File "parsemsg.py", line 65, in parse
imagefilename = handle_image_portion(second_part)
File "parsemsg.py", line 79, in handle_image_portion
raise SecondMIMEPartNotImage
SecondMIMEPartNotImage

* BodyNotUTF8 Problems

With mail with this header

MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="------------070602060307040204040407"


and with this body

This is a multi-part message in MIME format.
--------------070602060307040204040407
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Here Text

--------------070602060307040204040407
Content-Type: image/jpeg;
name="tristan-mobicarnet.jpg"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename="tristan-mobicarnet.jpg"

Here the encoding

The image is saved but the text is not posted to the blog.

the tracelog message is

Traceback (most recent call last):
File "mail2entry.py", line 34, in main
result = postentry.post(content)
File "postentry.py", line 25, in post
raise BodyNotUTF8
BodyNotUTF8

A few other things for the people who install it.

settings.py

"""Profile settings"""

username = "Photo User"
password = "somethingrelevant"
blogid = 3
publish = True
uri = "https://blog.example.org/cgi-bin/mt-xmlrpc.cgi"

"""Be careful, some servers do not have https authorized, you can use http simply instead"""

imagesdirpath = "/var/www/blog/archives/images"
imageurldir = "/blog/archives/images"
"""Be careful to not have a too long name. Images have already a long name and if the directory name is too long, some browsers will not be able to load the URI"""

template = "

%(caption)s
\n" + \
"
"
"""Change the template mark-up for span instead of div. Right now if you leave it like that the page will generate an invalid code by putting p around div."""
logfilepath = "/tmp/log-tracebacks.log"

* Mail redirection in sendmail.

Some sendmail installation does not authorize the piping at the /etc/mail/aliases level or /etc/mail/virtusertable. In this case, a possible solution is to put the piping in .procmailrc in the directory of the email address if it's a user.

like

:0 Hc
*
|/usr/local/bin/python /Path/mail2entry.py /yourMoblogPath/img


Hope it helps.

OK I think I start to understand the email parsing problems. I have read the source of your parsemsg.py. You made it too specific for Japanese community. You should make it more international.

I can read for example

def parse(fileObject):
"""Parse a MIME message w/ two parts, where the first one contains
text (ISO-2022-JP or ASCII) and the second one contains an image,
w/ an optional subject header (decoded to ISO-2022-JP or ASCII).
Successful parsing should return a tuple w/ a content dictionary
suitable for posting and filename for a file containing the extracted
image."""

Many emails in the world are now posted in iso-8859-1 or UTF-8

So I think one of the first think to fix is to handle more common encoding:


def unicodify(string, charset):
"""Converts an ISO-2022-JP or US-ASCII string to unicode. A string using
another charset results in an empty string being returned."""

result = ""

if charset:
charset = charset.lower()
if charset == 'iso-2022-jp':
result = unicode(string, "japanese.iso-2022-jp")
elif charset == 'us-ascii':
result = unicode(string, "us-ascii")
else:
result = ""
else:
result = unicode(string, "us-ascii")

return result


For testing purpose it would be good to have a flag to just have an output of the two part (1 file for the image and 1 file for the text output) and a verbose mode of error on the console.

I'm travelling now, but I'm back home on Monday, I will be able to start working on the code to make it more tolerant and general.

Does it suit your needs?

Thanks Karl!

If anyone wants something similar that uses perl instead of python (and doesn't mind getting their hands just a little dirty editing the perl to taste), I've got one here. Just get rid of any lines with decode_iso_2002_jp in them, create a suitable .blogpostrc file and add a recipe to your procmail.

See also system overview. It's a crock but it works :)

Thanks for this Dav. I put it in the Moblog Resources Page

I've been trying to set this up on my server but I'm a new hand at installing scripts. I saved a message in the correct format to use as a test file but when I run the test I'm getting an error. I can't determine what the problem is myself...hopefully someone here will be able to nail it down for me.

bash-2.05$ cat testmoblog.eml | python /home/postpunk/mail2entry/mail2entry.py /home/postpunk/mail2entry/settings/

Traceback (innermost last):
File "/home/postpunk/mail2entry/mail2entry.py", line 50, in ?
main()
File "/home/postpunk/mail2entry/mail2entry.py", line 37, in main
lfo.write(time.strftime("%Y-%m-%d %H:%M\n\n"))
TypeError: function requires exactly 2 arguments; 1 given

The error "TypeError: function requires exactly 2 arguments; 1 given" indicates that you're running an older version of Python. Joi's script requires >= 2.2.2, mainly for the e-mail handling functions. I've learned this the hard way. :( Anyone using a good, cheap hosting company running a respectable version of Python?

i'm curious, does your mail2entry script constantly check for mail? check at a certain interval? or does it automatically know when the mail is recieved?

Successful parsing should return a tuple w/ a content dictionary
suitable for posting and filename for a file containing the extracted
image."""

Hi, I have a big problem with mail2entry. I'm new to python and have no idea of the problem happening... Here my mail.log:

Dec 11 02:55:55 tribu qmail: 1071107755.875636 new msg 7506064
Dec 11 02:55:55 tribu qmail: 1071107755.875768 info msg 7506064: bytes 3644 from qp 23522 uid 64011
Dec 11 02:55:55 tribu qmail: 1071107755.896792 starting delivery 966: msg 7506064 to local ga-fl.net-cl@ga-fl.net
Dec 11 02:55:55 tribu qmail: 1071107755.897087 status: local 1/10 remote 0/20
Dec 11 02:55:55 tribu qmail: 1071107755.964642 delivery 966: deferral: /usr/lib/cgi-bin/blog/moblog/mail2entry.py:8:_SyntaxWarning:_import_*_only_allowed_at_module_level/__def_main():/Traceback_(most_recent_call_last):/__File_"/usr/lib/cgi-bin/blog/moblog/mail2entry.py",_line_50,_in_?/____main()/__File_"/usr/lib/cgi-bin/blog/moblog/mail2entry.py",_line_12,_in_main/____settings_path_=_sys.argv[1]/IndexError:_list_index_out_of_range/
11 02:55:55 tribu qmail: 1071107755.964642 delivery 966: deferral: /usr/lib/cgi-bin/blog/moblog/mail2entry.py:8:_SyntaxWarning:_import_*_only_allowed_at_module_level/__def_main():/Traceback_(most_recent_call_last):/__File_"/usr/lib/cgi-bin/blog/moblog/mail2entry.py",_line_50,_in_?/____main()/__File_"/usr/lib/cgi-bin/blog/moblog/mail2entry.py",_line_12,_in_main/____settings_path_=_sys.argv[1]/IndexError:_list_index_out_of_range/
Dec 11 02:55:55 tribu qmail: 1071107755.964762 status: local 0/10 remote 0/20

Have you any idea for "list index out of range"?
Thanks

1. SyntaxWarning:_import_*_only_allowed_at_module_level

Line 17 that says "from settings import *" should be changed to name all the fields that need to be imported.

As such:
"from settings import username, password, blogid, publish, uri, imagesdirpath, imageurldir, template, logfilepath"

2. UnboundLocalError: local variable 'result' referenced before assignment

If an error occurs before line 34, the variable result has not been assigned yet and thus the eror. I inserted a "result = 1" on line 11 which fixes the error.

3. I had to modify parsemsg.py to accept iso-8859 or else it was causing 'BodyNotUTF8' errors:

On line 126 I inserted
elif charset == 'iso-8859-1':
result = unicode(string, "iso-8859-1")

Leave a comment

29 TrackBacks

Listed below are links to blogs that reference this entry: mail2entry script for MT moblog.

TrackBack URL for this entry: http://joi.ito.com/MT-4.35-en/mt-tb.cgi/389

Yes! It's the First Annual Half-Past-Midnight-on-Christmas-Day emptying my bookmarks into my blog entry. Behold the mighty: Kevin Kelly's 'Out of Read More

Joi Ito's Web: mail2entry script for MT moblog. Thank you Joi, Thank you Joi. You've just saved me days of Read More

Joi Ito's Web: mail2entry script for MT moblog. Thank you Joi, Thank you Joi. You've just saved me days of Read More

Joi Ito has released a script for MT that will allow you to blog from email. I think that email is a great way to moblog, it works with most mobile devices. Once it get back home and settled in, i think i'll be setting this up myself. Read More

Hello from Berlin. Via a easyeverything i'm checking up on the blogs which i've missed for the pass two day. Read More

I've been getting excited about various endeavours from Joi Ito and others in the "moblog" arena, that is blogging from mobile devices like cellphones. Joi and others put together a special New Year's Eve moblog event which demonstrates some of Read More

I've got a mailbox called "RSS" that gets all of the announcements, product releases, occasional mailings from sweetcode or ditherati, Read More

Ever since reading about moblogging over at v-2 Organisation , I've been extremly interested. The idea of being able to Read More

Toys from Main Blog
January 4, 2003 3:18 AM

One thing that fascinates me these days is how fast new blogging tools keep popping up. Ben Hammersley had this Read More

Joi Ito's script for can also be used in restrictivemobloggin environments, those that would limit access to postings or that Read More

Moblogging from NeemaNet Daily
January 25, 2003 5:05 AM

I am working to install a script to let me blog from my phone via SMS. There is a python Read More

J'ai r馗emment pris connaissance d'un ph駭om鈩e, le moblogging, ou le fait de blogguer depuis un t駘駱hone mobile si on traduit ce terme litt駻alement, mais qui semble le plus souvent utilis〓pour d駸igner le fait de poster instantan駑ent des ... Read More

mt hackin' from Snapping Links
March 5, 2003 7:27 AM

Looking into more interesting things to do, particularly sending email to my weblog. Read More

Post ot MT via e-mail from Perceptions .. Everybody has them..
April 24, 2003 11:42 PM

How to post to MT via e-mail Read More

Post ot MT via e-mail from Perceptions .. Everybody has them..
April 24, 2003 11:43 PM

How to post to MT via e-mail Hmm... Read More

Want to post to your MT Blog with email. Check this out. Joi Ito's Web: mail2entry script for MT moblog... Read More

Sascha Carlin thought this page could be useful, so he sent it to his linkdump. Read More

メールからのエントリにmoblog mail gatewayを本当に使おうか迷っ... Read More

links for 2005-07-07 from Brianmitchell.com Blog
July 7, 2005 10:18 PM

DKU-2 USB Data Cable For Nokia Cellular Phones - Nokia 9300 (tags: to_read) Audio Bible Player - Read and... Read More

About this Archive

This page is an archive of recent entries in the Business and the Economy category.

Books is the previous category.

Computer and Network Risks is the next category.

Find recent content on the main index.

Monthly Archives