Fri, 07 Dec 2007

Copying while preserving file attributes

cp -p will do the trick - will preserve file attributes, including timestamps, when copying files. So much for low-level copying. I love man pages ... Obviously this is not a solution for FTP, but I can make a copy of my Blosxom blog on the remote server before I start changing files with my script.

Posted on 07 Dec 2007 at 14:34 in /technology. -- Permalink

Relative links in Blosxom (3)

The final version of my little shell script is here ... I had to keep the original files in order to set the timestamps of the corrected files back. This is why the original files couldn't simply be overwritten with the corrected ones. In the end, however, obviously the original files had to be discarded and the corrected files had to be given the same names as the original ones.

#/bin/bash
for i in *.txt;
do sed 's/href="blosxom\.cgi/href="http://\/\/tstigers\.net\/cgi-bin\/blosxom\.cgi/g' $i > $i.new;
touch -r $i $i.new;
done;
rm *.txt;
rename 's/\.new//' *.new

This works fine locally. If anything goes wrong when I run it on my remote server, I'll have wrecked my whole Blosxom blog. But what's the alternative? Blosxom's dependence on file timestamps makes it difficult to create backups. I guess I should have a look at low-level copying.

But maybe it's enough to write a script which copies all files and folders of the Blosxom blog and sets the timestamp of each copied file according to the timestamp of the original file. Such a script wouldn't change anything in the original place - no writing there, only reading.

So the problem of recursively entering directories comes up again. Well, I guess it's still easier to run such a script in every single directory and pass it the correct path to the original files every time.

Posted on 07 Dec 2007 at 14:08 in /technology/internet/blosxom. -- Permalink

Relative links in Blosxom (2)

I found writing a script which changes relative to absolute paths less trivial than I had anticipated.

Editing multiple files in vim ("argdo") is OK - but what happens when your command doesn't apply to all open files (e.g. because you want to substitute something which in some of these files just isn't there)? vim doesn't just skip them but refuses to do anything in any file.

Regarding sed, at first I didn't realise that by default all output is written to standard output, so you have to pipe the output into a file if you want your substitutions to be permanent. Now I've got a simple shell script which loops through all file names in a directory, substitutes relative links with absolute ones and writes the result in a new file:

#/bin/bash
for i in *.txt;
do sed 's/href="blosxom\.cgi/href="http://\/\/tstigers\.net\/cgi-bin\/blosxom\.cgi/g' $i > $i.new;
done

I'm sure that this is not the most elegant solution - but it works. It would be nicer to have a script which does the trick recursively for all directories; but with the handful of directories I have, entering every directory by hand and running the script there is probably easier than writing and testing such a fully automatic script.

Next step: reading the timestamp of each original file and setting the timestamp of each new file to the timestamp of the corresponding original file. It shouldn't be too difficult to do this by incorporating a "touch" command into my script. Finally, I'll need to run this script on my server: downloading all Blosxom text files to my local machine changes the timestamps, of course.

Posted on 07 Dec 2007 at 13:19 in /technology/internet/blosxom. -- Permalink

Thu, 06 Dec 2007

Bad Behavior - no solution - oh yes, a solution!

Acting on instructions I found on the Bad Behavior website, I accessed the databases of my two websites and searched the Bad Behavior table for my (workplace) IP address. On one of the websites, I found one completely innocent entry - in the other website, nothing at all.

Am I looking in the wrong place? "bad_behavior_log table" it says on the Bad Behavior website, and this table should have a "denied_reason" field. I can't see either.

Oh well. Found something after all. I had to look through other users' comments to the instructions I mentioned. Some of them had the same problem as I. One of them, however, mentioned another post on the same website, which I then found. Well, that's convenient ... :-(

Apparently, the creator of Bad Behavior ruined his own plugin by mistake yesterday and, as a result, found himself locked out of his blog, too. Now he's made a new version available for download. However, only his latest post provides a download link to the updated version.

I mean, this guy offers a great WP plugin for free and I'm obviously very grateful for that. However, a link or just a few words on the page where most people will probably start looking for help when they find themselves blocked by Bad Behavior - this would have saved me, and presumably a lot of other people, hours of googling, reading, searching their databases, trying this and that ...

Posted on 06 Dec 2007 at 15:50 in /technology/internet. -- Permalink

FreeBSD on my laptop?

If you want to run FreeBSD on a laptop there is a jungle out there. That doesn't sound too good ... Even though I'm adventurous (well, I think trying new things on my computer is fun), running Ubuntu on my ThinkPad is so much fun, I really don't see myself risking to lose that.

My old Toshiba laptop is no doubt too old for such an experiment. I mean, if even on a ThinkPad X60s there's kernel panic after kernel panic ...

I find even less optimistic information on running OpenSolaris on my box.

So - how about buying another computer? :-/

Posted on 06 Dec 2007 at 14:30 in /technology. -- Permalink

Relative links in Blosxom

I realised I made a stupid mistake - I used relative paths instead of absolute ones for the internal links on this blog. The relative links work fine when you click on them from the main page; they lead to nirvana when you're on a different page, e.g. after clicking on a category path. The reason is that Blosxom creates a pseudo-path for such a page, appending stuff to the name of the Blosxom script file (blosxom.cgi).

Fixing this will be quite a hassle: editing the relevant files (using sed or just vim) is easy enough but resetting the timestamp individually for each file will be cumbersome. Can I do this automatically with a script?

Posted on 06 Dec 2007 at 13:02 in /technology/internet/blosxom. -- Permalink

Bad Behavior problem

When I tried to log onto my "entries" blog this morning, I found myself locked out. I got a 403 message and was told that my IP address was "associated with malicious behaviour", or something like that. The "fix this problem yourself" link didn't get me anywhere; and of course I could have contacted the site administrator (= myself) by e-mail, quoting the "technical support code" I found on the error page - but I couldn't log onto my administrator account either. So what the heck?

The problem was caused by the Bad Behavior plugin for WordPress, and deleting the relevant directory over FTP solved the problem. Apparently, Bad Behavior had gathered so much incriminating information on me in its log file that it decided to block me.

Interestingly, I was also blocked from accessing the Amity website. It ran Bad Behavior, too - until I deleted it this morning, too. How come that the plugin on both websites came to the conclusion that my IP address was part of the axis of evil?

Maybe Bad Behavior was correct after all? Maybe terrible stuff is being sent out from my IP address all the time because my computer was infected by a Trojan or something? I'm kind of worried, even though it seems very unlikely that anything is wrong with my computer. Maybe I should have had a close look at the log files instead of deleting the plugin directories on both websites right away.

Posted on 06 Dec 2007 at 11:22 in /technology/internet. -- Permalink

Wed, 05 Dec 2007

OpenSolaris

What is Solaris anyway? And haven't I long wanted to find out about BSD, too? OpenSolaris.org - FreeBSD.org.

Posted on 05 Dec 2007 at 17:46 in /technology. -- Permalink

Dictators of Wikiland

Wikipedia is devoted to democratic consensus and the free exchange of ideas - right? Wrong! Apparently, it is run by an evil junta, whose members use a secret e-mail list to destroy anyone questioning their dictatorial rule. (Kerim twittered about this.)

Posted on 05 Dec 2007 at 14:24 in /technology/internet. -- Permalink

Gnome Blogs

I've been looking at a few of the blogs listed on the Gnome Blogs site. Most of the 98 blogs do seem rather dead, but there's some pretty good stuff, too.

Posted on 05 Dec 2007 at 14:13 in /technology/internet. -- Permalink

Lynx under Windoze

I have these periods when I love using text-based browsers. Lynx works under Windoze, sort of. Chinese characters are a big problem. - I prefer Links, but this I'd need to compile myself: fine in Linux, of course; way too much hassle in Windows.

Posted on 05 Dec 2007 at 13:55 in /technology/internet. -- Permalink

Tue, 04 Dec 2007

Will SpamFree make our website spam-free?

In addition to Akismet and Bad Behavior, I also installed WP-SpamFree on the Amity website. I'm curious to see if this plugin will really "virtually eliminate automated comment spam" as promised.

Posted on 04 Dec 2007 at 14:27 in /technology/internet. -- Permalink

ZoneAlarm woes

I spent one and a half hours this morning trying to install and run a "ZoneAlarm Security Suite" update. The problem is, it simply doesn't run - the only thing it did is make my computer hang. The "Windows Task Manager" displayed CPU load constantly at 100%. Finally I uninstalled it and went back to the previous version. Also, I disabled the anti-virus function - I'm using Avira AntiVir now, a free and (in my experience) reliable program.

I had to reboot 4 times in the process. Under Linux, normally you needn't even shut down any program while updates are being installed.

I wonder why I fell into the trap again ... It's not the first time that ZoneAlarm publishes an update which wreaks havoc on my computer. Well, thank you ZoneLabs, and thank you Windows in the first place.

Posted on 04 Dec 2007 at 11:18 in /technology. -- Permalink

Mon, 03 Dec 2007

Granule vocabulary flashcard program

That's a fine little piece of software! It makes it so much more convenient for me to memorise and revise my Chinese words, and it's so much better to have everything in one place, neat, clean, searchable, than to use paper flashcards (which I'm still using in addition to Granule).

(My earlier blog entry has links to the program and a related Wikipedia entry.)

Posted on 03 Dec 2007 at 13:09 in /technology. -- Permalink

Sun, 02 Dec 2007

Video editing under Linux: Kino

Kino, a less complex, less powerful video editor, works fine. Loading/inserting files which you want to use is a rather painful process though - that's definitely better in Cinelerra, where you simply select all the files you want, then wait until they've all been loaded. In Kino, you have to go through the "load file" dialogue individually for every single snippet.

Kino wasn't able to export ("render") anything as long as I didn't run it with root privileges. Apparently it needs a certain kernel module, which it can't load without these privileges. I wonder if this was the cause of my problems with Cinelerra. The beauty of Kino is that it actually lets you know if there's any problem, and what the problem is. With Cinelerra, I just couldn't figure it out. I don't think I'll install and try Cinelerra again - glad I finally found something which works OK; and Kino probably has all the functions I'll ever want to use.

Posted on 02 Dec 2007 at 23:15 in /technology. -- Permalink

Installing Cinelerra (3)

OK - no difference to the status quo ante. No rendering.

Actually yes, there is a difference. Or did I just fail to notice this before? When I load files which I want to edit, I get the most horrible "audio block/sample failure" messages. A hopeless case, it seems.

Posted on 02 Dec 2007 at 22:34 in /technology. -- Permalink

Installing Cinelerra (2)

OK, now CVS doesn't cause any problems anymore, but I get several "possibly undefined macro" errors as a result of creating a "configure" file (autoreconf -i --force). I'm reinstalling Cinelerra from the Ubuntu repository now, just to see what will happen.

Posted on 02 Dec 2007 at 22:16 in /technology. -- Permalink

Installing Cinelerra (1)

As expected, svn worked like a breeze. Creating the "configure" file didn't work, however, because I didn't have CVS installed. Well, that can be helped.

Posted on 02 Dec 2007 at 22:11 in /technology. -- Permalink

Going for the newest version of Cinelerra

I prefer installing software from the Ubuntu repositories but in this case, it just might help to compile the latest Cinelerra version from the subversion repository. First of all, I had to get tons of libraries and "dev" headers. Using subversion will be a pleasure, but I'm somewhat curious to see what problems will come up during the compilation process.

Posted on 02 Dec 2007 at 22:05 in /technology. -- Permalink

Video editing: pleasures and pains of Cinelerra

It took me some time to find my way through Cinelerra but now it feels comfortable to select, cut and put together snippets of video. The software comes with lots of other functions, most of which are probably relatively easy to use, too. I haven't yet tried many.

The problem is, I just can't render video, even though the manual describes in some detail how to render video for a DVD. What I don't understand is that I don't even get any error messages (when I run Cinelerra from a terminal); it doesn't even crash, it just silently refuses to do anything. Audio rendering works fine. I guess the video rendering process needs something which I haven't installed - but as it is, I'm left without a clue as to what this could be.

Since video editing is pointless if you can't get an end product (to burn onto a DVD, upload to the web or just watch on a computer), I'm stuck here.

Posted on 02 Dec 2007 at 18:06 in /technology. -- Permalink

Video editing under Linux

No, I'm not really trying to "Become a digital video editing guru using Linux tools" (a helpful article!) but, after I've learned how to grab digital video from tape and save it on my hard disk (using dvgrab), I'm now moving on to the actual editing.

sudo dvgrab --format raw -buffers 200 --autosplit --size 100 source_

Posted on 02 Dec 2007 at 11:00 in /technology. -- Permalink

Sat, 01 Dec 2007

IP addresses

Some basic information - useful for understanding what IP addresses are and how all this stuff works. Also, the relevant Wikipedia article.

Posted on 01 Dec 2007 at 14:55 in /technology/internet. -- Permalink

Fri, 30 Nov 2007

Plone for Belgium

Belgian local governments cooperate to develop Plone-based applications which manage documents, schedule appointments etc.: Networks effects: Plone for Belgium and beyond (a study by IDABC - European e-government services). I guess if it's good enough for Belgium, it'll work for us too ...

Posted on 30 Nov 2007 at 15:48 in /technology/internet. -- Permalink

Thu, 29 Nov 2007

Gmail POP works again

Apparently they fixed the problem - whatever it was. I hope it'll stay this way for some time. In the meantime, I changed my e-mail address for quite a number of newsletters, web accounts etc. to the e-mail account on my own server.

Posted on 29 Nov 2007 at 20:01 in /technology/e-mail. -- Permalink

Password problem (3): Gmail

After configuring Thunderbird for Gmail IMAP, I receive a message saying that my "credentials" weren't correct. I guess that's IMAP slang for "authentication error" - the same problem. Gmail's help pages and Google Groups help group don't provide any clues either.

Best to get out of Gmail altogether, I guess.

Posted on 29 Nov 2007 at 10:35 in /technology/e-mail. -- Permalink

Password problem (2): Gmail and Thunderbird

Thunderbird, too, gets an authentication error message from the Gmail server when it tries to download e-mails. I still don't understand why this problem first appeared after I changed my Gmail password but it't pretty obvious now that something's wrong on Gmail's side.

Apparently the Gmail guys are having trouble enabling IMAP support for all users ("If the option to enable IMAP isn't available in your Google Mail account's 'Forwarding and POP' settings, you can enable IMAP for your account by simply enabling POP access. We're currently working hard to make enabling IMAP easier for all our users."). So maybe my problem is somehow related to that.

Posted on 29 Nov 2007 at 10:18 in /technology/e-mail. -- Permalink

Password problem: Gmail and Evolution

I changed my Gmail password, now Evolution Mail can't log on and download my e-mails anymore. That's extremely annoying and I've already spent more than an hour trying to solve this problem. I learned that in 2005 apparently there was a bug in Evolution which prevented the program from changing its saved passwords; I guess this has been fixed since. In any event, this is obviously not the cause of the problem I've encountered. I even changed my password back to what it was before; the error persists.

Is it a Gmail problem? Does Gmail's POP server return an authentication error although Evolution sends the correct username and password? Message forwarding to a different e-mail account isn't working either. But why would this problem start after I changed the password? Strangely, I can access my Gmail account on the web and Google Talk works fine, too.

Posted on 29 Nov 2007 at 00:00 in /technology/e-mail. -- Permalink

Wed, 28 Nov 2007

Guide to intrusion detection etc.

A "compilation of resources that explain what intrusion detection and prevention are, how they work, troubleshooting, configurations and more", by SearchSecurity.com and SearchNetworking.com: Intrusion Detection and Prevention Learning Guide.

Also, 2 blogs which look promising: Conspicuous Chatter and TaoSecurity.

Posted on 28 Nov 2007 at 19:28 in /technology/internet. -- Permalink

One-time-pad encryption

How to unbreakably encrypt a message using a "one time pad" and the Vernam algorithm - it's surprisingly simple, but there's one big drawback: since this isn't public-key encryption, how do you get the one time pad to the person who is supposed to decrypt your message? You have to make sure that nobody except this one person gets hold of the pad. Moreover, you have to use a different pad for every single message.

Posted on 28 Nov 2007 at 17:27 in /technology. -- Permalink

File upload - success

I got around the problem (whatever it may have been) by going back to the original files and folders and creating 2 separate .zip files from them instead of 1. The drawback of this solution is that it isn't foolproof: the person who is supposed to download and unzip the stuff may easily put them in 2 different places, which will wreck the whole thing. Foolproof or internet-proof, which do you prefer?

Posted on 28 Nov 2007 at 16:35 in /technology/internet. -- Permalink

Failed file upload (2)

Hmm ... Now, after 98%, I get a "Broken pipe" and "Remote site local filesystem disconnected" message. And I can't get over this point. What the heck is wrong here?

Posted on 28 Nov 2007 at 15:22 in /technology/internet. -- Permalink

Failed file upload

We tried to upload a file of about 110 MB to the server - and, at the office, got nowhere. The connection broke so often that we just had to give up after reconnecting to the server for several dozen times. In the end, the connection broke after just a few seconds and this didn't get any better. Was it a problem with FileZilla? With Windoze? A (probably temporary) problem with the internet connection at the office? At home now, I'm trying again - and it goes through without a hitch. 93% of the file are on the server already.

Posted on 28 Nov 2007 at 15:18 in /technology/internet. -- Permalink

Salt (2)

Two lovely introductions to the concept of salt etc.: one at hackerthreads.org, the other at ASPHeute.

Posted on 28 Nov 2007 at 13:53 in /technology/internet. -- Permalink

Tue, 27 Nov 2007

Salt

Salt is very closely related to the concept of nonce. I like Wikipedia. I also like the term "dictonary attack". I enjoyed reading an article about using Google for such an attack. Light Blue Touchpaper: another blog found on the way, and definitely one I'll go on reading. Hat tip to Schneier.

Oh, and the radio recording was OK.

Posted on 27 Nov 2007 at 17:31 in /technology/internet. -- Permalink

FTP extension for Firefox

FireFTP works really well - and it's nice to be able to upload files to your server and see the results right away without ever leaving Firefox.

Posted on 27 Nov 2007 at 14:44 in /technology/internet. -- Permalink

Mon, 26 Nov 2007

Creating RSS from HTML (2)

Been there, done the learning, got the T-shirt. I wrote a PHP script, part of which I adapted from something I found (so I didn't actually do all the learning ...), which reliably produces the XML data I need for an RSS feed. These data are the output of the script when it is called; additionally, the script writes this output to a separate file named "index.rss". Strangely, Firefox syntax highlighting (in source view) doesn't work for this latter file - although it does for the former, which is named "index.php". Firefox also displays both files correctly, the result looks just like what you get when you open any out-of-the-box feed in the browser.

Google Reader, however, doesn't like my 2 files - it recognises neither the feed title nor any items. What I get is a feed named "(title: unknown)" without items. The markup looks exactly like the markup of another feed I have which works perfectly fine with Google Reader.

Might there be anything the matter with file headers which I don't understand? I just don't get it at all!

Posted on 26 Nov 2007 at 20:30 in /technology/internet. -- Permalink

Creating an RSS feed from an HTML file

I was rather surprised to find that PHP does not offer ready-made functions for parsing HTML, something similar to simplexml etc. for XML. OK, it's not so difficult to read an HTML file and hack it to pieces with regular expressions. But still ...

First of all, in order to get something to display for all 6 boxes on my new feeds page, I wrote an RSS file by hand for my photo galleries. This does the trick - but of course it would be so much nicer if I didn't need to update more than one file when I add another photo gallery. A PHP script should take this one file and produce a corresponding XML file whenever it is called.

Obviously, the perfect solution would be a PHP script which looks at the photo galleries directory and produces both an RSS feed and an "index.html" file on the fly. PHP has nice file functions! Yet where would the title of each feed item come from? Anyhow, for now I still have plenty to learn from trying to extract the relevant information from an HTML file and embed it in an XML file.

Posted on 26 Nov 2007 at 16:29 in /technology/internet. -- Permalink

Sun, 25 Nov 2007

Creating a feeds page with PHP

Fed up with dysfunctional downloadable scripts, I sat down and wrote some code to fetch and display my feeds. No need to look at the K2 sidebar widget.

Turns out that PHP has all you need - and what you need is no more than one specialised function: simplexml_load_file(). Any object created with this function has properties which are the XML nodes, e.g. ->channel, ->channel->item->title or ->item->link (depending, of course, on the structure of the XML file in question). Throw in a few control structures (if, foreach) and string functions (strlen, substr) and you're done. Tweaking the stylesheet so the output of my script is displayed nicely took me more time than writing and testing the script. I found the relevant chapter of the PHP manual at PHPBuilder very useful.

Since del.icio.us publishes an RDF feed, not a "normal" RSS one, I had to come up with 2 different versions of my "fetchFeed" function. (In an RDF file, an "item" is a child of the document - in an RSS file, it's a child of a "channel", which in turn is a child of the document.) Adapting the function to Atom or whatever is probably easy, too.

Posted on 25 Nov 2007 at 15:59 in /technology/internet. -- Permalink

Sat, 24 Nov 2007

Tweaked the "blox" plugin ...

... so now it creates <br /> tags instead of <br>. I'm using the following as a test:

a line
another line
a third line

Posted on 24 Nov 2007 at 13:10 in /technology/internet/blosxom. -- Permalink

Changing the date (etc.) of a file

Well, that was easy ... "touch -t 200711131100 filename" will change the date of this file to 2007 (year), 11 (month), 13 (day), 11 (hours), 00 (minutes). So, after correcting it, the relevant post goes back to where it came from. Nice to have shell access!

Posted on 24 Nov 2007 at 12:38 in /technology/internet. -- Permalink