22.04.2009 20:49

Personal e-mail solution

I wrote about my home LAN earlier and described my personal backup solution and now I'd like to talk about my mail system in the same environment. Thinking about this article I realized that they are in a way poor mans solutions, and I have plenty of those around to write about.

My LAN gateway is running a MTA and all workstations use it to send mail. My personal favorite MUA is Pine, now known as Alpine to which I'll come back later. So I run a very small mail system, with only a few users, yet with so much spam going around the system still processes thousands of e-mails every day. Which brings me to the SPAM filter. Probably best known and most used filter is SpamAssassin which is a huge monster that eats children, in my initial testing it would bring the system to its knees by processing only a handful of e-mails. There are many alternatives, like Dspam but let's not waste time on those. With some luck I managed to find a true gem, the CRM114 Discriminator which is an extremely small and lightweight filter. With some patient and smart training you can achieve good results and high accuracy.

I skipped a step didn't I, the MTA first forwards the message to the MDA, in my case that's procmail. It will analyze the message, forward it to the SPAM filter and then deliver to the respective mailbox. If you are interested in my own recipes one of my procmailrc files is available on-line.

Most people use IMAP these days, and maildirs, but I still stick to the mbox format. Besides, Alpine supports both mbx and mix variants so I have no urge to switch. Additionally mbox plays an important role in my setup. On most of my accounts I just setup forwarding to one central address. For a while that address was on a remote server which was also an extremely small mail system. When I needed to sync all those messages to my local mailbox I didn't really want to setup a POP daemon (although popa3d is really small and efficient, and would be a good fit) nor switch to IMAP. In the end I used rsync over SSH, a cronjob would need to sync only a few (mbox) files, transferring only the changes while using compression and with SSH providing encryption. So simple yet so effective, I was saving bandwidth while avoiding a pop3 daemon (with mandatory SSL, in case of popa3d stunnel would need to be used).

Enough about receiving e-mail, let's talk about sending. Alpine unlike some other MUAs doesn't require a local MTA or an SMTP client (by the way it also doesn't require a MRA like fetchmail). So there is not much to say about Alpine, you define your SMTP servers and you send your e-mail. But not having a local MTA can be a problem if you want to receive messages from cron, or so it might seem. All distributions ship with mailx and an easy solution is to setup your "~/.mailrc" like this:

set smtp=gateway.local.domain
set from=user@hostname.local.domain
You can then pipe your cronjobs to mail/mailx and send them where ever you want. My personal favorite mailx variant is mailx heirloom and some distributions like Slackware use it by default, others like Arch unfortunately use different implementations (by default).

An addition to this article will be about integrating Alpine with the desktop environment. It doesn't really belong here, but it also deserves some attention.


Written by anrxc | Permalink | Filed under main

20.04.2009 20:55

Notes on audio players

Amarok was once my favorite application. Best audio player ever, on any platform. First signs of an uncertain future showed a few years ago during a developer meeting in Holland, following that congress Amarok had a major change of its user interface. It was hideous and it was a disaster, the common question was: "what the hell were they smoking over there?". The mistake was acknowledged and the old interface was restored quickly. But following in the footsteps of KDE 4 (which as we all know was a major disaster) the developers decided to release Amarok 2, a complete rewrite of the application.

Which is today at release 2.0.2 and it's still completely broken, probably will be until around 2.2 comes out... such a long time to wait. Remember they didn't have some average product to build on, they already had the best application out there. Bugs are numerous, and I don't even care about the big ones such as broken database import function but the little ones drive me crazy. For instance working with podcasts for more than a few minutes is bound to freeze them, it's impossible to expand/retract or do anything else with them until you restart the whole application. Oh and did I mention the bug where all your podcasts (which had to be manually imported remember) would disappear on shutdown. I noticed a lot more problems, but that's enough - it's unusable, as simple as that. One thing that fascinates me though - Amarok v1 compiled and packaged on my system takes up 4MB while Amarok v2 is close to 14MB. What the hell is in there, for it sure isn't basic functionality (EQ anyone?).

Six months ago I started searching for a replacement. All those GTK Amarok clones (Exaile, Banshee, Rhythmbox...) were not even close to Amarok 1 and I discarded them. What do I need from my audio player; tagging support, streaming support (Shoutcast integration is a plus) and podcast support.

During my search I tried many players, most of them are regarded as simple audio players, like Audacious and Sonata. But none had everything I needed, most important thing being podcast support. I thought about that one a lot, could we still call them simple if they did have podcast support? It's such a common thing today, so I believe they should all have it. In the end I managed to find a true gem, and even though it doesn't have podcast support (yet) it deserves your attention. This player is called Goggles Music Manager and it's a fantastic application. Its interface is similar to that of Foobar2000 and it has some good functionality. Most notable thing is that it uses the FOX toolkit so it's extremely lightweight and fast.

I actually almost never used Amarok to just play/listen to music. When I needed to sort my music collection, clean up tags, listen to radio/podcasts... I would invoke it. But when I just want to play some music I always use cplay which is a command line audio player. To be more exact it's a frontend to many audio players and it's written in Python. Development of cplay is inactive for some time now, and even its home page disappeared a few months back. Last released version had a few smaller problems and I also missed some functionality so I patched it a lot and I plan to release those patches or even the package as a whole... but that is a story for a future article. In the meantime you can at least check the simple color patch.


Written by anrxc | Permalink | Filed under code, media

19.04.2009 22:04

Recording sound from ALSA with JACK

Few months ago I wanted to record some music from a flash widget. It was supposed to be simple as per ALSA wiki. You just need to copy the PCM channel:

# ~/.asoundrc
#
# Recording sound going trough the card, capture with: 
#   $ arecord -t wav -D copy test.wav
pcm.copy {
    type plug
    slave {
        pcm hw
    }
    route_policy copy
}
Well guess what it didn't work, no surprise there as Linux sound state is a complete mess. You can also try setting one of your channels as a capture source and then record from it... boring and unreliable, but let's mention it:
Try with this first:
$ amixer set 'Mix' cap

Or if you don't have a 'Mix' channel:
$ amixer set 'Capture' cap
$ amixer set 'Capture' 10%

Record with:
$ arecord -t wav -f cd test.wav
After I while I did get some results but they were terrible. After a few more web searches I saw that just about everyone recommends using Audacity for this. Well it turns out that Audacity tries to do the same thing, hence does it terribly. Interesting enough it was also one of the popular solutions for doing this on Windows too, I even tried it in moments of desperation and results were terrible even there. But all was not lost as there are numerous apps for Windows that can do it. I tried a few and to my surprise they were all more or less crap, those that did record something recorded it badly, sound was full of static, too low or too high etc.

But I didn't give up and ultimately I found a perfect solution. Resulting recordings were perfect - an exact copy of the original source. I did it with a little help from JACK and jack_capture. I won't cover the installation and setup of jackd but there is a nice graphical frontend called qjackctl that will get you started in no time. Many applications support JACK, and have audio output drivers for it (i.e. Mplayer). Once jackd is running and your application is using it you can start jack_capture and play the audio.

But what about applications that don't support JACK? To make it worse I needed to record sound from a flash widget and Adobe's flash plugin doesn't have JACK support. Here comes the tricky part, you have to route the sound from ALSA to JACK and then capture it using jack_capture. To do this you will need the ALSA plugins package which has a jack plugin. Once installed you can setup routing in your "~/.asoundrc":
# ~/.asoundrc
#
# Connect ALSA only applications to JACKD
#  - alsa-plugins package needs to be installed
pcm.!default {
        type plug
        slave { pcm "jack" }
}
pcm.jack {
        type jack
        playback_ports {
               0 alsa_pcm:playback_1
               1 alsa_pcm:playback_2
        }
       	capture_ports {
               0 alsa_pcm:capture_1
               1 alsa_pcm:capture_2
        }
}
When you visit the web page; flash plugin will send the audio to ALSA which will route it to the jack daemon to which you connected jack_capture which is by this point recording sound - a perfect and exact copy of the source (in "wav" format) which you can later convert to a high quality "ogg" with:
$ oggenc -q 10 file.wav -o file.ogg


Written by anrxc | Permalink | Filed under media

16.04.2009 23:08

Vimperator for Emacs users 2.0

Its been a month since the initial release of my Vimperator setup. Some people contacted me just to say thanks, and I appreciate that, since my first announcement was actually on Arch BBS and there were no takers! Not even one... it seems I stumbled into the lair of vi users. In the meantime Vimperator developers were hard at work and they released Vimperator 2.0. I must say it has some great new features but unfortunately the completion is completely broken for me. Most of the time it will completely freeze both Firefox and my system. I'm sticking with v1.2 for now but my setup works just fine on v2.0.

I did notice some weird behavior with "C-g" as "Esc" in some cases, but didn't investigate thoroughly. I also filed a feature request to enable bindings in "INSERT (menu)" mode so we can have "C-n" and "C-p" bindings in drop down menus as next/prev. It's the only thing I miss, to make it worse the "C-n" binding is Firefox default for "New Window" so I constantly spawn them when absorbed in my work. Request was accepted so stay tuned for that one.

Most noticeable change is the introduction of color-schemes, and my snippet example for userChrome.css will not work as before. I started modifying the default theme to match my previous look but never quite finished it as I reverted to v1.2 soon. But it's a good start to roll your own theme. Themes should be placed in "~/.vimperator/colors/" as "theme-name.vimp" and can be loaded with ":colorscheme theme-name" both from the command line or from .vimperatorrc.


Written by anrxc | Permalink | Filed under desktop, code, emacs

15.04.2009 20:02

Laptop-mode-tools

These days my main workstation is a laptop so battery power is an important issue. There are many tweaks one can use on GNU/Linux to conserve it and most of them were joined in the laptop-mode-tools project ("lmt" later in the text). Basically it's just a shell script (that gets run on AC events) with a bunch of configuration files controlling its behavior. Let's talk results first, from initial 1:30h of battery power I went to 2:00 - 2:30h, good results considering that I have very modest hardware - a CPU that can't be scaled nor undervolted and a lousy battery from SANYO. My power consumption went from initial 21W to 16W when running on battery. Some people with better hardware and some smart tweaks can go as low as 9-10W extending their battery life much longer.

As I said lmt covers most of the tweaks, not all, and at first I had to patch it extensively to add support for my hardware and additional tweaks. But it gets better and better in every version as it adds more tweaks. These days I only need the patch to add support for rfkill on my Acer so I can completely shut down bluetooth.

One very important function is "auto hibernate on low battery", I often get absorbed in my work and loose track of remaining battery power, lost a few important documents that way. As I mentioned, I have a lousy battery which also lacks alarm support. At first lmt relied on ACPI events so I used this script. The alternative was to invoke lmt every few minutes from a cronjob, not so elegant when you are trying to minimize disk reads/writes. Recent version of lmt do support battery polling trough a module-helper which then again calls lmt every few minutes, so I have to patch that too.

Another interesting functionality is the "start-stop-programs" module which allows you to start/stop any service or process on AC events. I wrote about awesome window manager earlier and its widgets. Depending on what they do and how often and aggressive they do it, they could cause quite a few wake-ups. So I combined start-stop-programs module with wicked built-in suspend/resume functionality. The result is this script which is run on AC changes and automatically suspends/resumes widgets. Although awesome 3 and wicked are much better than awesome 2 and amazing were - powertop used to report them as the top cause of wake-ups while now they barely make the list of top 20.


Written by anrxc | Permalink | Filed under desktop, code

14.04.2009 15:25

GPRS on GNU/Linux

I still use GPRS often as WiFi coverage is not always good in the suburbs around my city. I have a basic SonyEricsson mobile phone that doubles as a GPRS modem for my laptop. I wrote about the quality service T-Com provides and GPRS is no exception; I pay 3$ per 10MB of traffic. Using Opera Mini on your phone maybe you could stretch that trough the month but otherwise it's the amount of traffic I generate in an hour of browsing and reading e-mail... so what could I do to save bandwidth?

First let's cover the basics. Once you connect your phone with an USB cable besides standard USB core modules you will also need cdc-acm and ppp-generic modules. For a PPP dialer you can use wvdial or KPPP (provided by KDE). I use wvdial which is set up like this:

; wvdial configuration
;   /etc/wvdial.conf
;
; T-Com Web'n'Walk
[Dialer wnw]
baud = 115200
Modem = /dev/ttyACM0
Dial Command = ATD
Carrier Check = no
Init = at+cgdcont=1,"IP","web.htgprs"
Phone = *99***1#
Username = none
Password = none
You can initiate a connection with:
# wvdial wnw
Once connected I could barely open a few pages (not to mention that the general connection speed is pathetic) so I started thinking about saving bandwidth. The first thing I setup was SSH as a SOCKS proxy, with compression enabled. Further I considered adding a HTTP caching proxy to my home router such as Privoxy which has great filtering capabilities, so I could strip not only ads but maybe even all images and further modify and twist the traffic... I can't be the first person thinking about these issues yet all web searches came up empty. Or, they did until I found the Toonel service this winter. Their client routes all your traffic trough their servers while compressing it, and additionally they run a caching proxy. Their client is cross-platform and it works for a number of protocols, seems to be exactly the solution I was thinking of...

But I passed because I imagine they don't encrypt the traffic and SSH as a SOCKS proxy proved to be a good enough solution. Once wvdial is connected I initiate an ssh connection to some server:
$ ssh -C -D 40000 somehost.tld
I then have a SOCKS proxy running on localhost port 40000 which many applications directly support (Firefox, Gajim...) and tsocks can be used for apps that don't. Argument "-C" enables compression and it saves me some bandwidth in the long run. I know many people tweaked network settings of Firefox (or use Fasterfox) to gain speed, but you might consider a more conservative set of rules when using GPRS because it could backfire on you.


Written by anrxc | Permalink | Filed under main

09.04.2009 20:36

Electronic book readers

I am very excited about electronic book readers such as the Amazon Kindle and Sony PRS devices. Since Kindle was released I've read numerous discussions on whether they can replace real paper and real books. The common conclusion is always: no. This technology is for now available mostly to Americans and they are the majority partaking in these discussions, which always revolve around that issue.

But they are all missing a very important point and probably wouldn't understand it because any book imaginable is available in their bookstores and they can even buy paperback editions of any book for around 5$. Can Kindle replace paper - no, but what if the paper is not even available to you? The publishers are not interested in good books, only in good money and so, here in Croatia, they mostly publish best selling romance and crime novels when it comes to foreign authors. Let's take a popular writer like William Gibson as an example, only Neuromancer and Virtual Light were translated and published here. Both are only a part of a trilogy - well isn't that a fuck-up but additionally I had to pay over 50$ for my copy of Neuromancer (which turned out to be a lousy translation mind you).

That's why electronic book readers are my salvation and if they really won't be able to replace paper I don't care because I know it in my hearth and bones that they are certainly the future for me. If we can ignore the fact that most (even remotely) popular books are already available for free on-line, as pirate copies, that still leaves us with most e-books that are sold for no more than 5-10$ by legal resellers.

In the future I will be able to read any book I desire. It will cost a few dollars and it will instantly be placed on a device that is as big as just one book but holds thousands previously inaccessible titles. Many people already read e-books from their phones, PDA's and laptops and they've been doing it for years. But for me they all have too much disadvantages and in the case of laptops there is just no way I can feel comfortable with it while laying in bed, it's just too heavy and impractical.

Current technology and software used in these devices still have too much imperfections and their price is just too high taking that into account. It's the reason I didn't buy one yet, the price doesn't bother me as much because I've spent so much money on books that an e-book reader would in short term be a big expense but in the long run would save me some serious money. I've been tracking communities like MobileRead and there are still much drawbacks and problems (i.e. with formatting of PDF's and other formats) for me to buy one. But I'm sure it will all be solved soon, I don't want to cash out for a BeBook and struggle with formatting while three months later something called "CeBook" puts all current models to shame.


Written by anrxc | Permalink | Filed under books, media

06.04.2009 23:57

Elegance of VDR

In early 2000s ADSL was an unreachable dream here, a myth. I guess ze Germans were just starting with SDSL/whatever and their old equipment was not yet available for transfer to Croatia. I had to wait 19PM for lower dial-up rates (also expensive mind you), and then spend the whole night on-line (thank you for the lovely sleep disorder). Anyway, after reviewing my options only solution was SatDSL. I signed up with Netsystem, and as a part of their service I received a PCI DVB-S card, it was a SkyStar2 card from TechniSat. At the time I was completely ignorant to all other benefits this beautiful piece of hardware brings... well, not for long anyway.

Today I still have the same card in a dedicated PC that is running VDR for watching SAT TV. In the age of eye candy and flashy things like "Boxee", "XBMC" and "MythTV" it manages to stay remarkably simple (in design, not necessarily in implementation) yet very powerful, it perfectly fits the KISS philosophy of Slackware or Arch that I learned to appreciate so much. Combined with Oxine you get a full blown media center without any complexities that solutions like MythTV imply (SQL being one). But thanks to some smart design choices VDR is infinitely expandable via plugins, so it can still offer any functionality other systems have. For example, using plugins it's possible to watch analog TV as well as IPTV (and of course DVB-C/T are natively supported) all from one interface. Speaking of DVB it also has the upper hand over MythTV as the latter was started with analog TV in mind while VDR was built around DVB from ground up. People in my country recognize VDR as a premium STB solution, yet very few know their way around GNU/Linux and VDR became kind of a myth that everyone recommends yet no one uses (sticking to lesser Windows alternatives). That bothered me as VDR is the simplest solution out there, so a few years ago I wrote a wiki article guiding people trough each step of the process. I didn't count all the thank-you notes I received since then but they are close to a hundred now.

With this introduction concluded, in the future I can write about some more interesting bits and pieces of the setup, like software CAM emulation, streaming and so on...


Written by anrxc | Permalink | Filed under dvb, media

01.04.2009 23:32

Employing org-mode

Since I started using org-mode I employed its simple format on many different documents, not just limiting it to manage my agenda. For example it's a great format for documenting systems I maintain; all system information, changes, security updates etc. are stored in per-system org files. I also use it for all projects I work on. Once terms are set I will open a new project.org file and start outlining tasks. Later all kinds of sensitive information will be added, and this brings up the issue of security, I wrote about encrypting file-systems but additionally I also encrypt every sensitive file with GPG. This is where EasyPG for Emacs fits in, it allows transparent en/decryption of files while ensuring nothing gets leaked outside of Emacs.

It's not as simple as some (custom) systems I've seen but it's a complete solution. Freelancing from home I had to come up with something to make my life easier and my hands weren't tied by some office rule or regulation. The same setup can be used to keep your passwords, logins, bank account and other personal information. Org's table mode is a good format for those files.

Along with project.org I also have report.org files for every project, holding various information, documenting everything I did and so on. When I'm done I export the report as a PDF and send it to clients, feedback was always very positive. It's simple but it makes a difference. I'll mention just one more use case. This winter I was at a hackmeeting and someone asked me to talk about video streaming. It took me about an hour to write an org file on the topic and I used it directly with Emacs to do a presentation. I did it by strategically expanding and folding relevant sections as I went over the topics, and it turned out really well. After the event I exported it to a few different formats and shared them with participants, once again I had a complete solution...


Written by anrxc | Permalink | Filed under crypto, work, emacs

29.03.2009 23:01

Publishing with NB and Emacs

Picking an engine to publish my writings I wanted something extremely simple. First thing that implies is no SQL. I wanted an engine written in Python and pyblosxom was interesting but in the end I went with NanoBlogger. NB is a blog engine written in Bash that uses standard UNIX utils like sed and grep to create static HTML content. How to write and publish the content is left as an exercise to the user.

NB can automatically generate templates on new entries or articles, but it's not perfect when it comes to GNU Emacs. I wanted at least the correct mode to be started. So I wrote nbpost.sh, it creates a new entry, inserts a template and starts the correct mode. It also recognizes when I'm editing an existing entry, and can start Emacs if it's not already running. As for the mode, html-mode is OK, but I found post-mode to fit perfectly. It was written for mutt and editing e-mails but it has other uses too.

Once written and HTML generated, an article needs to be published to the web server. Doing this manually would be tedious so I wrote nbsync.py. It uses rsync for publishing content over SSH. Rsync will transfer only changes between files, it uses compression and ssh provides encryption... very simple yet very effective.

With that in place I still wanted better integration of NB in my environment. I use ZSH which has great completion and a web search brought me to completion functions written in 2006, a little outdated but very good. I updated them and published _nb here. It completes all the commands as well as categories and entries.

I now have a complete solution for publishing from Emacs. Which also brings its related benefits; syntax highlighting, spell checking, well known key bindings and automatic publishing of content. Not to mention that org-mode found it's use too. I keep a journal.org file outlining all my entries and categories, related TODO tasks and ideas for future articles.


Written by anrxc | Permalink | Filed under main, code, emacs