Java thread pooling and connection pooling: How to handle proper connection cleanup on thread exit.

I wanted to have a fixed pool of threads for executing tasks that needed MySQL database access. Also, I couldn't afford to create and destroy a connection for each task; I needed to have up to one persistent connection per thread, so that the many tasks that will successively execute on that thread, will re-use the same connection.
Another problem that I had was how to clean up the connection if the ThreadExecutor decides to close my thread. It's all solved in the following code.

The following codes runs OK for me in Java 1.6 SE. Please do let me know if you find flaws with it.


import java.sql.Connection;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.atomic.AtomicInteger;

public class MyManager {
    private ExecutorService pool;

    public static void main(String[] args) throws Exception {
        (new MyManager()).run();
    }

    private void run() throws Exception {
        pool = Executors.newFixedThreadPool(10 /* thread pool size */,
                new MyThreadFactory());

        WorkerCounter counter = new WorkerCounter();

        while (true) {
            for(int i = 1; i < 1000; ++i)
                pool.execute(new AngolanTask(counter));

            /*
             * This is a requirement I had; you may not need this: I had to execute 1000
             * tasks (for loop above), then way for every task to finish, and loop again (while loop).
             * That's why I need the WorkerCounter class I created.
             */
            counter.blockTillEveryonesDone();
        }

        // You may care to:
        // pool.shutdown();
    }
}

/**
 * The actual task run code.
 */
class AngolanTask implements Runnable {
    private WorkerCounter counter;

    public AngolanTask(WorkerCounter counter) {
        this.counter = counter;
    }

    /**
     * Called by the thread pool's thread
     */
    public void run() {
        counter.workerStarting();
        try {
            /*
             * Put your actual task code here. In particular, I need my thread-local connection.
             */
            Connection conn = DbDispenser.getDB();
        } finally {
            counter.workerDone();
        }
    }

}

/**
 * Allow the master to know when all threads have finished. Inspired on
 * http://www.ibm.com/developerworks/library/j-thread.html
 */
class WorkerCounter {
    private int activeWorkers = 0;

    public synchronized void blockTillEveryonesDone() {
        while (activeWorkers > 0) {
            try {
                wait();
            } catch (InterruptedException e) {
                // keep trying
            }
        }
    }

    public synchronized void workerStarting() {
        activeWorkers++;
    }

    public synchronized void workerDone() {
        activeWorkers--;
        if (activeWorkers == 0)
            notify(); // alert a thread that's blocking on this semaphore
    }
}

/**
 * Creates threads that clean up connection if finalized.
 *
 * Copied from Executors.defaultThreadFactory()
 */
class MyThreadFactory implements ThreadFactory {
    static final AtomicInteger poolNumber = new AtomicInteger(1);
    final ThreadGroup group;
    final AtomicInteger threadNumber = new AtomicInteger(1);
    final String namePrefix;

    MyThreadFactory() {
        SecurityManager s = System.getSecurityManager();
        group = (s != null) ? s.getThreadGroup() : Thread.currentThread()
                .getThreadGroup();
        namePrefix = "pool-" + poolNumber.getAndIncrement() + "-thread-";
    }

    public Thread newThread(Runnable r) {
        Thread t = new AngolanThread(group, r, namePrefix
                + threadNumber.getAndIncrement(), 0);
        if (t.isDaemon())
            t.setDaemon(false);

        // Set thread priority here, if needed
       
        return t;
    }

    class AngolanThread extends Thread {
        public AngolanThread(ThreadGroup group, Runnable target, String name,
                long stackSize) {
            super(group, target, name, stackSize);
        }

        @Override
        public void run() {
            try {
                super.run();
            } finally {
            }
        }
    }
}

// Connection's are not thread-safe.
// http://www.ibm.com/developerworks/java/library/j-threads3.html
class DbDispenser {
    private static class ThreadLocalConnection extends ThreadLocal {
        public Object initialValue() {
            try {
                return null; // Create your connection here.
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
    }

    private static ThreadLocalConnection conn = new ThreadLocalConnection();

    public static Connection getDB() {
        return (Connection) conn.get();
    }
}

How to debug a Zend Framework application in Ubuntu

Why I wanted to debug Zend Framework
I got started with CodeIgniter, and I later found out that it's designed to not work well with debuggers. There are CodeIgniter forum posts dating as far back as 2007 where you can get instructions on how to patch the source code and tweak the config files so that query string (GET params) work, which XDebug and Zend Debugger both need to work. I spent hours trying to debug with Aptana PHP (plugin 1.0 and 1.1 Beta) and Eclipse PDT, and failed. I decided to trash a framework so backwards as CodeIgniter.

How to
If you don't want to install Zend Platform (the paid PHP server), it may not be much too intuitive how to hit the ground running with a sample Zend Framework project and be able to debug it. Here's who I finally managed. (I use Ubuntu 9.04)

• Install Zend CE, the free Zend Community Edition server, as explained in http://files.zend.com/help/Zend-Server-Community-Edition/deb_installation.htm

echo 'deb http://repos.zend.com/deb/ce ce non-free' | sudo tee -a /etc/apt/sources.list
wget http://repos.zend.com/deb/zend.key -O- | sudo apt-key add -
sudo aptitude update

sudo apt-get install zend-ce

Add PHP to your path:
sudo ln -s /usr/local/zend/bin/php /usr/local/bin/php

• Install Zend Studio for Eclipse from http://www.zend.com/en/products/studio/downloads. I used Studio 6.1. Zend Studio is not free! It costs $400 and comes with a 30-day trial. You can try also with Eclipse PDT, which is the free (scaled-down) version of Studio.
  

• Download the project Zend Quickstart and unzip it. I renamed my directory zend-quickstart. We'll call the root <path to project's root>

In <path to project's root> run, as instructed by the README.TXT file:
php scripts/load.sqlite.php --withdata
chmod -R a+rwX data

• Counter intuitively, the Debug command in Zend Studio doesn't actually deploy your project to the PHP server, so you need to do that yourself. By default the document root for Zend CE is /var/www
  

ln -s <path to project's root> /var/www/myprojecturl   (you can change myprojecturl obviously)
Now you can access the project at http://localhost/myprojecturl

• Back in Studio, go to Run==>Debug Configurations. Configure this:
  

File: zend-quickstart/public/index.php
Server Debugger: Zend Debugger

Click PHP Server Configure. Go to Path Mapping tab, and Add:
* Path on Server: myprojecturl
* Local path: /zend-quickstart

Make sure the URL generated is correct. Should be /myprojecturl/public/index.php

• Cool! Click the Debug button now, and the debugger should be stopped at the first line of the index.php script. Breakpoints should work.
  

My favorite Firefox addons

To add a Firefox addon, go to the Tools->Addons menu, and on the first (Get Addons) tab, search for the name of the addon.

• Cookie Swap: Instantly switch between different accounts in the same web site (e.g., Gmail). Forget about logging off and back.
• Firebug: No web developer can do without it.
• Google Toolbar: It's good to know the PageRank of the company you're checking out. It's like asking the rest of the world how good they are, and getting a single digit for an answer.
• NetVideoHunter: Download videos from video sharing sites (e.g., Youtube and others)
• TabGroups Manager: Organize your zillions of open tabs!
• Tabs Open Relative: Who at Mozilla came up with this idea that when you open a link in a new tab it should open at the far right? Fix it with this plugin.
• Restart Firefox: Don't want to close all your open firefox windows before restarting? Use this. (You should have many firefox windows if you're using TabGroups, but...)
• Tiny Menu: Reclaim your valuable vertical screen space.

Ubuntu/linux, how to fix error: "checking for zlib.h... no configure: error: You must have zlib.h and zlib installed"

Solution:
sudo apt-get install zlib1g-dev

"Facebook API "Fatal error: Call to undefined method FacebookRestClient::stream_get() in ..."

Fatal error: Call to undefined method FacebookRestClient::stream_get() in /home/aptana/domains/x.aptanacloud.com/web/htdocs/index.php on line 18

NOTE: If you're using the official PHP client library and you downloaded it from the example code link on the Developer Application, you may have gotten an outdated release that doesn't contain the stream_get method and others released recently, and thus you'll get runtime errors. Use the official Facebook PHP Client Library package from SVN for the latest code.

Export Microsoft Outlook contacts and calendar into Evolution

Using Evolution 2.26.1, if you go to the program's help and search for "outlook", you'll get instructions to export your contacts & calendar from outlook as CSV (DOS). In Outlook 2003, doing that for contacts doesn't work (when you import, you get thousands of bogus contact data); doing that for calendar doesn't work either (nothing gets imported).
Also using Outport for exporting calendar didn't work for me; evolution didn't recognize the ics file as valid.

The solution I found was to install Outlook 2007 (if you don't have it, grab the free Trial from Microsoft), and:
- Export contacts as CSV as before

- Export calendar by using Save As, as explained here. This way, not only did it work well, but also avoided problems with recurring events, which don't get exported correctly by exporting to CSV.

You can import these into Google Calendar (nice to have SMS notifications!) and Google Mail contacts. The only problem seems to be that characters like "ñ" get scrambled.

My favourite Joomla extensions

Article version control: http://www.fatica.net/downloads/simple-content-versioning.html . Never again overwrite contents accidentally!

Editor: JCE. www.joomlacontenteditor.net  - http://extensions.joomla.org/extensions/88/details

Photos: Embed your picasa web albums as a slideshow in your articles: http://picasa.google.com/support/bin/answer.py?hl=en&answer=66969

To upload your albums, the easiest way is by using the Picasa desktop software.

AutoHotkey Replacement for Linux / How to Activate Applications with Hot Keys

I love AutoHotkey for Windows because you can set up a key like Ctrl+Alt+F to activate the Firefox window if it's running.

From this thread, I discovered wmctrl and keylaunch:

sudo apt-get install keylaunch wmctrl

Then create a file ~/keylaunchrc. For example:

# Format:
# key=...KeyName:Command
#
# ... No modifier
# *.. Shift
# .*. Ctrl
# ..* Alt
key=.**f:wmctrl -a Mozilla Firefox   # Activate a window that contains 'Mozilla Firefox' in its title when you press Ctrl+Alt+F


Finally, you need to run keylaunch to get the hot keys working.

Also, check out launchy: It's very handy for launching an application by typing the name.


Matt

How to run a VMWare/VMplayer machine in VirtualBox

At least as of VirtualBox 2.2.2 for Linux, you don't need to convert the VMWare hard disk. You can just create a new machine in VirtualBox, and as a disk select the virtual hard disk file in you VMWare machine. Done! (You'll just need to install the VirtualBox machine additions in the new machine to enjoy the same level of integration)

And remember, as the Help of VirtualBox proclaims...

Warning
Do not run other hypervisors (open-source or commercial virtualization products) together with VirtualBox! While several hypervisors can normally be installed in parallel, do not attempt to run several virtual machines from competing hypervisors at the same time. VirtualBox cannot track what another hypervisor is currently attempting to do on the same host, and especially if several products attempt to use hardware virtualization features such as VT-x, this can crash the entire host.

How I installed Ubuntu 9.04 in a Dell Vostro 1510 notebook

Update: My pains to this day with Ubuntu 9.04:

• Reboots some times when plugging in the USB mouse / keyboard (maybe 3 times a week)
• VirtualBox (for running a few Windows XP programs): Reboots some times when starting (rare), and more often the virtual machine can't be resumed, so it must be forcefully reset (the guest, not the Ubuntu host). The sound is very choppy in the virtual box. Could use the paid VMWare workstation, but it doesn't work with sound. 
• Video playback in sites like Youtube is so-so (particularly full-screen; it gets choppy)
• Have yet to find a program that can speed up video playback reliably like I used to do with BSPlayer on XP (VLC and MPlayer both have the capability, but it doesn't work well in my Ubuntu)
• Skype is not worth using. It often times consumes 100% of the CPU, audio is bad, it's a pain to configure, and chat works only some times.
• OpenOffice.org takes 3-4 seconds to open even a small document, even when OpenOffice is already open and no physical disk access is needed (vs. Office 2003, that takes 1s).
• Hibernation takes about a full minute to resume! Stand by takes almost a full minute to shut off (but resumes quickly, though some times it does not resume at all!)
• Can't play videos in one particular site that uses WMA codec (spent a couple of hours trying, even with Medibuntu)
• Applications like Firefox and OpenOffice seem to take a bit too long to update the screen, such as when switching between applications.

Pros:

• Very RAM frugal! I like the fact that I can work with two instances of the bloated Eclipse suite and other heavy-weight programs running and only consume a paultry 1.4 GB of RAM (whereas Vista would be at twice that much or more).
• Linux command line ;-)

Conclusion: I like Linux, but if I had to choose now an OS, I would go for XP: it's 8 years old, unsafe and much more, but it generally works out of the box with most software, and forum answer hunting nights are a foreign experience. It's a matter of productivity.

To Install Ubuntu 9.04 in a Dell Vostro 1510 notebook

First of all, don't download the official ubuntu 9.04 CD. Download the Dell one: http://linux.dell.com/files/ubuntu/jaunty/iso-images/ubuntu-9.04-dell-reinstall.iso (from http://linux.dell.com/wiki/index.php/Ubuntu_9.04 ). With the Dell one, apparently, sound works off the shelf.

Wireless: I have Dell 1505 Wireless-N. Follow https://help.ubuntu.com/community/WifiDocs/Driver/bcm43xx/Feisty_No-Fluff    with step 2d

Audio: (Because I installed the official Ubuntu CD, not the Dell one)

(Option 1. Described below. I started with this, got it to work, and a few days after (lots of system tweaking action in the middle), it stopped working, so I did Option 2 below. So my advice is, forget about option 1.)

Option 2.

You can use this to reload the alsa config: sudo alsa force-reload   

I got the following by booting from the Dell live cd (using usb-create). Put the following into /etc/modprobe.d/alsa-base.conf

# autoloader aliases
install sound-slot-0 /sbin/modprobe snd-card-0
install sound-slot-1 /sbin/modprobe snd-card-1
install sound-slot-2 /sbin/modprobe snd-card-2
install sound-slot-3 /sbin/modprobe snd-card-3
install sound-slot-4 /sbin/modprobe snd-card-4
install sound-slot-5 /sbin/modprobe snd-card-5
install sound-slot-6 /sbin/modprobe snd-card-6
install sound-slot-7 /sbin/modprobe snd-card-7

# Cause optional modules to be loaded above generic modules
install snd /sbin/modprobe --ignore-install snd $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-ioctl32 ; /sbin/modprobe --quiet --use-blacklist snd-seq ; }
#
# Workaround at bug #499695 (reverted in Ubuntu see LP #319505)
install snd-pcm /sbin/modprobe --ignore-install snd-pcm $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-pcm-oss ; : ; }
install snd-mixer /sbin/modprobe --ignore-install snd-mixer $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-mixer-oss ; : ; }
install snd-seq /sbin/modprobe --ignore-install snd-seq $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq-midi ; /sbin/modprobe --quiet --use-blacklist snd-seq-oss ; : ; }
#
install snd-rawmidi /sbin/modprobe --ignore-install snd-rawmidi $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq-midi ; : ; }
# Cause optional modules to be loaded above sound card driver modules
install snd-emu10k1 /sbin/modprobe --ignore-install snd-emu10k1 $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-emu10k1-synth ; }
install snd-via82xx /sbin/modprobe --ignore-install snd-via82xx $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq ; }

# Load saa7134-alsa instead of saa7134 (which gets dragged in by it anyway)
install saa7134 /sbin/modprobe --ignore-install saa7134 $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist saa7134-alsa ; : ; }
# Prevent abnormal drivers from grabbing index 0
options bt87x index=-2
options cx88_alsa index=-2
options saa7134-alsa index=-2
options snd-atiixp-modem index=-2
options snd-intel8x0m index=-2
options snd-via82xx-modem index=-2
options snd-usb-audio index=-2
options snd-usb-us122l index=-2
options snd-usb-usx2y index=-2
options snd-usb-caiaq index=-2
# Ubuntu #62691, enable MPU for snd-cmipci
options snd-cmipci mpu_port=0x330 fm_port=0x388
# Keep snd-pcsp from beeing loaded as first soundcard
options snd-pcsp index=-2

(Option 1. 

With the following I made it possible to play music in linux while having VirtualBox running on ALSA and playing sound (you must configure the VirtualBox sound setting to ALSA).

From http://www.stupent.at/2008/06/18/dell-vostro-1510-with-ubuntu-804/
added to administration->software sources->updates, this: pre-released updates & unsupported updates. From: http://linuxtechie.wordpress.com/2007/10/19/getting-intel-ich8-family-rev-3-sound-card-to-work-in-gutsy/ (this is referenced by the first URL)
sudo apt-get install linux-backports-modules-jaunty
Also: “inserting the following line to /etc/modprobe.d/alsa-base did a great job for now:
options snd-hda-intel model=toshiba position_fix=0 enable=yes

“)

How I Solved my Python, Django & jQuery Javascript Unicode Problems

JavaScript:

The JavaScript encode() function doesn't handle Unicode. See http://xkr.us/articles/javascript/encode-compare/ for alternatives.

The jQuery serialize() function works well with Unicode.

Python:

urllib.urlencode(), urllib.quote() and urllib.quote_plus don't handle unicode strings. So instead of

urllib.urlencode(u'a unicode string')

... do:

urllib.urlencode(u'a unicode string'.encode('utf-8'))

In the above case I'm assumming that the string is already Unicode (which seems to be the case when you got it off Django, maybe from a GET parameter). If it's in another encoding, you must previously do:

'a byte string encoded in X'.decode(encoding_goes_here).encode('utf-8')

": needs update fatal: Entry '' not uptodate. Cannot merge."

I got this error when trying
git pull
": needs update
fatal: Entry '' not uptodate. Cannot merge."

Solution:
git commit

Windows Vista deception -- "Error: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions"

I just got started on Vista after years of developing on XP. When I tried Django's
manage.py runserver
which defaults to localhost:8000, I got this error: 
Django version 1.1 alpha 1 SVN-10020, using settings 'mashup.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
Error: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions

It turns out Aptana Studio is also using port 8000 for it internal server, so Vista instead of reporting that port 8000 is already binded to, rambles about access permissions. I had been doing the same thing in Windows XP, but apparently XP lets you run two servers on the same port without complaining (crazy!)

Solution:

manage.py runserver localhost:8001

Matt