Quantum Mechanics and Random Not Random (Emergent Behavior, AI)

Sometimes I have little epiphanies and nowhere to save them. These are likely wrong but I think they help me to understand things better. Maybe not perfectly, maybe not even correctly, but better. Here's one.

QM invites all sorts of woo ideas that are, in a word, bad. It has things that exist in superpositions that don't manifest until they are "observed". It's this mysterious thing that is completely random and unpredictable, except that it's not: it is completely predictable and one of the best tested scientific theories so far developed. The behavior of individual elements are unpredictable but the system as a whole is not. This can be called the system's emergent behavior (note again that this is not my profession and this detail may be wrong), but it is this system behavior that is well-described by various equations like ideal gas laws and fluid dynamics, etc.

QM has a "wave function" and people debate whether this math is representative of some reality, or if it is just math that works for some reason (the "shut up and calculate" school). People ask how wave functions collapse, and whether there are hidden variables. I think my analogy implies hidden variables (which Bell's Theorem discounts) but I write this because it makes emergence appear less magical.

Imagine you have a gumball machine. In this machine are a number of gumballs of different colors. The machine has 12 red, 7 yellow, and 5 blue. Each time you pull out a gumball, the color will be completely random. But you know that after 2 dozen withdrawls you will have 12 red, 7 yellow, and 5 blue. The result is known even if every intermediate step is completely random.

This is also how the double split experiment behaves. If you don't look at the mechanism, the gumball you get is random and the photon goes through "both slits". If you watch to see which gumball will or will not be selected, you see which slit the photon "chooses" to pass through. (This is clearly wrong as the double slit experiment is about interference patterns which disappear, but let's continue past this division by 0.)

Now some text about AI.

LLM (Large Language Models) are the current "best" AI technique available for Artificial General Intelligence. But all they are are really sophisticated autocomplete machines. Based on the text scraped from the open internet, libraries of books, etc, they predict the most likely "next tokens" (letters, spaces, random parts of words) based on the text that is given to them. They incorporate randomness like stable diffusion to make the responses "fuzzy" because fuzzy is better.

An interesting thing about these models is that they just autocomplete text. But if you ask an LLM to digest a large block of text and summarize it, somehow it does. Nobody trained it to do this, it just does. It "knows how" implicitly. It "learned," as an emergent behavior. What is not currently clear is if it can do anything novel.

The English language is a mess of weird spellings. In large part this is because of a few "ivory tower" individuals trying to revive classical Latin words, applying their spellings to a language that was written in a runic alphabet and not a Latin one, during the time of a great vowel shift, at the same time that Gutenberg's Movable Type Printing Press was being invented for a different language. The Americans underwent a spelling reform in the early 1900s that the British did not, dropping a bunch of unnecessary "u"s and not enough "ough"s, but it's still a mess.

Today's AI is not intelligent. It has been developed using a lot of old text, but also a whole lot more new text, using the open internet as source material. The current internet has a lot of sci-fi text describing AIs that go rogue and that people end up powerless to try to control. These stories present no solutions, instead referring to this as the "alignment problem". AI is being fed this source material, as the Press was being fed English words and a Latin alphabet.

This material input - today's 4chan reality, today's biases and prejudices, today's "haves" versus those not publishing content on the open internet - is rapidly being locked in. The open web is quickly filling with AI slop, which is text being generated by AI using the text it has been trained on. New text will be less and less human generated and more regurgitation of today's current mores. Without meaningful change, we will use AI to baby-bird ourselves for some indefinite future.

I have no conclusion for this essay but would love to hear any novel thoughts about AI or emergence.

KDE screensaver in Ubuntu 13.04, part 2: disable locking

Some security updates upgraded kde-workspace-bin, which upgraded /usr/lib/kde4/libexec/kscreenlocker_greet, which restored the infuriating behavior I've previously described: notably, that the screen locker runs after ~4 seconds of idle.

It's still not entirely clear to me what's happening, but it looks like the scenario goes something like this: ksmserver runs kscreenlocker_greet all the time. kscreenlocker_greet looks like it examines the configuration file(s) $HOME/.kde/share/config/kscreensaverrc and/or /usr/share/kde4/config/kscreensaverrc for settings and triggers off of those.

This can't be how it's supposed to work, because kscreenlocker_greet is only run once at the beginning of each idle period. I've verified this by replacing kscreenlocker_greet with this script:

#!/bin/sh
echo "`date` - start: $0 $@" >> $HOME/kscreenlocker_greet.log
exit 0

I can then "tail -f" the log file and watch as KDE tries to lock my screen.

Fortunately, kscreenlocker_greet appears to read the kscreensaverrc configuration file, which I've modified to read as this:

Enabled=false
LegacySaverEnabled=false
Lock=false
PlasmaEnabled=false
Saver=krandom.desktop
Timeout=1800

What all of those mean I cannot say, but "Enabled=false" and "Lock=false" seem to be helpful.

The part that's still troublesome is that while the screensaver is being activated, my keystrokes are being consumed, so it appears that I'm always dropping letters as I try to type.

Whatever's going wrong is doing it in ksmserver, and that program, naturally, has no man page. For now, this config file change should prevent the annoyance from recurring after an upgrade. I'll leave digging into ksmserver for another day...

Fix to prevent line breaks at hyphens in HTML

The problem

When writing HTML you may want to include something like a phone number in the form 1-800-555-1212. The problem is that this might be displayed as:

... 1-800-555-
1212...

This is probably not what you want - you'd prefer the entire thing to appear on one line.

Solution(s)

Non-breaking Hyphen

This is the most obvious logical solution - we have non-breaking spaces ( ) so all we need is a non-breaking hyphen, right? Well, not quite.
First, there is no standard entity for a non-breaking hyphen. Instead, this is a character defined in the Unicode specification as character #8209. If you want to use this in your HTML you can use either ‑ or ‑. The problem here is the font - most fonts probably don't include this character and will render it as some box with either digits or punctuation inside. What other options are there?

CSS white-space

This is the most correct answer. It requires two steps. First, define a CSS class:

.nobreak { white-space: nowrap; }

Then use it your HTML as:

<p>Lorem ipsum <span class="nobreak">1-800-555-1212</span>.</p>

The especially nice thing about this is that it's generic and will work equally well with:

<p>Dolor amit <span class="nobreak">1 (800) 555-1212</span>.</p>

My hack: super-under

And just in case you're curious what kind of craziness I can come up with, here's my third possible solution:

<p>Sibiliy si emgo: 1^_800^_555^_1212.</p>

Yes, that's right: Take an underscore character ('_') and superscript it. A work of genius I say ...

Moving virtual hard disks in VirtualBox

Under Linux, VirtualBox likes to put it's virtual disks in a relatively inconvenient directory. Newer versions are more obnoxious, and have removed the ability to easily manage your virtual disks. As most of the information I find online is old and does not apply to VirtualBox 4 or 4.2 (the specific version I have, shipped with Ubuntu 13.04), or may work but just seems too hard, here's a nice little recipe:

To move a virtual disk from point A to point B:

1. Ensure the virtual machine(s) owning the disk are powered off.
2. Open the settings for the virtual machine(s) owning the disk.
3. Go to "Storage", select the disk, and click the Remove icon. Then click OK.
• Repeat for each associated VM as necessary
4. From the main VirtualBox panel, select File->Virtual Media Manager
5. Click on the Virtual Disk image, verify the Location, and that it is not attached to any VMs.
6. Click the Remove icon along the top *
7. Click Close.
8. Now, from the commandline or file manager, move the newly-freed VirtualImage.* files from point A to point B.
9. Finally, back in VirtualBox, select the virtual machine configuration and select "Settings".
10. Go to "Storage" and select the Controller where the virtual disk used to be.
11. Click on the Add Attachment icon and select Add Hard Disk.
12. Select Choose existing disk, navigate to point B, and select your virtual disk.

If you try to skip steps 4-7 VirtualBox will complain about adding a disk image with an already existing UUID, so that image has to be removed from the Virtual Media Manager first. If you have multiple VMs attached to the same disk this can get tedious, but it's better then the other suggestions I've read, including cloning the disk or VM, or even removing the virtual machine itself (including all your settings!)

This also has the nice benefit that *you're never going to be prompted to delete the files from disk. (And if I'm misremembering that and you are prompted, but sure to select NO.) You may be able to select "Detach" from the Virtual Media Manager instead of going through each configuration, but I'd recommend doing it the hard way so you can be sure to take notes and verify what you're doing as you do it.

Crash course for CVS users switching to git

I've used CVS for a long time and have been generally happy with it. Some of its quirks have caused me to consider subversion but none were really pressing enough for me to do so. I've tried to use git in the past - honestly, I have - and have always ended up extremely frustrated. But I have crossed the mental rubicon and wanted to share what was so difficult for me to grasp, in case there are any dinosaurs like me still out there.

How I use(d) CVS

It is probably best to describe my usage of CVS first. CVS has a model of a single central repository, and multiple simultaneous editors. Each checkout by an editor is referred to as a 'sandbox'. Editors work in their sandbox and commit to the repository.

One additional thing to note is that CVS has a concept of 'tags', which is a way to label a snapshot of all the files in the repository (it could also apply to a subset, but we're keeping this simple). A quirk here is that a "tag -b", or "branch tag" is a label that can be used to denote a separate line of development.
This model has allowed me to use the following workflow:

• Create some product and release version 1.0
• cvs tag -b VERSION_1_0
• The name format is another CVS quirk, but this branch tag lets me not only label the files that make up version 1.0, but create a branch point where I can develop new features.
• Begin working on version 2.0. As features get completed, commit to the (main branch of the) repository. NEVER commit a broken configuration as this will upset other developers when they check out the code.
• A bug is reported in version 1.0
• Go to a different directory, and cvs checkout -r VERSION_1_0
• This new sandbox will be devoted to identifying/fixing this bug.
• Fix the bug, and cvs commit it. Then cvs tag BUG_1001.
• This effectively creates VERSION_1_0 / BUG_1001.
• Continue until you release and branch tag VERSION_1_1.
• Always do the tag in a clean sandbox (with a fresh checkout) to ensure it functions.
• Meanwhile, the other sandbox can continue developing VERSION_2_0
• Prior to release of version 2.0, review bug fixes to see if they are applicable.
• cvs merge BUG_1001, etc. as necessary.
• Delete any sandboxes used for failed experiments or resolved bugs

This is an iterative process that gets more involved with more people and features, etc, but the key is that simultaneous development happens, each in its own sandbox.

This is not how git wants to work.

Development under git

There are two key things to recognize when developing with git:

• Discard the idea of a sandbox
• Discard the idea of a commit

I'll also add:

• Discard the idea of a repository

In git, most frustratingly to me, the 'sandbox' and the 'repository' are gone. Instead, there is only your work area. When you commit files, they go into (by default) the .git directory in your work area. If you remove your work area, you remove your repository (the .git directory). These CVS ideas must be completely disabused in order to use git well.

Also in git, branches are not 'clean'. Under my CVS workflow, a sandbox was a branch. I would commit what I wanted and delete the entire sandbox after I'd verified my commit. Any stray files created would be removed. In git, any files created in your work area will be carried around from branch to branch as you checkout.

The key idea of git that you have to get used to is what is called the 'index'. You can think of the index as your commit target. When you are happy that your code doesn't break things, you stage it in the index. Think of this as a lightweight 'commit':

• Create files onefish.txt and twofish.txt
• Modify files redfish.txt and bluefish.txt
• See these files and see that they are good.
• git add '*fish.txt'

The result of 'git add' isn't just "Hey, add these files", like it is in CVS -- instead, it's "Hey, I want these changes to be staged in the index". When this command completes, the 'index' has been updated, just as though you had done a 'commit' to it. Then, a 'git commit' pushes the changes in the index into the repository. The files in your work area are not a part of this equation AT ALL.

If you edit README, 'git add' README to the index, edit README again, and then 'git commit', you are committing the first set of changes because the second set was not staged before the commit. I suspect many people will use the 'git commit -a' form of the command by default, which does an implicit 'add' to the index of all modified and removed files, but not new files not yet 'git add'ed. This actually makes good sense because, as I noted above, git branches are not clean.

I don't like the 'commit -a' form as it seems too risky at this time -- I prefer to manually 'git add' each change before the commit. But sometimes I forget to add a new file that I created. Fortunately, git makes this really easy to fix: 'git commit --amend'. No matter how many things you got wrong with the last commit, you can just fix it like this. Add new files, new changes, fix the comment, etc. That's really handy.

So this is a workflow that I'm happy with for now. Creating branches is a simple 'git checkout -b branchname', switching branches is the same without the '-b' parameter. Crap in your work area remains in your work area, and the index remains as well. Some may like this because you can commit to a different branch than you started out on, but CVS has a simpler "cvs commit -r banchname" (and git probably does too).

The problem I have is the crud that persists in your work area as you switch branches. Technically I don't know where else it would go, but it seems bad form to me to carry this around. What do people do here? Do you check this in-progress and probably-won't-compile code into the branch while you switch to work on a bug? Is "never commit something that doesn't work" no longer a valid rule? Or do you leave it and 'git clone' the repository into another 'sandbox'?

Using 'git clone' seems like an acceptable solution, except it makes me ill at ease to 'rm -rf' the directory because it carries around the entire repository in its .git folder. And without a central server, there is no backup if you make a mistake and accidentally remove the last copy of the project's .git folder. Under CVS you may lose your changes, but in git you lose everything.

So I've set up git as a server which I'm using as my master repository and developing in multiple "sandboxes" cloned from the central repository, using minor feature branches in each work area as I go. When I believe I am feature complete, I push to the server as my final 'commit' and, if appropriate, 'tag'.

Once CVS users can get into the flow of adding to the index as a lightweight commit, the rest of git is fairly intuitive. Though I clearly still have much more to learn, I think that this fundamental understanding will make me a very happy git user.

Next project: do something about Blogger's horrid editor and/or my blog styling.

Part 2: Create a SOAP::Lite Server that uses Basic Authentication for password verification

Today is a better day than yesterday (see yesterday's part one of this post for details). Despite Google appearing to be ever more useless in the chaff that is today's WWW, and Bing no better, I've made some good progress with my SOAP experiment. It turns out the key to my next problem was solved in 2004. But let's recap:

I want to write a SOAP server. I want to authenticate clients somehow - username and password are a good start. I want every request to be authenticated, and a Basic Authentication Realm will work just fine.

Why make every request authenticate? Because the web is a stateless protocol. When you "log in" to a website, you really are just requesting a cookie. This cookie is a magic number that is stored on the server and means "your session". When you load another page on that same site, the browsers sends that cookie along as well. But the server still has to authenticate the validity of that cookie because it will be sent on the next page load, whether it's made in 2 seconds or 2 years. Basic Authentication is similar, except it's built-in to the browser. This has downsides though, the biggest is that your browser asks you to log in so the website developer can't make a fancy branded prompt. But I digress...

The second issue I will run into is that I want to be able to retrieve those credentials from the class methods being called. There are a number of reasons to do this: perhaps different clients will get different responses, or perhaps I want to log some AAA data. This is where I need to search back to 2004.

So let's revisit with some source code.

The SOAP server

• soapDaemon.pl

    use strict;
    use SOAP::Transport::HTTP +'trace';

    # don't want to die on 'Broken pipe' or Ctrl-C
    #$SIG{PIPE} = $SIG{INT} = 'IGNORE';
    $SIG{PIPE} = 'IGNORE';

    #my $daemon = SOAP::Transport::HTTP::Daemon
    my $daemon = BasicAuthDaemon
          -> new (LocalPort => 8000, Reuse => 1)
          -> dispatch_to('myClass')
          ;

    print "Connect to SOAP server at ", $daemon->url, "\n";
    $daemon->handle;

This is the same code that can be found in the documentation all over the internet. I've added tracing and disabled the ^C handler, but this should be straightforward to understand. Again, note that I am replacing the default HTTP::Daemon with my custom BasicAuthDaemon. We'll run on port 8000 and send requests to the 'myClass' object.

The modified HTTP::Daemon class

• soapDaemon.pl

    package BasicAuthDaemon;
    use strict;
    use warnings;
    use MIME::Base64;
    use SOAP::Lite +'trace';
    use SOAP::Transport::HTTP ();
    our @ISA= 'SOAP::Transport::HTTP::Daemon';

    sub handle {
        my $self = shift->new;
        while ( my $c = $self->accept ) {
            while ( my $r = $c->get_request ) {
                $self->request($r);

                my ($type, $creds) = split /\s+/, $r->headers->authorization;
                my ($user, $pass) = split /:/, decode_base64( $creds )
                    if( $type eq 'Basic' );
    print "type: [$type], user:[$user], pass:[$pass]\n";

                if( $user eq 'user' and $pass eq 'password' ) {
                    $self->{'auth'} = "123123123";
                    SOAP::Transport::HTTP::Server::handle $self;
                    #$self->SUPER::handle;
                } else {
                    $self->response( $self->make_fault(
                          $SOAP::Constants::FAULT_CLIENT, 'Authentication required',
                          'Give authentication credentials for Basic Realm security'
                      ));
                  }
                  $c->send_response( $self->response );
              }

    # replaced ->close, thanks to Sean Meisner <Sean.Meisner@VerizonWireless.com>
    # shutdown() doesn't work on AIX. close() is used in this case. Thanks to Jos Clijmans <jos.clijmans@recyfin.be>
              $c->can('shutdown')
               ? $c->shutdown(2)
               : $c->close();
              $c->close;
        }
    }
    1;

This code is in the same file as the server code above. I actually have this as the top of the file but it takes a bit more explaining. Firstly, this is still just test code because you're unlikely to use HTTP::Daemon in production. I think this actually gets easier in production because the handle() method doesn't have a service loop where requests have to be processed. See here for some clarification. 

However, what I've done is gone into the file perl5/SOAP/Transport/HTTP.pm and found the handle() method within the SOAP::Transport::HTTP::Daemon package, and copied it into my code. I've left the original comments and expanded formatting to show that this is a copy-paste and not my own code. My additions are bold and removals are italic.

My added logic takes the HTTP request and references into the authorization headers that are sent for the Basic Authentication mechanism. You can use a similar routine to extract and verify cookie data, etc.

If the username and password match, I jam a value into $self, which is a SOAP context. I'm simply creating a hash called 'auth', which could be enhanced. Is there a 'stash' in this architecture? I don't think so... Should I call it something more unique like 'myAppAuth'? Probably. But this is where we begin.

Aside from adding my custom logic, one more modification was necessary: remove the call to $self->SUPER::handle. This is necessary because our superclass has the same event loop and will just hang waiting for another client. Instead we need to call the superclass of our superclass, which is SOAP::Transport::HTTP::Server. I pass $self as the first argument because this is how perl seems to do OO things, and it works nicely.

Finally, if the credentials don't match, I return a SOAP error.

The client (in perl)

• soapClient.pl

    use strict;
    use SOAP::Lite +'trace';

    my $soapClient = new SOAP::Lite
        uri => 'http://example.com/myClass',
        proxy => 'http://user:password@localhost:8000/',
        ;

    my $result = $soapClient->hi( 'this is how the world ends', 'kabaam' );
    unless ($result->fault) {
        print "\nresult: [" . $result->result . "]\n\n";
    } else {
        print join ' --=-- ',
        $result->faultcode,
        $result->faultstring,
        $result->faultdetail, "\n";
    }

This is the same client as before. Again, the uri is the method being called and the host name appears to be ignored. The proxy is the actual web server, and is where we place the username and password.

The client (in PHP)

• soapClient.php

    $client = new SoapClient(null,
        array('location' => "http://user:somepass@localhost:8000/",
              'uri'      => "http://test-uri/myClass",
              'login'    => "some_name",
              'password' => "some_password",
        ));

    $res = $client->hi( 'from php!' );
    print "got: $res\n";

Here's the same SOAP client written in PHP for comparison. Placing the username and password in the URL doesn't work in PHP, so these have to be explicitly added. Note again that the server doesn't seem to care about the server portion of the URI. On error, PHP throws an exception.

The SOAP class

• myClass.pm

    package myClass;

    use strict;
    use vars qw(@ISA);
    @ISA = qw(Exporter SOAP::Server::Parameters);
    use SOAP::Lite;# +'trace';

    sub hi {
        my $evp = pop;
        my $context = $evp->context;

        my ($name, @args) = @_;
        print "[$name] got arguments: [@args]\n";

        return "hello, world, auth user=$context->{'auth'}";
    }

    sub bye {
        return "goodbye, cruel world";
    }
1;

Finally, here's the class handling the SOAP calls. Not being a perl expert, there is magic happening in here that I don't fully understand.

First, this code must exist in an external file. I surmised that this could be placed in the same file as the daemon code, and it does work for how I was using it yesterday. However, this version is a subclass of SOAP::Server::Parameters (doesn't that seem like how it should work?)  For reasons I don't understand, the subclassing doesn't work if this code is in the same file, even if I place it in the BEGIN{} block. If anyone knows why, I'd love to hear an explanation.

So now that we're a subclass of SOAP::Server::Parameters, we automatically get a new parameter added as the last argument to every function call we make. We pop this value off the end of the parameter stack, saving it as our environment pointer. This is really a SOAP::SOM object but what I want is the original SOAP context. Fortunately, this is easy to access through the ->context method.

Once I get the context, the 'auth' hash I created is ready available, and I return it in the call to hi() for verification. Fortunately, it works like - and appears to be slightly - magic.

Create a SOAP::Lite Server that uses Basic Authentication for password verification

Today was a rough day. I'm writing a SOAP service in perl. The basics required to make this work are fairly straightforward, but, as usual, the documentation and I can't seem to understand one another very well. As far as I can tell, this recipe exists nowhere on the internet.

Let me start with the basic example. This is what a standalone server looks like:

    use strict;
    use SOAP::Transport::HTTP;
    # don't want to die on 'Broken pipe'
    $SIG{PIPE} = 'IGNORE';
    my $daemon = SOAP::Transport::HTTP::Daemon
        -> new (LocalPort => 8000, Reuse => 1)
        -> dispatch_to('class::method')
        ;
    print "Contact to SOAP server at ", $daemon->url, "\n";
    $daemon->handle;

How much simpler can this get? We use the required SOAP module, setup a signal handler, and create an HTTP daemon. We set it to run on an unprivileged port (8000) and setup socket reuse. It has a single method that can be called from the 'class' package. We print out the URL and start handling requests.

If you're really just starting out, this should work to define the 'class::method' handler, for reference:

    package class;
    use strict;
    sub method { return 'Hello, world!' };

Go ahead and put it at the bottom of the file with the daemon code.

Now, to test this service you also need to write a SOAP client. Here's that code:

    use strict;
    use SOAP::Lite;
    my $soapClient = new SOAP::Lite
        uri => 'http://example.com/class',
        proxy => 'http://user:password@localhost:8000/',
        ;
    my $result = $soapClient->method( 'some_args' );
    unless ($result->fault) {
        print "\nresult: [" . $result->result . "]\n\n";
    } else {
        print join ' ** ',
            $result->faultcode,
            $result->faultstring,
            $result->faultdetail, "\n";
    }

It's a bit longer because of the error checking, but still fairly straightforward. We use the SOAP module, create a client, and call the method. Then we either print the result or the fault information. There are a couple things to note in here:

1. The 'uri' parameter can contain any hostname. The only thing the server seems to care about is the name 'class'. This identifies what code is going to be dispatched.
2. The name 'class' in the client 'url' must match the dispatch_to parameter in the server.
3. The 'proxy' is the actual URL the request will be sent to. It can be http, https, mailto:, etc.
4. We have added username and password credentials to the URL. This is important later.

Now, running this should Just Work. $soapClient becomes a remote representation of 'class', and we can call method() as though it were a local function. Of course it's not a local function and the web is stateless, so each request also passes that username and password we want to use to authenticate.

Here's what needs to happen on the server to receive and process the username and password. At the top of the file with the server code, add this:

    # subclass the default soap server daemon to handle authenticated requests
    package BasicAuthDaemon;
    use strict;
    use MIME::Base64;
    use SOAP::Transport::HTTP ();
    our @ISA= 'SOAP::Transport::HTTP::Daemon';
    sub handle {
        my $self = shift->new;
        while ( my $c = $self->accept ) {
            while ( my $r = $c->get_request ) {
                $self->request($r);
                my ($type, $crypt) = split /\s+/, $r->headers->authorization;
                my ($user, $pass) = split /:/, decode_base64( $crypt )
                    if( $type eq 'Basic' );
                print "user:[$user], pass:[$pass]\n";
                if( $user eq 'user' and $pass eq 'pass' ) {
                    #$self->SUPER::handle;
                    SOAP::Transport::HTTP::Server::handle $self;
                } else {
                    $self->response( $self->make_fault(
                        $SOAP::Constants::FAULT_CLIENT, 'Authentication required',
                        'Give authentication credentials for Basic Realm security'
                    ));
                }
                $c->send_response( $self->response );
            }
            $c->can('shutdown')
                ? $c->shutdown(2)
                : $c->close();
            $c->close;
        }
    }

What we're doing here is taking the code that can be found around line 688 of SOAP/Transport/HTTP.pm (in the SOAP::Transport::HTTP::Daemon package) and copying it into the 'BasicAuthPackage' that we are defining here. We use SOAP::Transport::HTTP and declare ourselves to be of the same type of the package we are subclassing.

Assuming the code is still legible despite the formatting, the code in blue that we added serves to extract the authentication data from the request headers, which looks like "Basic BASE64DATA==". We call the BASE64DATA "crypt" in the code but there's nothing secure about this - be sure to send credentials over SSL in production. We then decode the Base64 data to get the "user:password" string that was passed before the '@' in the client's URL. It is up to the reader to expand the code to do proper database lookups for production release.

However, do note the commented code in red: the call to $self->SUPER::handle will hang if left inline. Instead, I've replaced this with an explicit call to the handle method for HTTP::Server, which appears the be the super class for Transport::HTTP::Daemon. I'm also not an expert on SOAP so the make_fault may not be technically correct but it's working for now.

To complete the transition, we then simply redefine our daemon to use our derived class:

    #my $daemon = SOAP::Transport::HTTP::Daemon
    my $daemon = BasicAuthDaemon

Upgrading to Kubuntu 13.04 from 12.10 breaks Ubuntu 13.04 in VirtualBox

Another post in my short saga of upgrading to Kubuntu 13.04.

The situation

So here's my scenario: I'm running Kubuntu 12.10 and all is well (enough). In a VirtualBox instance I have a copy of Ubuntu 13.04 that I'm keeping up to date as it gets closer to release. In all, everything in working fine as I work inside the VM and prod at it from the outside.

Suddenly my system starts to act a bit flaky, and ultimately freezes up completely. I suspect this is actually an audio driver issue, and I figure Kubuntu 13.04's new kernel may just fix it, so I decide to upgrade to Kubuntu 13.04 after doing a hard-reset to recover from the lockup.

Debugging a hosed virtual machine

The upgrade goes smoothly enough, but my virtual machine is now completely baffooned. The screen comes up but is just the default purple wallpaper. Clicking does nothing. Eventually I get an error message that there was a system failure and perhaps rebooting will make things better. There is a button on the dialog that says "Continue" (or "OK") but doesn't show up until you hover over it. Rebooting does not help.

From the VirtualBox 'Machine' menu I can send the Ctrl-Alt-Delete signal and this is captured correctly. I can log out and log back in, but this changes nothing. I can get to a command prompt by hitting <Host>-<F1> and log in there. I create a new user with 'adduser', hit <Alt>-<F7> to return to the GUI, and log in as the new user ... but the new user has the same problem. This means that the default user account is likely fine and the problem is elsewhere.

I go to the Devices menu in VirtualBox and mount the new Guest Additions. I go back to the command prompt and install these as root. While this step is probably necessary (this is a new version of VirtualBox, afterall), it is insufficient to fix the problem. As the root user, I can run 'startx -- :1' but this presents the same type of problems.

So I reset the machine and think a bit. The VM is setup to auto-login the default user, so I leave X running and go back to the command prompt via <Host>-<F1>. I log in and try running things from the command line:

DISPLAY=unix:0 compiz

Interesting - compiz is crashing with I/O errors and complaining about not being able to access sessions, waiting for dbus to respond, not able to access pixmaps, etc. In short, complete madness. But I can run

DISPLAY=unix:0 gnome-terminal

and, after installing metacity,

DISPLAY=unix:0 metacity --replace

Some searching suggests these compiz errors have been around for a long time and therefore aren't likely to help. However, one post recommends reinstalling ubuntu-desktop. This package is already up to date, but they go on to suggest installing it from the Ubuntu Software Center. This doesn't make much sense but what could it hurt? Suddenly, I find myself strace'ing software-center trying to figure out why it's crashing and, furthermore, why it's complaining about not being able to find icons.

The ugly fix

I try reinstalling the icons it appears to be complaining about but that doesn't work. Eventually, I give up, and decide to just reinstall the entire system. For me, this doesn't mean getting out a CD, however. It means -- (a) print a list of every available package - (b) print the names of the packages that are installed - (c) filter out the packages that contain circular dependencies - and (d) reinstall them live.

apt-get install --reinstall `dpkg -l '*' | awk '/^ii/ { print $2 }' | egrep -v '(mountall|perl-base|libpam-modules|kbd|ntpdate|resolvconf|ifupdown|dpkg|bash|debconf)'`

Somewhere in there, despite some unnecessary exclusions, magic. The best part is that all my settings are retained so I don't need to reconfigure any of my settings.

Kubuntu 13.04 and immediate screen blanking after upgrade

Upgrading from Kubuntu 12.10 to Kubuntu 13.04 is a less-than completely smooth experience. I'll skip any niceties and dig right into the problems.

Screen lock problems

I don't want to spend a lot of time debugging this, but it doesn't even appear possible to shut off the screen blanker so I have to do something. My google-fu is completely overwhelmed by another screenlock bug in KDE 4.10 that appears to lock the screen if you tell it to blank but not lock the screen.

This is not my problem. My problem is that the screen locker appears not to be able to tell time. If I'm idle for more than about 15 seconds, the screen locks. Suffice it to say, this gets annoying really quickly.

The dirty hack

Here's what I've done to fix the issue, and to completely confuse myself the next time the package gets upgraded:

mv /usr/lib/kde4/libexec/kscreenlocker_greet /usr/lib/kde4/libexec/kscreenlocker_greet.old

I welcome real solutions in the comments.

Linux, Ubuntu, INotify - flash and pulseaudio sound stutter

This post was initially drafted on March 6.

Having issues with Linux slowing to a crawl over time? I certainly seemed to have this problem when running Ubuntu 12.10 and KDE (Kubuntu 12.10).

Initial symptom: Audio stutter using flash (e.g: YouTube)

Recently I had problems with Shockwave Flash misbehaving (frozen, stuttered video) in Chrome so I used the task manager (shift-escape, or right click in the tabs area) to kill the plugin and GPU processes. Unfortunately, reloading flash didn't fix the stutter, so I took a step further and killed pulseaudio. Pulseaudio manages access to the audio hardware and because has caused weird effects on my system's performance in the past it was next on the list.

pulseaudio refuses to start

However, trying to restart pulseaudio by typing 'pulseaudio' on the command line kept giving me this error:

E: [pulseaudio] module-udev-detect.c: inotify_init1() failed: Too many open files
E: [pulseaudio] module.c: Failed to load module "module-udev-detect" (argument: ""): initialization failed.
E: [pulseaudio] main.c: Module load failed.
E: [pulseaudio] main.c: Failed to initialize daemon.

Debugging the error

"Too many open files" is one of those generic and woefully overloaded errors that tends to provide little help in figuring out what's really happening. I checked my file descriptor limits using "ulimit -a" but the number of open files used by chrome was under this limit. In retrospect, this is not helpful information because 'ulimit' is used to set a limit on the number of file descriptors that can be opened by subsequent commands; it does not tell you how many files are currently open whether globally or in that shell. But running "pulseaudio -vvv" didn't provide much more useful detail:

shell$ ulimit -a[...]open files                (-n) 1024

The next tool in the toolbox is 'strace' - running "strace -fff pulseaudio" shows the error at the point that the system call being made:

inotify_init1(O_NONBLOCK|O_CLOEXEC)     = -1 EMFILE (Too many open files)

I checked the manpage for inotify_init1 which describes the error EMFILE as: "The user limit on the total number of inotify instances has been reached". By combining that with some additional searching, it appears that the system limit may be set too low for the number of inotify user instances. I found that my kernel defaults to 128:

shell$ cat /proc/sys/fs/inotify/max_user_instances 

128

Finding and fixing the root cause

And thanks to a pointer from https://github.com/nex3/rb-inotify/issues/23 to here:
http://unix.stackexchange.com/questions/13751/kernel-inotify-watch-limit-reached I updated this limit permanently:

root# sysctl -w  fs.inotify.max_user_instances=512 

fs.inotify.max_user_instances = 512 

root# grep inotify /etc/sysctl.conf 

root# vi /etc/sysctl.conf

Once I knew what I was looking for, this was easier to find:

http://askubuntu.com/questions/154255/how-can-i-tell-if-i-am-out-of-inotify-watches

Newtown redux: Raising Adam Lanza

By pure chance I went to see what science documentaries were available on Hulu and ended up at PBS, thinking I might find something interesting on Nova. Yes, I do have other things I should be doing, but finding Frontline's story, "Raising Adam Lanza" gave me an opportunity to publicly admit I was wrong in a previous post.

Frontline's story on the shooting appears to be a two-part series starting on February 19, but some portion of the story has already been printed at the Hartford Courant. Despite some of my assumptions in my previous post - and the fact that Adam was sent to a private Catholic school for a time - the statement that I heard said by Adam's aunt that his mother bought the guns she bought "for protection" was likely errant.

To the contrary, the story describes this as "... a portrait of a mother, apparently devoted but perhaps misguided, struggling to find her son a place in society, ..." and reading the article certainly paints it that way. I can't say the article explains anything, though, because the descriptions remain frustratingly superficial. I perceive a "blame the mother" vibe that I don't really care for but we aren't given much to empathize with either. In the end, I suspect that reality will be just as frustrating.

Annoyed by news - CT shooting

[Disclaimer: This is a rant. I don't know Adam Lanza or his mother. I don't know anyone's motives; the latest I've heard is that Mr. Lanza killed his mother and wanted to destroy everything that she loved (more than him, presumably.) Facts are unknown and there are many missing pieces which may require changes to this story. I'll post a corrected story in due time if anyone is interested, but for now this is a rant.]



I get annoyed easily; news reports of the shooting in Newtown, Connecticut not excepting.

News is always asking people what they think about the events, and many people seem to have responses at the ready. I haven't heard it yet, but I fully expect words like "evil" and "coward" to be bandied about by the same people who say there's no need for stricter gun control laws.

This is where I get most annoyed and want to put the blame for the shooting - this shooting and arguably every mass shooting for the past decade - squarely on Fox News.

You may consider this a logical incongruity but in the past few years and especially in recent weeks there have been a lot of shootings. Of the most recent, people are drawing parallels to the Columbine shooting, of which there are none. But there was a recent mass shooting at a mall in Portland, Oregon and not too long before at a movie theatre in Colorado. I've heard mention of the Virginia Tech shootings. Oddly, the one shooting I haven't heard about again is the massacre of a group of Amish children in Pennsylvania. This is the only shooting even remotely similar to the one in Connecticut.

Meanwhile, Florida has issued it's one-millionth concealed firearm permit. The nation is, in my opinion, now officially gun-crazy.

But why? From where does this insanity originate?

The only news broadcasting station that I hear repeatedly harp on about guns is Fox News. Their incessant message about Obama taking away people's guns is absolute baloney and a present danger to American society. I'm not going to say that only crazy people listen to Fox, but crazy people listen only to Fox. Fox helps this along by repeating that everyone else is lying to them. It is the central repository for conspiracy theorists and paranoid schizophrenics in mainstream society.

And it's all lies. The only gun law Obama has signed has been to expand the ability of people to carry firearms into national parks.  And now, because of the rash of shootings, there will come a serious effort to bring about changes regarding gun ownership in America.

I do not support additional gun control laws, but it's going to happen, and it's going to happen specifically because of Fox News and their appeals to the insane.

I briefly happened to catch the local Fox News broadcast coverage of the Connecticut shooting as well. I watched less than three minutes before the anchor suggested that public schools are too dangerous for you to send your kids there, thus promoting their alternate views of private, at-home, conservative education over, you know, actual knowledge.

Meanwhile, I think of the girl in Pakistan who was shot in the head for the sole reason that she was a girl and she was in school. I think of all the shootings over there where girls' schools are routinely shot up because their religious education teaches that women are not to be informed. And I think of all the parents that send their daughters to school anyway because it's the right thing to do. And I'm saddened for the future of America.

I'm sad because our society tolerates this Fox nonsense. I'm sad that Fox is encouraging us to be more like Pakistan: to fear public schools and turn into ourselves - just our families and our guns and our tribes and our ignorance and superstition; to believe that anything we try to do together as a society is a failure. I'm sad that there is no meaningful distinction between Fox News and the Republican Party and I'm sad that either of them still exist.

And so I wonder.

Would we be better off without guns? Do they serve a role in society? Why does a female substitute kindergarten teacher feel the need to own a Glock, a Sig Sauer, and a .223 Bushmaster M4 carbine rifle?

Can we rely on our second amendment protections to make us a more thoughtful society: not to rush to say that guns must be banned, but to peer into society itself to see the diseases within. Can it be argued that the tumult of gun violence augers a early-warning signal that society itself is deteriorating? That the deaths of a few innocents, as tragic and incomprehensible as they are, are sirens warning us to take heed, and save us as Americans from the deeper, truer threats to our civilization?

Or are we content to just say that this is what evil cowards do and that guns are both the problem and the solution?

Florida's "Stand Your Ground" Controversy

disclaimer: This has been sitting far too long as a draft. I'll add links and/or expand this post later, as I see fit.

Swimming Against the Tides of Madness

 

A brief discussion about Stand Your Ground

 

Florida's "Stand Your Ground" laws, which became in effect in 2005, have recently come under scrutiny because of the shooting of Trayvon Martin by George Zimmerman. I assume you've heard of this.

According to recent surveys, it is generally asserted that this law is good policy. Here are the pertinent numbers:

38. Florida has a law that says a person is legally entitled to fight back with deadly force if they feel threatened, even if they could retreat instead. Do you support or oppose that law?

                     Tot    Rep    Dem    Ind    Men    Wom    Wht    Blk    Hsp
 
Support             56%    78%    32%    58%    65%    48%    61%    30%    53%
Oppose              35     15     59     35     31     39     31     56     36
DK/NA                 8      7      9      8      4     12      7     14     11
 
                     North/                             AREA...............
                     Panhd  Bay    Cntrl  SthW   SthE   Urban  Suburb Rural
 
Support              65%    59%    57%    57%    46%    50%    58%    67%
Oppose               30     33     34     32     44     39     37     26
DK/NA                 6      8      9     11     10     12      6      6

White and Hispanic generally support the laws more than Black; Men more than Women; and Panhandle/Rural more than Urban.

Recently, a man described by neighbors as the neighborhood nutjob shot and killed a door-to-door salesman because, he claims, he was "in fear" of the salesman. First, he shot the salesman in the shoulder as he approached. Then, after falling to the ground, he shot him once in the back of the head.

Can "Stand Your Ground" be used a defense here? What is the law?

First, a disclaimer: I am not a lawyer, but I have heard lawyers talk about this issue. I am writing this up so that I have a single page where I can point others to my thoughts.

Why was "Stand Your Ground" Needed?

 

Prior to 2005, Florida had two primary protections for people who use "self-defense" as a legal defense. The first is called the "Castle Doctrine". Under the concept that a Man's Home is His Castle, you have the right to use deadly force at any time from within your home or vehicle. If someone arrives at your door with a hammer and screwdriver, you can shoot them right through the door as long as they have no legal right to enter.

The second protection is self-defense in a public place. Here, the victim is obligated one action: the duty to retreat, and this duty is limited to those times where it can be done safely. Does the assailant have a gun? You cannot outrun a bullet and so have the right to use deadly force. If you run away will they be able to grab you? If yes, then you have the right to use deadly force.

Do we think that these laws are too restrictive? That they penalize the victims of assault?

What does Stand Your Ground do then?

So what is it that "Stand Your Ground" actually does? In short, it removes the duty to retreat. It applies the Castle Doctrine to anywhere you can legally be. But it does more than that as well. Here is the pertinent part of the text:

[...] person who is not engaged in an unlawful activity and who is attacked in any other place where he or she has a right to be has no duty to retreat and has the right to stand his or her ground and meet force with force, including deadly force if he or she reasonably believes it is necessary to do so to prevent death or great bodily harm to himself or herself or another or to prevent the commission of a forcible felony.

Here is the text in the actual statute, where it is split up among the many defensible uses of force.

So there are a couple new things in there ... we're not talking self-defense any more when someone can use deadly force to (a) "prevent ... bodily harm to ... another" or (b) "prevent the commission of a forcible felony".

Let's imagine that you're part of a membership organization that has some cross-town rivals. The Bloods and the Crips, say, or the Hatfields and the McCoys. Everyone's just hanging out polishing their guns when one of them suddenly goes off. Being in fear of their lives, everyone starts shooting in order to prevent bodily harm to someone else.

But wait - it was an accident you say. There was no imminent harm. The law doesn't care if there was a real threat or not - only that the person using deadly force "reasonably believes" there is. And this person doesn't need to feel personally threatened as long as they think they are preventing a forcible felony.

Congratulations, you've just legalized full-out gang warfare.

But wait, there's more!

So, wait a second ... what's a forcible felony?

776.08 Forcible felony.—“Forcible felony” means treason; murder; manslaughter; sexual battery; carjacking; home-invasion robbery; robbery; burglary; arson; kidnapping; aggravated assault; aggravated battery; aggravated stalking; aircraft piracy; unlawful throwing, placing, or discharging of a destructive device or bomb; and any other felony which involves the use or threat of physical force or violence against any individual.

Aha - discharging a destructive device is a forcible felony. So if you reasonably believe that you need to kill someone in order to prevent them from shooting their gun (again), in our scenario, go right ahead. According to the Stand Your Ground law, you are:

immune from criminal prosecution and civil action for the use of such force.

So the OK Corral it is!


But let's start with something simpler. "Aggravated assault" - that sounds dangerous, too. What is it?

An Aggravated Assault is (emphasis mine) an Assault (a) with a deadly weapon without intent to kill or (b) with intent to commit a felony. An Assault is "an intentional, unlawful threat by word or act to do violence to the person of another, coupled with an apparent ability to do so, and doing some act which creates a well-founded fear in such other person that such violence is imminent."

Hmm.

Perhaps you may want to think twice about the next time you say something along the lines of "Come over here so I can beat you with this hammer." Especially if you actually have a hammer. It may just be the last thing you ever say.

Now why, exactly, is this necessary?

Random thought:

I've heard people say that Zimmerman can't use SYG as a defense because he initiated the confrontation. Well, here's what the law says (again, emphasis mine):

776.041 Use of force by aggressor.—The justification described in the preceding sections of this chapter is not available to a person who:

(1) Is attempting to commit, committing, or escaping after the commission of, a forcible felony; or

(2) Initially provokes the use of force against himself or herself, unless:

(a) Such force is so great that the person reasonably believes that he or she is in imminent danger of death or great bodily harm and that he or she has exhausted every reasonable means to escape such danger other than the use of force which is likely to cause death or great bodily harm to the assailant; or

(b) In good faith, the person withdraws from physical contact with the assailant and indicates clearly to the assailant that he or she desires to withdraw and terminate the use of force, but the assailant continues or resumes the use of force.

So, here's the scenario: I chase you down. I start a fight. You turn around to defend yourself and I indicate that I am withdrawing and wish to terminate my use of force. You accept and begin to walk away. I punch you again. This time when you turn around and I clearly state my intention to withdraw you decide you've had enough and take a swing at me. Bang, you're dead, and I'm free. Even better, my buddy defends me so we have witnesses with corroborating accounts.


What is most sad is the ignorance espoused online. David Gray claims SYG was established for very valid, necessary, and common sense reasons. I would like to actually hear what those reasons are. I think the reality is more like the Tampa Bay Tribune states: these laws are really being used by violent criminals. They are unnecessary and not good policy.

Speaking of the Tampa Bay Tribune, they have published an excellent exposé on Stand Your Ground and shown numerous examples of who this law is really protecting.

And here's the final report to the governor, which I have not read: http://floridastandyourground.org/

Oil Imports and Debt Crisis

People, in general, are incredibly bad at math, numbers, and scale.


Numbers, Inflation, Scale

The ancient Sumerians used a Sexagesimal numbering system, from whence most of us get our counting systems for things like clocks (60 minutes, seconds) and degrees (360° (6×60) in a circle).  This was most likely helpful in calculating fractions, but I ponder whether it represents some human limit of scale.

If I were to ask someone what they were doing 60 seconds ago, I could get an answer pretty quickly.  If I asked what they were doing 3600 seconds ago I would get a blank stare for a few seconds, until they realized that that's only an hour ago, and then I'd get an answer pretty quickly.  Somewhere between 60 and 3600 lies a comprehension boundary - our minds simply can't handle that level of scale.

Given that, what is the likelihood that we can understand a number like one million?  It's really pretty low.  As fast as you can answer, what were you doing one million seconds ago?  Is that a month?  A year?  How about one billion seconds ago?

Numbers like "million" and "billion" have no meaning because they are literally incomprehensible to us.  One million seconds was about 11 days ago; one billion seconds is about 32 years.  Notice that I didn't say 11,500 days, nor did I say 2 weeks and 1,653 weeks ... quickly: About how many years is 1,653 weeks?)

The decimal system lets out throw out these numbers without doing divisions by 7 or 60 quite easily, but that doesn't mean we truly understand them or that the issue is limited to measurements of time.  See the video link for a simple demonstration much better than I can describe.

Now, what does this have to do with inflation?  Inflation is bad for many reasons.  It's the fairest of the flat taxes - a simple devaluing of purchasing power.  Yes, it's a tax paid to bankers, but they only take 6% and pass the rest on to the Treasury (here in the U.S.)

But the insidious part is that they make all the numbers bigger, and by extension, less comprehensible.


Deficits, Interest, Money

So - the deficit this year is estimated to be 1.56 trillion dollars, a bit above our legal debt limit of 1.4 trillion dollars.  (Quickly: what does one trillion dollars look like?) Well, what is a deficit? Quite simply, that's how much more we will spend than we earned.  Or: that's how much we had borrow so far this year (the U.S. fiscal year ends in September so this isn't all from January.)

That's a lot of cash.  The debt -- how much we owe in total -- is about 5 times larger.  And each year it increases by that year's deficit.  Taking a quick look at the 2010 budget we see that interest on the debt was $150 billion.  Income taxes brought in about $1 trillion (I'm using round numbers in the most optimistic way).  That means that for each dollar you paid in income taxes, 15 cents went to pay interest.

Except that this isn't paid to the bank (well, it is, now that the Fed is buying treasuries, taking 6%, and then giving the rest back to the treasury department), but also to other nations, like China, who now hold more U.S. debt than any other nation - about 20%.  So for every dollar you pay in taxes, 3 cents goes directly to China just for holding our debt.

3 cents -- 3%.  That's the same amount that everyone was an in uproar about when Obama wanted to raise the tax rates on people earning over $250,000 a year - from 33% to 36% and from 35% to 39.6%.  And that's only on the amounts over $250,000.

If you look at the projections things start to look pretty grim.  I'm going to gloss over some stuff here, but suffice it to say that the Federal Reserve is pursuing inflationary policy in order to sustain housing prices.  Loss of housing value means foreclosures which means bank losses which means minimum capital reserve requirements failures which means loss of bank credit which means loss of dollars which means deflation which means more defaults (because money literally becomes scarce - similar in effect but different to the liquidity crisis) which eventually means the end of the world monetary system as we know it.  While that would probably not be a bad thing, it would be very painful and lots of politicians and bankers would likely lose their jobs so don't expect to see that any time soon.

So the first item of business is to tackle the deficit.  People like to call this "get spending under control".  This is said out or either ignorance or irresponsibility.  In short, the problem is that the rent is too damn high, and anyone supporting the elimination of social security or medicare will not get reëlected.  People who want to cut defense spending tend to find themselves in the same boat.


Spending, Oil, Retirement

Listen: the problems are fairly simple.  Yes, we need to cut spending.  That appears to mean that we also need to stop electing people who call themselves conservative republicans.  We're sending money to "hostile" nations in exchange for oil.  We need to end this dependence, but today we can't.  The economics of it don't work because oil is still cheap.  We cannot spend ("invest") more money in this battle as a nation -- because we don't have any.

So what do we do?  We impose a constitutionally-allowed tariff on all imported oil.

Today, oil trades around $90 per barrel. We can impose a $1 per barrel tariff on all imported oil.  Domestic producers win because they can charge more for their goods.  The higher price allows renewables to be more cost competitive.  Supply and demand dictates that usage wanes, so we can forgo all the cap-and-trade and carbon tax talk.  Efficiencies will increase, mostly because they always do, but now there's additional incentive.  And the revenue can pay down the deficit.

... or can it?

We hear a lot about imported oil, and we do import a lot: somewhere in the neighborhood of 10 million barrels a day in fact.  (Quickly: How much is 10 million barrels?  Google tells me that it converts into about 1.5 million cubic meters, but how big is 1 million cubic meters?)  Good - we can bring in $10 million a day.  That's a big number.

It's ... $3.5 billion a year.  A bit shy of our $1.5 trillion goal.  We'll have to increase that tariff from $1 per barrel to about $500 per barrel.  Do you think Americans would change anything if gas and fuel oil went from $3 per gallon to $15 per gallon?

But the best part of this news is that this is just to eliminate the deficit - to balance the budget.  Once we've done that we have to start to figure out how to pay off the other $14 trillion in existing debt.  That assumes, of course, that we don't elect any more conservative republicans and that none of the Baby Boomers retire.

But, in reality, how much additional federal spending would 76 million retirees actually add?

I can't imagine.  How about you?

Simple MPEG Trickery

In a previous life I occasionally created MPEG files from still images. I recently found some notes I'd taken and printed out. Since cloud services are all the rage these days, and because this may actually be useful to someone else (who knows?) I thought I'd save the notes here.

Disclaimer: I assume no responsibility for the horrendous formatting of blogger.com.

How to Convert Images to MPEG



Method 1

cat <file>               \
 | anytopnm              \
 | ppmtoy4m -n <n> -r    \
 | mpeg2enc -F 4 -q 1 -Q 5 -H -a 1 -2 1 -4 1 -o <file>.mpg

Explanation:

1. Convert the file to a pnm (portable anymap)
2. Convert the pnm to a y4m (YUV4MPEG2) stream.
• -n: Output a total of <n> frames.
• -r: Repeat last frame until <n>.
3. Convert the stream to MPEG
• -F 4: Format 4: Standard SVCD
• -q 1: Quantization (1..31; 1 is best quality)
• -Q 5: Quantization Reduction (0.0..5.0; 5 for sharp images)
• -H: Keep as much High Frequency information as possible.
• -a 1: Aspect ratio (1=1:1; 2=4:3; 3=16:9; 4=2.21:1)
• -4 1: Initial 4x4 subsample search quality (1..4; 1 for best quality)
• -2 1: Secondary 2x2 subsample quality (1..4; 1 for best quality)

Sadly, that's not as clean as it could be. It would seem that the quality settings do not affect the initial image. The longer the clip, the clearer the MPEG gets, until the new GOP, at which point it resets. I think it's a bug. The following produces larger images and is more complicated, but they are crystal clear. There must be an easier way to do this as well, preferably with a pipe. Transcode is just a wrapper ... I should be able to turn on debugging and find out what it is really doing under the hood.

Method 2

Step 1:

• Create the input source

cat <file>| anytopnm | ppmtoy4m -n <n>-r > <file>.y4m

Step 2:

• Transcode the Y4M to MPEG

transcode -C 3 -y ffmpeg,null -F mpeg1 -Q 5 -V -k -i <file>.y4m -o <file>

This creates <file>.m1v. MPEG1 produces slightly smaller files with no appreciable difference in quality.

Explanation

1. Convert the file to a pnm (portable anymap)
2. Convert the pnm to a y4m (YUV4MPEG2) stream.
• -n: Output a total of <n> frames.
• -r: Repeat last frame until <n>.
3. Convert the stream to MPEG
• -C 3: Anti-alias mode 3: Process Full Frame
• -y ffmpeg,null: Use ffmpeg video codec, no audio codec
• -F mpeg1: Option to tell ffmpeg to use mpeg1 encoder
• -Q 5: Encoding quality (0=fastest, 5=best)
• -V: Use YV12/I420 as internal video codec (fastest processing)
• -k: Swap red/blue (Cr/Cb) in video. Without this, people get blue faces
• -i: Input file name
• -o: Output file form (.m1v or other extension automatically appended).

Method 3

This looks like a very promising method (needs to be fleshed out)

cat <file>.y4m | ffmpeg -f yuv4mpegpipe -i - -target vcd -f mpeg1video -y <output>

or something like

ffmpeg [-f (yuv4mpegpipe)] -i <file> -target vcd [-f mpeg1video]? -y <output>

Explanation

1. Start with y4m (see above to create)
2. -target vcd seems to be the magic
3. -y is force overwrite

Method 4

This is a pretty good way to extend (loop) an MPEG. First, convert it to a YUV stream:


transcode -i <file> -o output -y yuv4mpeg -k

• The -k is the chroma conversion we need with YUV video

Next, there are two ways to proceed. mpeg2enc is slower, a little less quality, but better compression. ffmpeg is faster, better quality, and the commandline arguments make sense, but it produces bigger files.

Both of these work with yuv4mpeg streams as created in the steps above. The yuv4mpeg stream is text-based, and contains one line at the beginning of the file that need to be removed for looping. Repeat the sed command in the examples below to extend the MPEG duration.

1. mpeg2enc
• (cat <file>; sed 1d <file>) | mpeg2enc -v 1 -I 0 -f 8 -F 4 -n <n> -a 2 -o <ouput>
2. ffmpeg
• (cat <file>; sed 1d <file>) | ffmpeg -f yuv4mpegpipe -i - -target dvd -b 2000 -y <ouput>

Note: the '-b 2000' overrides the default bitrate to 2mbit. Raising the bitrate gives better quality video at the expense of larger file sizes.

Method 5

This is an all-in-one shot that I've been using with success:


cat <input>               \
 | anytopnm               \
 | pnmtoy4m -r -n 100     \
 | ffmpeg -f yuv4mpegpipe -i - -target vcd -f mpeg1video -y <output>.mpg

Converting Short MPEGs into Long MPEGs

Method 1

This is really just extracting a still frame and then extending it into an MPEG using Method 5 above:

1. Extract the first frame:
   
   transcode -i <short_mpeg> -o <framename> -y ppm -c 0-1
   
   
2. Convert it into an MPEG:
   
   cat <framename>000000.ppm            \
    | ppmtoy4m -r -n ((<seconds> * 30)) \
    | ffmpeg -f yuv4mpegpipe -i - -target vcd -f mpeg1video -y <output>.mpg
   
   

My Toaster is Amazing

So here's a simple example that shows my continuing childlike wonder .... or something.

My toaster is amazing.  Not that there's anything special about my toaster - it's a 2-slice Toastmaster.  I expect I bought it some 15 years ago for $15 or less, and most likely at Walmart.  I suppose what I find amazing about it, aside from toasters being one of the most reliable devices you'll ever buy, is that I don't really know how they work.

Now, I generally understand the electricity makes heat part, but the "brownness" selector is something of a mystery.  I always assumed that this was tied to a variable resistor that affected some timer.  Not so.

I don't really care to go shopping.  You may wonder if my brain just skipped the tracks.  A reasonable assumption but bear with me:  Because I don't care to shop, I tend to avoid it.  This means that I tend to "stock up" when I do go shopping.  Long story short, this means I have lots of bread in my freezer.

One day I thought I'd defrost some bread by popping it into the toaster.  It may be uneven and a bit crisp on the edges, but I can deal with that - or so I thought.  It turns out that the toaster toasts my bread to perfection even when it's initially frozen solid.  Clearly, this toaster is a more complex device than I had imagined.


In an assuredly unrelated topic, turkey and cheese on frozen wheat bread - 3 stars; turkey and cheese on toasted wheat bread - 11 !

A word-game riddle

A while ago I came up with a simple riddle while driving (no, I don't need a cell phone to be distracted, TYVM).  I haven't really figured out the best way to describe it, but this is the premise:

There exist sets of five words (in english) such that the only difference between the words is that one of the letters can be replaced by each of the five vowels.  As an example, the pattern "b_g" expands into "bag", "beg", "big", "bog", and "bug".  I believe all the three letter combinations are that plus "f_r", "p_p", "p_t", and "t_n".

I know of a similar set consisting of words of seven letters.  The challenge is to find it.  Is it the longest?

A secondary challenge is to simply find other sets, of any length.

Newspaper Clippings

Since living in Florida I've often wondered if the sun makes people insane.

I don't have much of a cohesive thought behind this, but I've recently been reading the local newspaper's website.  For lack of any better commentary, I'll simply quote the thoughts of those from my community.

The paper reports a story regarding raccoons becoming acclimated to humans and behaving aggressively.  Apparently they are being fed, either directly or indirectly by the people who feed the local feral cats, and as state law prevents the raccoons from being relocated, they will be trapped and chemically euthanized.  Among the 45 comments to the story are these gems:

Momekit writes:  "Did you know that Racoons also carry TB along with rabies --and they are also know for loving to eat your cats and killing your dog if they can.-They will attack you--as I have been almost run down by a pack. If they run away I say relocate them WAAAAAAAY out in no mans land. f they stand their ground --put them down- I had them try to rip into my camping tent with me and my two cats inside. I do not like them anymore -ever no not ever..WE had cats killed by them I know what I say as true along with 28 chickens -and a couple of geese..I know of a lady police woman tht had one for a pet that snapped 3 of her finger off before she could get them away from it. Some pet. Wild animals when wrown are dangerous and they will always be WILD. My kitties had a riht to life along with the chicken-of which some were down right pets--Now they are all dead becasue of racoons-Racoons do not belong around people or their pets.By the way they climb and sleep in trees eat N baby birds- squirls N kittens"

kittengirl replies:  "Momekit, Also watch out for the coyotes. They are now in our area. They have migrated to all over the Southern U.S. and Florida. They eat cats, dogs, raccoons, chickens, ducks, and children. One cornered me in my yard while I was gardening one time, and was sneaking up on me silently with a demon-like grin on his face. I barely got in my house in time. They run in large packs and you and your pets or children DO NOT STAND A CHANCE of getting away from them unhurt if the pack attacks.Even if you are in a camping tent, that will be no protection from them. They are far more dangerous than raccoons, and carry rabies.They should all be removed from our area."

I'd like to think that kittengirl was exercising her sharp wit here, but I'm not convinced that's the case.  What's more, the paper carries a ranking system of sorts, and these are the highest "Recommend"ed comments on the page.

Of course, there are also interesting threads about the feral cat problem.  Freelens has the problem solved:  "I bet if people stop feeding the feral cat colonies there would be a lot more birds laying a lot more eggs that the racoons could be eating."

Sadly, it appears that the feral cats are actually zombies.  From imqwerty: "Google "Florida feral cat attacks" for more...
VOLUSIA COUNTY, Fla. — The Volusia County Health Department issued a rabies alert for 60 days Friday, following two unprovoked attacks on humans by feral cats within the past month.
In one case, a cat wandered into a house and attacked a woman inside. In the other, passing motorists who stop to aid a cat that had been hit by a car were attacked."

Elsewhere in the news, there is a community blog site.  rodrinck3 posts the following entry titled "Toe nail fungus": "A natural colon cleansing method is sure to make you healthier, boost your energy levels.Find how toe nail fungus treatment works.for more details go to genital warts"

Indeed.

Microsoft Security Essentials (SOLVED: 0x8004ff01)

Initially published 2010/03/18; updated 2013/05/01 to edit title in a poor-man's SEO effort

So ... ignoring the whole I'm actually installing Windows thing for a moment, in order to look at some software I'm installing Windows XP (XP because apparently I bought something that came with a license for it) in a virtual machine.

So I installed XP and then upgraded to service pack 3 via Windows Update.  It seemed like the right thing to do.  Meanwhile, Windows is complaining about my not having antivirus software installed.

I find something called Microsoft Security Essentials that can be installed for free, but it keeps failing with error code 0x8004ff01.  There's a link that can be clicked for help, but no help is to be found by following it.  The site even provides a search box but it knows nothing about the error code.

I figure maybe if I installed IE8 it would pull in whatever was hip and cool, but no such luck - the same error continues to manifest itself. While searching around on the web for how to solve this I find all sort of abstruse instructions involving editing the registry and rebooting a dozen times.  Ahh, Windows, how I never missed you.


However, using the gray stuff in my head (aka: taking a SWAG) I found the the answer is relatively straightforward: Download MSI 4.5 (the Windows Installer), which can be retrieved from here:  http://www.microsoft.com/downloads/details.aspx?FamilyId=5A58B56F-60B6-4412-95B9-54D056D6F9F4&displaylang=en

Clicky-clicky and away you go.  Your Windows XP with Service Pack 3 (SP3) gets a fancy new icon in the system tray, the antivirus warning icon goes away, and your computer is apparently suitably inoculated for travel to the internets.

Flash (savior of the universe?)

This morning I looked at my Watt's Up? meter and it read 160 watts.  I was expecting it to read around 110 based on my baseline measurement the other day.  I noticed that "Shockwave Flash" was consuming some 80% of my CPU so I killed it.  Now the meter reads about 97 watts.

Because I may be taking a trip soon I went to the Travelocity website.  They have a banner advertising their own service (thanks - I never would have heard of you otherwise) that for some reason just HAD to be done in flash.  Because, you know, you can't make a 4 frame GIF or something.

My Watt's Up? meter now reads 110 watts again.

Fight global warming: disable Flash.