learning simple images using if-then machines

Today, let's play with simple images from ages ago. BTW, I call them "simple images" because we don't need to translate, rotate, magnify or otherwise align (which we would with more general images), and we restrict pixel values to 0 or 1. This is to make things easier. We will of course eventually try for more general or typical images sometime in the future, but they are distinctly harder! And require many layers of if-then machines. eg, the brain has roughly 20 layers in the visual cortex.

Here are our images:

|letter: H>
#   #
#   #
#   #
#####
#   #
#   #
#   #

|noisy: H>
    #
#   #
#   #
### #
#    
#   #
#   #

|noisy: H2>
#   #
#    
# ###
#####
##  #
#   #
### #

|letter: I>
#####
  #
  #
  #
  #
  #
#####

|noisy: I>
####
  #
  
  
  #
  #
# ###

|noisy: I2>
##  #
 ###
  #
  #
  ###
####
#####

|letter: O>
######
#    #
#    #
#    #
#    #
#    #
######

Now, let's define our 3 if-then machines:

load H-I-pat-rec.sw

image |node: 1: 1> => pixels |letter: H>
image |node: 1: 2> => pixels |noisy: H>
image |node: 1: 3> => pixels |noisy: H2>
then |node: 1: *> => |letter H>

image |node: 2: 1> => pixels |letter: I>
image |node: 2: 2> => pixels |noisy: I>
image |node: 2: 3> => pixels |noisy: I2>
then |node: 2: *> => |letter I>

image |node: 3: 1> => pixels |letter: O>
then |node: 3: *> => |letter O>

the |list of images> => |node: 1: 1> + |node: 1: 2> + |node: 1: 3> + |node: 2: 1> + |node: 2: 2> + |node: 2: 3> + |node: 3: 1>
which-image |*> #=> then select[1,1] similar-input[image] image |_self>

Note that today I used "select[1,1]" instead of "drop-below[]". This just means select the first element in the superposition, and noting that similar-input[op] sorts its results.
Now, put "which-image" to use:

sa: which-image |node: 2: 3>
1.0|letter I>

sa: which-image |node: 1: 2>
1.0|letter H>

-- now, choose images randomly, and see what we get:
-- noting we are leaving in the INFO: lines, that I normally chomp out. This is so we can see which kets pick-elt has chosen.
sa: which-image pick-elt the |list of images>
INFO: ket: list of images
INFO: ket: node: 1: 2
INFO: ket: node: 1: 2
INFO: ket: node: 1: 2
1.0|letter H>

sa: which-image pick-elt the |list of images>
INFO: ket: list of images
INFO: ket: node: 3: 1
INFO: ket: node: 3: 1
INFO: ket: node: 3: 1
1.0|letter O>

sa: which-image pick-elt the |list of images>
INFO: ket: list of images
INFO: ket: node: 1: 3
INFO: ket: node: 1: 3
INFO: ket: node: 1: 3
1.0|letter H>

sa: which-image pick-elt the |list of images>
INFO: ket: list of images
INFO: ket: node: 2: 2
INFO: ket: node: 2: 2
INFO: ket: node: 2: 2
1.0|letter I>

sa: which-image pick-elt the |list of images>
INFO: ket: list of images
INFO: ket: node: 2: 3
INFO: ket: node: 2: 3
INFO: ket: node: 2: 3
1.0|letter I>

-- and so on!

Now for a couple of comments:

1) if you look under the hood, the above is quite boring! We are not making much use of similar-input[op] at all, in that we are feeding in, and detecting, exact patterns. The only interesting bit is that it is pooling the different image types. Hrmm... let's try for some noisy examples:

sa: then select[1,1] similar-input[image] absolute-noise[1] image |node: 1: 1>
0.919|letter H>

sa: then select[1,1] similar-input[image] absolute-noise[1] image |node: 2: 3>
0.907|letter I>

sa: then select[1,1] similar-input[image] absolute-noise[30] image |node: 1: 2>
0.761|letter H>

sa: then select[1,1] similar-input[image] absolute-noise[30] image |node: 3: 1>
0.738|letter O>

Heh. Even at absolute-noise[30] we are still matching at over 70%. And now we are clearly using the similarity metric, and "fuzzy matching".
2) support vector machines talk about patterns to classify as linearly separable. Well, in the world of superpositions, linearly separable doesn't really even make sense. And similar-input[op] doesn't care either, and works on any superposition type.
3) "which-image" is linear, which we can see with this:

sa: which-image the |list of images>
3|letter H> + 3|letter I> + 1.0|letter O>

4) finally, here is what we now know:

sa: dump
----------------------------------------
|context> => |context: H I pat rec>

pixels |letter: H> => |pixel: 1: 1> + |pixel: 1: 5> + |pixel: 2: 1> + |pixel: 2: 5> + |pixel: 3: 1> + |pixel: 3: 5> + |pixel: 4: 1> + |pixel: 4: 2> + |pixel: 4: 3> + |pixel: 4: 4> + |pixel: 4: 5> + |pixel: 5: 1> + |pixel: 5: 5> + |pixel: 6: 1> + |pixel: 6: 5> + |pixel: 7: 1> + |pixel: 7: 5>
dim-1 |letter: H> => |dimension: 5>
dim-2 |letter: H> => |dimension: 7>

pixels |noisy: H> => |pixel: 1: 5> + |pixel: 2: 1> + |pixel: 2: 5> + |pixel: 3: 1> + |pixel: 3: 5> + |pixel: 4: 1> + |pixel: 4: 2> + |pixel: 4: 3> + |pixel: 4: 5> + |pixel: 5: 1> + |pixel: 6: 1> + |pixel: 6: 5> + |pixel: 7: 1> + |pixel: 7: 5>
dim-1 |noisy: H> => |dimension: 5>
dim-2 |noisy: H> => |dimension: 7>

pixels |noisy: H2> => |pixel: 1: 1> + |pixel: 1: 5> + |pixel: 2: 1> + |pixel: 3: 1> + |pixel: 3: 3> + |pixel: 3: 4> + |pixel: 3: 5> + |pixel: 4: 1> + |pixel: 4: 2> + |pixel: 4: 3> + |pixel: 4: 4> + |pixel: 4: 5> + |pixel: 5: 1> + |pixel: 5: 2> + |pixel: 5: 5> + |pixel: 6: 1> + |pixel: 6: 5> + |pixel: 7: 1> + |pixel: 7: 2> + |pixel: 7: 3> + |pixel: 7: 5>
dim-1 |noisy: H2> => |dimension: 5>
dim-2 |noisy: H2> => |dimension: 7>

pixels |letter: I> => |pixel: 1: 1> + |pixel: 1: 2> + |pixel: 1: 3> + |pixel: 1: 4> + |pixel: 1: 5> + |pixel: 2: 3> + |pixel: 3: 3> + |pixel: 4: 3> + |pixel: 5: 3> + |pixel: 6: 3> + |pixel: 7: 1> + |pixel: 7: 2> + |pixel: 7: 3> + |pixel: 7: 4> + |pixel: 7: 5>
dim-1 |letter: I> => |dimension: 5>
dim-2 |letter: I> => |dimension: 7>

pixels |noisy: I> => |pixel: 1: 1> + |pixel: 1: 2> + |pixel: 1: 3> + |pixel: 1: 4> + |pixel: 2: 3> + |pixel: 5: 3> + |pixel: 6: 3> + |pixel: 7: 1> + |pixel: 7: 3> + |pixel: 7: 4> + |pixel: 7: 5>
dim-1 |noisy: I> => |dimension: 5>
dim-2 |noisy: I> => |dimension: 7>

pixels |noisy: I2> => |pixel: 1: 1> + |pixel: 1: 2> + |pixel: 1: 5> + |pixel: 2: 2> + |pixel: 2: 3> + |pixel: 2: 4> + |pixel: 3: 3> + |pixel: 4: 3> + |pixel: 5: 3> + |pixel: 5: 4> + |pixel: 5: 5> + |pixel: 6: 1> + |pixel: 6: 2> + |pixel: 6: 3> + |pixel: 6: 4> + |pixel: 7: 1> + |pixel: 7: 2> + |pixel: 7: 3> + |pixel: 7: 4> + |pixel: 7: 5>
dim-1 |noisy: I2> => |dimension: 5>
dim-2 |noisy: I2> => |dimension: 7>

pixels |letter: O> => |pixel: 1: 1> + |pixel: 1: 2> + |pixel: 1: 3> + |pixel: 1: 4> + |pixel: 1: 5> + |pixel: 1: 6> + |pixel: 2: 1> + |pixel: 2: 6> + |pixel: 3: 1> + |pixel: 3: 6> + |pixel: 4: 1> + |pixel: 4: 6> + |pixel: 5: 1> + |pixel: 5: 6> + |pixel: 6: 1> + |pixel: 6: 6> + |pixel: 7: 1> + |pixel: 7: 2> + |pixel: 7: 3> + |pixel: 7: 4> + |pixel: 7: 5> + |pixel: 7: 6>
dim-1 |letter: O> => |dimension: 6>
dim-2 |letter: O> => |dimension: 7>

image |node: 1: 1> => |pixel: 1: 1> + |pixel: 1: 5> + |pixel: 2: 1> + |pixel: 2: 5> + |pixel: 3: 1> + |pixel: 3: 5> + |pixel: 4: 1> + |pixel: 4: 2> + |pixel: 4: 3> + |pixel: 4: 4> + |pixel: 4: 5> + |pixel: 5: 1> + |pixel: 5: 5> + |pixel: 6: 1> + |pixel: 6: 5> + |pixel: 7: 1> + |pixel: 7: 5>

image |node: 1: 2> => |pixel: 1: 5> + |pixel: 2: 1> + |pixel: 2: 5> + |pixel: 3: 1> + |pixel: 3: 5> + |pixel: 4: 1> + |pixel: 4: 2> + |pixel: 4: 3> + |pixel: 4: 5> + |pixel: 5: 1> + |pixel: 6: 1> + |pixel: 6: 5> + |pixel: 7: 1> + |pixel: 7: 5>

image |node: 1: 3> => |pixel: 1: 1> + |pixel: 1: 5> + |pixel: 2: 1> + |pixel: 3: 1> + |pixel: 3: 3> + |pixel: 3: 4> + |pixel: 3: 5> + |pixel: 4: 1> + |pixel: 4: 2> + |pixel: 4: 3> + |pixel: 4: 4> + |pixel: 4: 5> + |pixel: 5: 1> + |pixel: 5: 2> + |pixel: 5: 5> + |pixel: 6: 1> + |pixel: 6: 5> + |pixel: 7: 1> + |pixel: 7: 2> + |pixel: 7: 3> + |pixel: 7: 5>

then |node: 1: *> => |letter H>

image |node: 2: 1> => |pixel: 1: 1> + |pixel: 1: 2> + |pixel: 1: 3> + |pixel: 1: 4> + |pixel: 1: 5> + |pixel: 2: 3> + |pixel: 3: 3> + |pixel: 4: 3> + |pixel: 5: 3> + |pixel: 6: 3> + |pixel: 7: 1> + |pixel: 7: 2> + |pixel: 7: 3> + |pixel: 7: 4> + |pixel: 7: 5>

image |node: 2: 2> => |pixel: 1: 1> + |pixel: 1: 2> + |pixel: 1: 3> + |pixel: 1: 4> + |pixel: 2: 3> + |pixel: 5: 3> + |pixel: 6: 3> + |pixel: 7: 1> + |pixel: 7: 3> + |pixel: 7: 4> + |pixel: 7: 5>

image |node: 2: 3> => |pixel: 1: 1> + |pixel: 1: 2> + |pixel: 1: 5> + |pixel: 2: 2> + |pixel: 2: 3> + |pixel: 2: 4> + |pixel: 3: 3> + |pixel: 4: 3> + |pixel: 5: 3> + |pixel: 5: 4> + |pixel: 5: 5> + |pixel: 6: 1> + |pixel: 6: 2> + |pixel: 6: 3> + |pixel: 6: 4> + |pixel: 7: 1> + |pixel: 7: 2> + |pixel: 7: 3> + |pixel: 7: 4> + |pixel: 7: 5>

then |node: 2: *> => |letter I>

image |node: 3: 1> => |pixel: 1: 1> + |pixel: 1: 2> + |pixel: 1: 3> + |pixel: 1: 4> + |pixel: 1: 5> + |pixel: 1: 6> + |pixel: 2: 1> + |pixel: 2: 6> + |pixel: 3: 1> + |pixel: 3: 6> + |pixel: 4: 1> + |pixel: 4: 6> + |pixel: 5: 1> + |pixel: 5: 6> + |pixel: 6: 1> + |pixel: 6: 6> + |pixel: 7: 1> + |pixel: 7: 2> + |pixel: 7: 3> + |pixel: 7: 4> + |pixel: 7: 5> + |pixel: 7: 6>

then |node: 3: *> => |letter O>

the |list of images> => |node: 1: 1> + |node: 1: 2> + |node: 1: 3> + |node: 2: 1> + |node: 2: 2> + |node: 2: 3> + |node: 3: 1>

which-image |*> #=> then select[1,1] similar-input[image] image |_self>
----------------------------------------

And that's it for this post. And I need thinking time to find more interesting if-then machine examples.