Tutorial: Convolutional NN

Overview

In this section, we are building a convolutional NN to demonstrate how to integrate tensorflow functionality into tflon workflow.

Architecture

Although the Breast Cancer Data set is not natural for convolution, convolution is particularly well suited for processing data such as images where there is translational symmetry. For our example, we add convolution, exponential linear unit (elu), and max pooling preceeding our dense layers.

A simple way of doing this is to leverage the implementations tensorflow. Further, this example allows us to demonstrate how to seamlessly integrate tensorflow into our framework.

Model

We’re going to add an additional convolutional-maxpooling layer to our single layered neural network replacing the normalization layer.

In short, convolution is achieved by augmenting filters of smaller but fixed dimensions across the input features. By filter we mean a fully connected neurons over a subset of the input.

As mentioned in the Welcome tutorial, each layer in the definition expressed as a nested function. Thus, seek to apply tensorflow functions to the input before passing it to the toolkit modules. The key layers we’re adding are conv2d , exponential linear unit (elu), and max_pool. We recommend reading about the functions in tensorflow if you are not familiar with convolution.

For reference, the conv2d has the following signature:

tf.nn.conv2d( input, filter=[filter_height, filter_width, in_channels, out_channels], strides, padding )

In our example, we use a 3x3 filter striding by only 1 in each direction so that we saturate the input. Futher, we arbitrarially assume the image is single channel (ie gray scale). Then the output of the layer can have multiple channels representing the network learning to recognize certain visual patterns. In our case, we arbitrarially allow for 16 features.

weight = self.get_weight('w' , [3,3,1,16])

In the past, we defined the net function which we applied to the input.

out = net(I)

Now instead, we apply the tensorflow functions to the input producing nxt before passing it to net to add the dense layers.

out = net(nxt)

Giving us:

class ConvNN(tflon.model.Model):
   def _model(self):

       width = 16
       depth = 1
       acti = tf.nn.elu
       num_pixel_targets = 1

       I = self.add_input('images', shape=[None, None, None, depth])
       T = self.add_target('solution', shape=[None, None, None, num_pixel_targets])

       Lin = 1
       Lout = width
       weight = self.get_weight('w' , [3,3,Lin,Lout])
       bias = self.get_bias('b', [Lout])

       conv = tf.nn.conv2d(I, weight, [1,1,1,1], padding='SAME')
       nonlin = tf.nn.elu(conv+bias)
       conv_out = tf.nn.max_pool(nonlin, [1,2,2,1], [1,1,1,1], padding='SAME')

       net = tflon.toolkit.Dense(5, activation=acti) \
             | tflon.toolkit.Dense(1, activation = acti, name='output')

       out=net(conv_out)
       self.add_output('solution', out)
       self.add_loss('loss', tf.reduce_sum(tf.square(T - out)))

Custom Parameters

Now we introduct a final modification that allows us to customize the model. We migrate our hardcoded arbitrary choices including width so that we can customize it when we call it.

model = ConvNN(width= 32, activation=tf.nn.relu)

To complement this change, we must inform the _model to look for parameter values using self.inject_parameters().

class ConvNN(tflon.model.Model):
   def _model(self):
       self.inject_parameters()
       num_pixel_targets = 1

       I = self.add_input('images', shape=[None, None, None, self.depth])

       T = self.add_target('solution', shape=[None, None, None, num_pixel_targets])

       Lin = 1
       Lout = self.width
       weight = self.get_weight('w' , [3,3,Lin,Lout])
       bias = self.get_bias('b', [Lout])
       conv = tf.nn.conv2d(I, weight, [1,1,1,1], padding='SAME')
       nonlin = tf.nn.elu(conv+bias)
       conv_out = tf.nn.max_pool(nonlin, [1,2,2,1], [1,1,1,1], padding='SAME')

       net = tflon.toolkit.Dense(5, activation=self.activation) \
             | tflon.toolkit.Dense(1, activation = self.activation, name='output')

       out=net(conv_out)
       self.add_output('solution', out)
       self.add_loss('loss', tf.reduce_sum(tf.square(T - out)))

   def _parameters(self):
       return {'width': 16,
               'depth':1,
               'activation': tf.nn.elu,}

AUC: 0.9991015