#StackBounty: #python #tensorflow #conv-neural-network #generative-adversarial-network #niftynet Changing CNN to work with 3D convoluti…

Bounty: 100

I am working with code from here (paper here) that creates a GAN. I am trying to apply this to a new domain, switching from their application on MNIST to 3D brain MRI images. My issue is in the defining of the GAN itself.

For example, their code for defining the generative model (takes noise of dimension z_dim and produces an image from the MNIST distribution, so 28×28) is this, with my comments based on how I believe it works:

def generate(self, z):
    # start with noise in compact space
    assert z.shape[1] == self.z_dim

    # Fully connected layer that for some reason expands to latent * 64
    output = tflib.ops.linear.Linear('Generator.Input', self.z_dim,
                                     self.latent_dim * 64, z)
    output = tf.nn.relu(output)
    # Reshape the latent dimension into 4x4 MNIST
    output = tf.reshape(output, [-1, self.latent_dim * 4, 4, 4])

    # Reduce the latent dimension to get 8x8 MNIST
    output = tflib.ops.deconv2d.Deconv2D('Generator.2', self.latent_dim * 4,
                                         self.latent_dim * 2, 5, output)
    output = tf.nn.relu(output)  # 8 x 8
    # To be able to get 28x28 later?
    output = output[:, :, :7, :7]  # 7 x 7

    # Reduce more to get 14x14
    output = tflib.ops.deconv2d.Deconv2D('Generator.3', self.latent_dim * 2,
                                         self.latent_dim, 5, output)
    output = tf.nn.relu(output)  # 14 x 14

    output = tflib.ops.deconv2d.Deconv2D('Generator.Output',
                                         self.latent_dim, 1, 5, output)
    output = tf.nn.sigmoid(output)  # 28 x 28

    if self.gen_params is None:
        self.gen_params = tflib.params_with_name('Generator')

    return tf.reshape(output, [-1, self.x_dim])

And this is my code using niftynet convolutional layers, where z_dim and latent_dim are the same as before at 64, and I’ve added the results of the print statements:

def generate(self, z):
    assert z.shape[1] == self.z_dim

    generator_input = FullyConnectedLayer(self.latent_dim * 64,
                acti_func='relu',
                #with_bn = True,
                name='Generator.Input')
    output = generator_input(z, is_training=True)

    print(output.shape) # (?, 4096)
    #output = tflib.ops.linear.Linear('Generator.Input', self.z_dim,
    #                                 self.latent_dim * 64, z)
    #output = tf.nn.relu(output)
    output = tf.reshape(output, [-1, self.latent_dim * 4, 1, 18, 18])  # 4 x 4

    print(output.shape) # (?, 256, 1, 18, 18)

    generator_2 = DeconvolutionalLayer(self.latent_dim*2,
                    kernel_size=5,
                    stride=2,
                    acti_func='relu',
                    name='Generator.2')
    output = generator_2(output, is_training=True)
    #output = tflib.ops.deconv2d.Deconv2D('Generator.2', self.latent_dim * 4,
    #                                     self.latent_dim * 2, 5, output)
    #output = tf.nn.relu(output)  # 8 x 8
    print(output.shape) # (?, 512, 2, 36, 128)
    #output = output[:, :, :-1, :-1]  # 7 x 7

    generator_3 = DeconvolutionalLayer(self.latent_dim,
                    kernel_size=5,
                    stride=2,
                    acti_func='relu',
                    name='Generator.3')
    output = generator_3(output, is_training=True)
    #output = tflib.ops.deconv2d.Deconv2D('Generator.3', self.latent_dim * 2,
    #                                     self.latent_dim, 5, output)
    #output = tf.nn.relu(output)  # 14 x 14

    print(output.shape) # (?, 1024, 4, 72, 64)

    generator_out = DeconvolutionalLayer(1,
                    kernel_size=5,
                    stride=2,
                    acti_func='sigmoid',
                    name='Generator.Output')
    output = generator_out(output, is_training=True)

    #output = tflib.ops.deconv2d.Deconv2D('Generator.Output',
    #                                     self.latent_dim, 1, 5, output)
    #output = tf.nn.sigmoid(output)  # 28 x 28

    if self.gen_params is None:
        self.gen_params = tflib.params_with_name('Generator')

    print(output.shape) # (?, 2048, 8, 144, 1)
    print("Should be %s" % str(self.x_dim)) # [1, 19, 144, 144, 4]

    return tf.reshape(output, self.x_dim)

I am not really sure how to be able to get the 19 in there. Currently I get this error.

ValueError: Dimension size must be evenly divisible by 2359296 but is 1575936 for ‘Reshape_1’ (op: ‘Reshape’) with input shapes: [?,2048,8,144,1], [5] and with input tensors computed as partial shapes: input1 = [1,19,144,144,4].

I am also relatively new to building NNs and I also have a few questions. What is the point of the latent space when we already have a compact representation in z-space? How do I decide the size of the “output dimension” i.e. the second parameter in the layer constructor? Thank you!


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.