Image Based Generative Adversarial Neural Networks

Project Owner

  • Jim Cook (ICT, TechLab)

Project Credits

  • Jim Cook (ICT, TechLab)

Start Date

19th January 2018

Status

Initial pilot abandoned 25th January 2018

Follow up approach Completed 2nd February 2018

Problem Statement

One of a series of Neural Network experiments in various languages, to understand the core types of Machine learning that are becoming prevalent in industry and academia. This example leverages Theano in its first iteration and Torch in it’s second attempt. This example was developed to understand the methods by which GANs work. Primarily in image manipulation and Edge translation.

Final Brief

We took the data set film2disney (trained on every 12th frame of Disney’s The Lion King for seven epochs)

Then we fed this dataset every frame of Wintersun’s film clip, “Beyond the dark sun”.

The expectation was that the code would re-colour Wintersun’s film clip to match the colours of The Lion King, which it did and so much more. We were also able to generate edges2albumscover data sets, and others. The platform is very extensible.

Challenges & Learnings

  • C1. Python on Windows is notorious for struggling with library clashes, 32/64 bit versions and ghost libraries.
  • L1. During the migration to Pytorch, we implemented a Conda based workflow, which massively reduced the complexity of dependency and installation. It also allowed for a more focused approach to isolate the machine learning elements from the generic file operation tasks involved with this type of work.

  • C2. Disk Input/Output (IO) problems when performing operations on tens of thousands of files.

  • L2. During this experiment we migrated the BetaRay machine to solid state disks, as the platter based storage was creating a massive bottle neck.

  • C3. Tarball operations are primarily optimized for Linux environments.

  • L3. 7Zip can undertake tarball operations, but to create smart tarballs for use with Fuel you should use the only working script we were able to find on the Internet.

  • C4. CUDA machine learning interoperability is heavily version dependent.

  • L4. The use of CuDNN looks to overcome some of this limitation but introduces its own challenges. Careful documentation during workflows is the only viable way today to ensure this is repeatable.

Languages / Framework

Python, Theano, Conda, PyTorch, Cuda, Matplotlib, Scipy, OpenCV, Lasagne, MinGW, Visual Studio, Fuel, CuDNN

Links to Resources



Tags: