Generative Adversarial Network

GAN has two parts: a generator and a discriminator. In the case of numerical recognition, the generator takes in random numbers and returns an image. Then the images, either generated or given, are fed to the discriminator where the network makes a probabilistic decision on whether the images are authentic. The discriminator and generators are trained hand-in-hand: discriminator makes better decision while the generator learns by making more realistic images. Compared to VAEs, the images tend to be less blurry.

See Skymind.ai’s wiki on GAN

Liquide State Machine

https://polychord.io/neuralnetchart/img/LSM.png

LSMs are a type of spiking neural network. LSMs are sparse neural networks with activations replaced as threshold levels. The LSM models the behavior of the following: an input layer with randomly connected reservoir of neurons that add activations over the timesteps and fire after a threshold. The pattern involved with time varying inputs is called a spatio-temporal pattern (spiking neurons). Lastly, an output layer interprets the pattern and uses it for classification. LSMs are used for speech recognition or computer vision.

Extreme Learning Machine

Extreme Learning Machine

Extreme Learning Machines uses randomly chosen hidden nodes and analytically determines the output weights. This algorithm provides a good generalisation performance with extremely fast learning speeds. ELMs may seem haphazard, but it aims to resolve the key hindrances evident in FFNNs: slow gradient-based learning and iterative parameter tuning.

See this publication for more information about ELM

Echo State Network

Echo State Network

Echo State Networks are an instance of a general concept of Reservoir Computing. By incorporating the benefits of RNNs and not dealing with vanishing gradient problems, ESNs (1) assign random weights once initially, (2) feed training inputs, (3) linear regression comparison between target outputs and captured reservoir, (4) output weights are used as novel inputs. The theory behind is that random connections allow previous states to “echo” so that similar inputs will start to follow a predicted activation trajectory.

Deep Residual Network

Deep Residual Network

DRNs are small networks that aim to repeat the process to increase the depth of the network. Moreover, it also aims to give an intuitive insight in adding hidden layers. Therefore, new hidden layers learn something new by being fed inputs from previous hidden layers, essentially driving the new layer to learn something different. Fundamentally, DRNs are LSTM nodes but without gates. DRN works really well with image classification.

See Michael Dietz’s website on DRN
See Michael Dietz’s website on DRN

Kohonen Network

Kohonen Network

Kohonen Networks are used to produce a low-dimensional representation of the input. KNs are trained in an unsupervised learning manner. Every node is chosen at random from the set of training data and every node is examined to calculate which one’s weights are most like the input vector. The winning node is called the Best Matching Unit (BMU). The neighbouring node’s weights are adjusted to make them like the input vector, hence finishing training. Self organizing models are used pervasively in Image Browsing Systems, Medical Diagnosis, and Environmental Modelling.

See this website for more information about Kohonen Networks

Support Vector Machine

Support Vector Machine

SVMs are discriminative classifier defined by a separating hyperplane. Given labeled training data, the algorithm will find the optimal output to categorize the inputs.The tuning parameters include kernel, regularization, and gamma. Kernels do transformations and calculate separation lines, regularisation tells the algorithm the extent of misclassifying inputs, and the extent of how much input data is considered.

See Savan Patel’s explanation of SVM on Medium

Neural Turing Machine

Neural Turing Machine

NTMs are similar to a LSTM network, but instead of the memory stored in hidden state, the memory is stored somewhere external. The controller is the actual neural network, the memory is a matrix with memory size and item size as its axis, the read heads that “looks up” memory data, and a write head that decides where to write.

See Jonathan Hui’s explanation of NTM on Medium

The Difficulties Presented by High-Dimensional Data

A central use of the PolyChord tool is solving problems presented by high-dimensional data. High-dimensional data is present for many businesses in today’s world of BigData, where data has been gathered and there are many many variables. It is analysed in an attempt to create data models and formulas. If an accurate model is produced then the model can be used to make accurate future predictions. However, creating a model to fit high-dimensional data has been, until now, a notoriously difficult task.

This article by Sunil Sapra discusses the use of high-dimensional data and where it occurs in common commercial scenarios, and is a great explanation of the difficulties that come with fitting a model to high-dimensional data.

Here at PolyChord, we have been able to overcome a majority of these difficulties by using advanced Bayesian Data Science techniques. Unlike other tools, PolyChord takes an orthogonal approach using nested sampling to navigate thousands of dimensions and compute without compromises or approximation.

****Disclaimer: Dr. Sapra is not associated with the PolyChord organisation and has kindly given us permission to use his well written paper****

5 AI Podcasts for AI Newcomers

AI is becoming heavily integrated into everyday society, however, learning about AI can initially be difficult or confusing. Here are some helpful podcasts that you don’t need a PhD to listen to:

Linear Digressions

Hosted By: Katie Malone and Ben Jaffe

Why We Recommend It: This Podcast is extremely listener-friendly. Episodes average about 20 minutes a-piece and make advanced concepts easy to understand.

Click HERE to listen

Talking Machines


Hosted By: Neil Lawrence and Katherine Gorman

Why We Recommend It: This Podcast’s episode lengths vary and is still understandable for those not well-versed in the AI language. Talking Machines also features interviews with AI professionals, and takes questions from their viewers.

Click HERE to listen

Eye On A.I.

Hosted By: Craig S. Smith

Why We Recommend It: Episodes are on average between 20-30 minutes and feature interviews with fascinating AI innovators. These interviews help listeners to understand how AI advancements affect the world on a global scale.

Click HERE to listen

This Week in Machine Learning & AI

Hosted By: Sam Charrington

Why We Recommend It: While this Podcast’s episodes average about 50 minutes, it provides frequent updates on the newest innovations of the AI field. TWiML&AI also features exclusive and thought-provoking interviews with experts in the AI and Machine Learning community.

Click HERE to listen

Data Skeptic

Hosted By: Kyle Polich (mini-episodes are hosted by Linh Da Tran)

Why We Recommend It: This Podcast’s episodes range from about 20-45 minutes. Data Skeptic covers basic AI and Data Science concepts and how they relate to the world. Polich starts out with basic explanations and goes more in-depth as the Podcast continues.

Click HERE to listen

Those are just a few of our favorite podcasts about machine learning, big data, and artificial intelligence. If you have a favorite AI podcast not featured on this list let us know!

*Disclaimer: None of the above podcasts are affiliated with PolyChord and the views expressed by the hosts and guests are entirely their own*