Bilge, Uğur; (1993) Analysis, Representation and Mapping of Neural Networks onto Parallel Hardware. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Analysis, representation and mapping of neural networks onto parallel hardware..pdf Download (11MB) | Preview

Abstract

Neural networks provide solutions to a class of pattern recognition and optimisation problems that are hard to solve with conventional techniques. Currently, most neural network applications are computationally intensive simulations on conventional sequential computers. As a solution general-purpose parallel architectures are increasingly used to speed up simulations. Hence, there is a growing need for generic strategies for simulating neural networks on parallel computers. This thesis investigates generic representation and mapping strategies for neural networks on general-purpose parallel architectures. The research comprises three main parts; an analysis of neural network models, an analysis of neural network representations, and by utilising these analyses, the representation and mapping of neural networks on parallel hardware. To understand the computational and structural properties of neural network models, and to establish a generic representation, an in-depth analysis is carried out in the form of three case studies. The Hopfield, the Self-Organising Map and the Backpropagation models are used respectively in three appropriate real-world applications; pattern recognition, data clustering and financial forecasting. Neural network representations determine parallel mapping options and the subsequent efficiency of mappings. Function-oriented, object-oriented and matrix-based representations are examined with examples, stressing their advantages and disadvantages. A matrix-based C library MATLIB and a neural network library NETLIB are put forward as generic, modular and flexible means to represent neural networks and exploit parallel, general-purpose execution environments. The mapping of neural networks onto parallel hardware is a computational optimisation problem with two main constraints: processing costs and communications costs. The Mapper's task is to optimise for a fast and efficient execution, by partitioning and distributing neural network representations across a number of parallel processors, and scheduling the parallel execution. A Computational Analysis Tool (CAT) is developed to calculate processing and communications costs, and to detect parallelism in a given MATLIB definition. An Automatic Parallel Mapper (APM), using this analysis, can partition the representation and generate parallel or pipelined code with appropriate data exchange instructions between the parallel processing modules. The Esprit II Galatea General Purpose Neural Computer (GPNC) is used as a test and implementation domain for this research work. The GPNC is a multi-processor architecture consisting of a host and a number of parallel Virtual Machines (VM), each containing a local CPU and a co-processor board, communicating and interpreting a matrix-based intermediate- level language called VML. The Galatea Mapper is designed and developed for semi-automatic mapping of VML rules to a number of parallel VMs. To assess the performance of the mapping strategies, MATLIB definitions of the three neural network models are partitioned and simulated in parallel on a network of SUN workstations. CAT projections are used to authorise data or task parallel mappings automatically. Multiple neural network applications are also simulated with two or more neural networks cooperating or competing in the solution of a problem. This thesis shows that the matrix-based abstraction captures neural network properties, and the computational cost analysis based mapping strategy is generic, flexible and can be automated. In addition, the simulation results show that: (i) the three neural network models studied in this thesis are tightly coupled algorithms, and are not suitable for pipeline or task parallelism, (ii) data parallelism for these models can increase performance only if fast communications interfaces are provided, and (iii) current distributed computer networks can be used for multiple neural network simulations, producing clear gains in performance.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item